CN104898837A - Portable hand virtual rehabilitation experiment box based on gesture interaction and method - Google Patents
Portable hand virtual rehabilitation experiment box based on gesture interaction and method Download PDFInfo
- Publication number
- CN104898837A CN104898837A CN201510267751.4A CN201510267751A CN104898837A CN 104898837 A CN104898837 A CN 104898837A CN 201510267751 A CN201510267751 A CN 201510267751A CN 104898837 A CN104898837 A CN 104898837A
- Authority
- CN
- China
- Prior art keywords
- gesture
- control processor
- hand
- virtual
- rehabilitation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000003993 interaction Effects 0.000 title claims abstract description 16
- 238000002474 experimental method Methods 0.000 title abstract 3
- 238000012549 training Methods 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 10
- 238000009434 installation Methods 0.000 claims description 9
- 238000005286 illumination Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 claims description 7
- 230000003068 static effect Effects 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000000284 resting effect Effects 0.000 claims description 6
- 239000000463 material Substances 0.000 claims description 5
- 239000000284 extract Substances 0.000 claims description 4
- 230000001537 neural effect Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 229910000831 Steel Inorganic materials 0.000 claims description 3
- 230000037237 body shape Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 230000008676 import Effects 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 3
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 239000010959 steel Substances 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 230000004044 response Effects 0.000 claims description 2
- 230000008901 benefit Effects 0.000 abstract description 2
- 238000005192 partition Methods 0.000 abstract 1
- 229910000838 Al alloy Inorganic materials 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 235000001674 Agaricus brunnescens Nutrition 0.000 description 4
- 210000004247 hand Anatomy 0.000 description 4
- 230000009471 action Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 210000000707 wrist Anatomy 0.000 description 2
- 206010008190 Cerebrovascular accident Diseases 0.000 description 1
- 208000011092 Hand injury Diseases 0.000 description 1
- 208000006011 Stroke Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 239000011324 bead Substances 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 230000037396 body weight Effects 0.000 description 1
- 230000002490 cerebral effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003760 hair shine Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 238000003756 stirring Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 210000003857 wrist joint Anatomy 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H1/00—Apparatus for passive exercising; Vibrating apparatus; Chiropractic devices, e.g. body impacting devices, external devices for briefly extending or aligning unbroken bones
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B23/00—Exercising apparatus specially adapted for particular parts of the body
- A63B23/035—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously
- A63B23/12—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously for upper limbs or related muscles, e.g. chest, upper back or shoulder muscles
- A63B23/16—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously for upper limbs or related muscles, e.g. chest, upper back or shoulder muscles for hands or fingers
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
- A63B24/0003—Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
- A63B24/0003—Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
- A63B24/0006—Computerised comparison for qualitative assessment of motion sequences or the course of a movement
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
- A63B24/0021—Tracking a path or terminating locations
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
- A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
- A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
- A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/16—Physical interface with patient
- A61H2201/1602—Physical interface with patient kind of interface, e.g. head rest, knee support or lumbar support
- A61H2201/1635—Hand or arm, e.g. handle
- A61H2201/1638—Holding means therefor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/50—Control means thereof
- A61H2201/5058—Sensors or detectors
- A61H2201/5092—Optical sensor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2203/00—Additional characteristics concerning the patient
- A61H2203/04—Position of the patient
- A61H2203/0425—Sitting on the buttocks
- A61H2203/0431—Sitting on the buttocks in 90°/90°-position, like on a chair
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2205/00—Devices for specific parts of the body
- A61H2205/06—Arms
- A61H2205/065—Hands
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2230/00—Measuring physical parameters of the user
- A61H2230/62—Posture
- A61H2230/625—Posture used as a control parameter for the apparatus
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
- A63B24/0003—Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
- A63B24/0006—Computerised comparison for qualitative assessment of motion sequences or the course of a movement
- A63B2024/0012—Comparing movements or motion sequences with a registered reference
- A63B2024/0015—Comparing movements or motion sequences with computerised simulations of movements or motion sequences, e.g. for generating an ideal template as reference to be achieved by the user
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
- A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
- A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
- A63B2071/0647—Visualisation of executed movements
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
- A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
- A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
- A63B2071/065—Visualisation of specific exercise parameters
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B2220/00—Measuring of physical parameters relating to sporting activity
- A63B2220/10—Positions
- A63B2220/13—Relative positions
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B2220/00—Measuring of physical parameters relating to sporting activity
- A63B2220/80—Special sensors, transducers or devices therefor
- A63B2220/806—Video cameras
Landscapes
- Health & Medical Sciences (AREA)
- Physical Education & Sports Medicine (AREA)
- General Health & Medical Sciences (AREA)
- Orthopedic Medicine & Surgery (AREA)
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Epidemiology (AREA)
- Pain & Pain Management (AREA)
- Rehabilitation Therapy (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Processing Or Creating Images (AREA)
Abstract
Disclosed are a portable hand virtual rehabilitation experiment box based on gesture interaction and a method. The box comprises an upper box body and an lower body, wherein a display screen is mounted on the upper box body, a bracket is mounted on the upper box body, and a camera is mounted at the top end of the bracket; the lower box body is divided into left and right box bodies by virtue of a partition plate, and a game mark board is mounted at the opening of the lower box body; a soundbox is mounted in a front side plate of the lower box body; a control processor, a power supply module and a radiator fan are mounted in the lower box body at the right side; the input port of the control processor is connected with the game mark board, the camera, a power supply switch and a reset button through data lines; the output port of the control processor is connected with the soundbox and in addition, another output port passing through a display screen driving board is connected with the display screen through a data line; the control processor is connected with a USB interface and a network line port in an intercommunicating manner; the power supply interface is connected with the power supply module which is then connected with the control processor for power supply. The portable hand virtual rehabilitation experiment box based on gesture interaction provided by the invention has the advantages of being portable and convenient, complete in function, strong in interestingness, low in cost and the like.
Description
Technical field
The present invention relates to a kind of hand function rehabilitation equipment, especially a kind of Multifunctional rehabilitation system for hands based on gesture interaction and method.
Background technology
One of Functional tissue that staff is wanted as body weight for humans, its crisscross neural blood vessel, the muscle spreading all over full hand cause the treatment of hand injuries require high, difficulty is large, recovery period is long.Mechanical type hand rehabilitation material in the market, as finger-separating plate, spring-grip, finger rectification device etc. are tending towards ripe, but these hand rehabilitation material function singlenesses, often need patient to carry out dull repeating motion during use, patient takes exercise very uninteresting, is difficult to reach good rehabilitation efficacy.
In order to promote the interest of rehabilitation training, some researchers propose to utilize virtual reality technology to realize hand rehabilitation.Such as: the Multifunctional rehabilitation system for hands based on virtual reality of Central China University of Science and Technology's invention, having used virtual scene, take apparatus as template, only simple add recreational; Beijing Jiaotong University designs virtual reality training platform for the illness that patients with cerebral apoplexy wrist mobility is low, patient is played a role in virtual environment, become the part in virtual environment, undertaken alternately by training action and virtual environment, reach the object recovering wrist joints moving degree gradually; The method that Qingdao University of Science and Technology devises applying three-dimensional modeling realizes virtual combination, but three-dimensional modeling speed is slow, and efficiency is low, well can not reach rehabilitation efficacy.Said system is usually just simple adds game function in rehabilitation training, and only possess the recognition capability of one or several hand motion single, the sense of reality is poor, interaction is poor, and rehabilitation system does not have portability.Also have some research institution's usage data gloves of home and abroad to be combined the object reaching rehabilitation training by actual situation, but data glove is expensive, application is restricted.
Summary of the invention
The object of the invention is to provide a kind of virtual rehabilitation experimental box of the portable hand based on gesture interaction and method of cheap, interesting strong, feature richness, to improve the rehabilitation efficiency of the patient of hand movement function obstacle.
For achieving the above object, have employed following technical scheme:
The virtual rehabilitation experimental box of a kind of portable hand based on gesture interaction of the present invention, mainly comprise upper box and lower box, upper box and the hinged composition one of lower box, at upper box inner surface, display screen is installed, display screen drive plate is embedded in upper box inside, inside the folding limit of upper box, movable installation two sections of rotary type body of rod supports, install camera in cantilever tip; In lower box, lower box is divided into two casings in left and right by installed vertical dividing plate, inlays install game identification plate at the case mouth place of lower box; Installation audio amplifier is inlayed respectively in the front side board of two lower boxes in left and right; One piece of back up pad is embedded bottom the lower box of right side, control processor, power module and radiator fan are installed on the supporting plate, the side plate of right side lower box installs USB interface, cable interface, power switch, reset key, power interface being connected with control processor respectively, and on side plate, louvre is offered in corresponding radiator fan position;
The input port of described control processor is connected by data line with camera, power switch, reset key; The output port of control processor is connected with audio amplifier, separately has output port to be connected with display screen after display screen drive plate by data line; Control processor and USB interface, cable interface are connected; Power interface is connected with power module, and power module is connected with control processor again powers.
Further, install camera in cantilever tip, at a camera other installation illumination servicing unit, illumination servicing unit is made up of light activated switch and LED lamp holder, serial battery.
Further, on front side of lower box, top fixedly mounts the arm support of arc plate body shape.
Further, described game identification plate is rectangle background board, and wooden or hard plastic or steel material can be selected to make, and the case mouth size of size and lower box is supporting.Described game identification plate is divided into following four kinds of background boards:
(1) the first background board, front is black, and edge is provided with a circle white frame, the surface of black part is offered the circular hole varied in size, circular hole can assign mushroom-shaped plug pin;
(2) the second background board, front is black, and edge is provided with a circle white frame, and the surface of black part is depicted as gridiron pattern by line anyhow;
(3) the third background board, front is black, and edge is provided with a circle white frame;
(4) the 4th kinds of background boards, are stepped back up pad, each cascaded surface are drawn key or knocks key or musical string.
A kind of hand method of rehabilitation depending on the virtual rehabilitation experimental box of portable hand based on gesture interaction of the present invention, method step is as follows:
(1) by Gesture Recognition Algorithm, control processor is programmed, make control processor possess gesture recognition process function; By virtual reality fusion algorithm, control processor is programmed, the function making control processor possess virtual reality to combine alternately; By programming setting virtual game scene in control processor; Prestore rehabilitation gesture graph in control processor;
(2) select the kind of game identification plate as required, game identification plate is arranged on the case mouth place of lower box, adjustment camera height and angle, enable the complete panorama of catching game identification plate clearly of camera;
(3) staff can directly place or hold helpers be placed on game identification plate on, keep the static resting state of hand, staff video is captured by camera, import gesture video into control processor, control processor gathers a two field picture as key frame every 40ms, then key frame images is converted to gray-scale map, and extracts key frame hand profile;
The profile diagram of the static resting state of key frame hand profile and hand is carried out similarity contrast by Shape-based interpolation method for registering, if both are similar, identifies that current gesture is the rest position of hand;
If identify rest position in one's hands, target tracking algorism is then utilized to follow the trail of this gesture, synchronous extraction hand profile, and calculate dynamic gesture feature, comprise gesture principal direction, direction of motion, gesture geometric center in the position of game identification plate, speed and hand profile variations amount;
If hand profile changes, then hand profile and the rehabilitation gesture nominal contour of specifying are carried out similarity registration, if similarity exceedes defined threshold, then think that current gesture is consistent with plan rehabilitation gesture, then in conjunction with dynamic gesture feature; Utilize neural net method to judge gesture-type further, and send actuating signal controls Virtual Space object in display screen motion by control processor thus;
(4), during virtual game training, captured the image of staff, helpers and game identification plate by camera, image is converted to gray-scale map, and carries out binary conversion treatment, identify the peripheral rectangular edges angle point of game identification plate, Calculation Plane homography matrix, the profile of real-time follow-up staff or helpers, identify concrete gesture-type and hand, helpers center coordinate in the picture, in conjunction with homography matrix and hand size variation relation, determine hand or the helpers volume coordinate relative to the game identification plate lower left corner, thus determine virtual hand, helpers is relative to the position relationship of virtual world coordinate system, the corresponding object in virtual game scene is given further by this position, realize dummy object to move, give dummy object corresponding suitable response according to gesture-type, complete the virtual rehabilitation training of hand.
Compared with prior art, tool of the present invention has the following advantages:
1, adopt gesture identification and virtual reality technology, achieve the training of interesting hand initiative rehabilitation to greatest extent, improve training effectiveness, shorten recovery period;
2, system is flexible, can rehabilitation gesture enrich, simple to operate, with low cost, portability strong, do not need medical personnel or household to accompany training for a long time, is not only applicable to patient and carries out rehabilitation training in hospital, is also applicable to patient and carries out rehabilitation training in the family.
Accompanying drawing explanation
Fig. 1 is the spatial structure sketch of experimental box of the present invention.
Fig. 2 is the cut-away view of experimental box lower box of the present invention.
Fig. 3 is that the game identification plate of experimental box of the present invention is as schematic diagram during interpolation game identification plate use.
Fig. 4 is that the game identification plate of experimental box of the present invention is as matching used mushroom-shaped plug pin structural drawing during interpolation game identification plate.
Fig. 5 is the schematic diagram of the game identification plate of experimental box of the present invention when using as Trivial Pursuit Unhinged identification plate.
Schematic diagram when the game identification plate of Fig. 6 experimental box of the present invention uses as qin class game identification plate.
Fig. 7 is the gesture identification process flow diagram of the inventive method.
Fig. 8 a is interpolation class gaming example figure in embodiment.
Fig. 8 b is labyrinth class gaming example figure in embodiment.
Fig. 8 c is chess game exemplary plot in embodiment.
Fig. 8 d is hand trip class gaming example figure in embodiment.
Fig. 8 e is qin class gaming example figure in embodiment.
Fig. 9 is the system chart of experimental box of the present invention.
Drawing reference numeral: 1-upper box, 2-lower box, 3-display screen, 4-support, 5-camera, 6-play identification plate, 7-audio amplifier, 8-back up pad, 9-control processor, 10-power module, 11-radiator fan, 12-USB interface, 13-cable interface, 14-power switch, 15-reset key, 16-power interface, 17-louvre, 18-illumination servicing unit, 19-arm support.
Embodiment
Below in conjunction with accompanying drawing, the present invention will be further described:
As illustrated in fig. 1 and 2, the virtual rehabilitation experimental box of a kind of portable hand based on gesture interaction of the present invention, mainly comprise upper box 1 and lower box 2, upper box and the hinged composition one of lower box, at upper box inner surface, display screen 3 is installed, display screen drive plate is embedded in upper box inside, and inside the folding limit of upper box, movable installation two sections of rotary type body of rod supports 4, install camera 5 in cantilever tip; In lower box, lower box is divided into two casings in left and right by installed vertical dividing plate, and left side lower box forms storage area, can storing articles; Inlay at the case mouth place of lower box and game identification plate 6 is installed; Inlay respectively in the front side board of two lower boxes in left and right and audio amplifier 7 is installed; One piece of back up pad 8 is embedded bottom the lower box of right side, on the supporting plate control processor 9, power module 10 and radiator fan 11 are installed, the side plate of right side lower box installs usb 12, cable interface 13, power switch 14, reset key 15, power interface 16 being connected with control processor respectively, and on side plate, louvre 17 is offered in corresponding radiator fan position; Lower box right part side panel arranges 4 USB interface and power interface respectively, for connecting the first-class equipment of external power source, keyboard, mouse and shooting;
Camera can revolve aluminium alloy rod by two high pulling torques and form, aluminium alloy rod hollow, bottom aluminium alloy rod is screwed on upper box shell, top aluminium alloy rod and bottom aluminium alloy rod are screwed, and avoid sliding, and ensure that camera is stablized, its position can adjust at three dimensions, camera signals line is entered in upper box panel by aluminium alloy rod middle cavity, then is connected with control processor, to obtain best collection image.
When support does not use, after can folding, put into the inside of upper box; Camera is fixed on camera bracket, is connected with control processor by signal wire, for image capture; Camera side configures an illumination servicing unit 18, opens light source, guarantee system stable operation to increase illumination when low light shines; Illumination servicing unit is made up of light activated switch and LED lamp holder, serial battery.
As shown in Figure 9, described control processor adopts card form computer PCAduino, and the input port of control processor is connected by data line with camera, power switch, reset key; The output port of control processor is connected with audio amplifier, separately has output port to be connected with display screen after display screen drive plate by data line; Control processor and USB interface, cable interface are connected; Power interface is connected with power module, and power module is connected with control processor again powers.
Further, on front side of lower box, the arm support 19 of top fixed installation arc plate body shape, alleviates ancon pressure for patient in rehabilitation training.
Further, the rectangle background board of described game identification plate to be size the be interchangeable multiple same specification of 42cm*32cm, wooden or hard plastic or steel material can be selected to make, and the case mouth size of size and lower box is supporting.Described game identification plate is divided into following four kinds of background boards:
(1) as shown in Figure 3, the first background board is the auxiliary sign plate of interpolation class game, front is black, edge is provided with the wide white frame of a circle 5mm, the surface of black part is offered the circular hole varied in size, circular hole can assign mushroom-shaped plug pin, the corresponding pattern of game interpolation that patient uses mushroom-shaped plug pin to produce according to virtual game module on game identification plate.As shown in Figure 4, the described mushroom portion of nailing on is that I shape just takes formula design, and bottom is the cylindrical of undergauge gradually from top to bottom, and cylindrical diameter is corresponding with the diameter of circular hole on identification plate.
(2) as shown in Figure 5, the second background board, front is black, and edge is provided with the wide white frame of a circle 5mm, and the surface of black part is depicted as gridiron pattern by line anyhow, and gridiron pattern size is 1cm*1cm, uses as labyrinth figure or chessboard;
(3) the third background board, front is black, and edge is provided with the wide white frame of a circle 5mm, mainly works as background board for virtual game, captures chess piece play for patient, the training crawl of hand, movement, accurately the action such as to put off.
(4) as shown in Figure 6, the 4th kind of background board, be three layers of step shape back up pad, bottom ladder is piano key, and middle part ladder is for knocking key, and first ladder is string, main training testing finger flexibility and harmony.This type of back up pad needs actual situation combination just can complete, and corresponding screen has the qin identical with in back up pad, and the corresponding corresponding sound of respective keys, presses, hits or stir corresponding button/string and then send corresponding sound, form different music.
A kind of hand method of rehabilitation depending on the virtual rehabilitation experimental box of portable hand based on gesture interaction of the present invention, method step is as follows:
As shown in Figure 7, (1) is programmed to control processor by Gesture Recognition Algorithm, makes control processor possess gesture recognition process function; By virtual reality fusion algorithm, control processor is programmed, the function making control processor possess virtual reality to combine alternately; By programming setting virtual game scene in control processor; Prestore rehabilitation gesture graph in control processor; Described rehabilitation gesture comprises pendency, lifts, touches, pushes, impacts, dynamic operation (moving up and down), spherical grasp, spherical finger tip is held, column grasps, hook draws, two finger tip pinches, many finger tip pinches, side is pinched, wrist is bent, stretch, flexing and rotation;
(2) select the kind of game identification plate as required, game identification plate is arranged on the case mouth place of lower box, adjustment camera height and angle, enable the complete panorama of catching game identification plate clearly of camera;
(3) staff can directly place or hold helpers be placed on game identification plate on, keep the static resting state of hand, staff video is captured by camera, import gesture video into control processor, control processor gathers a two field picture as key frame every 40ms, then key frame images is converted to gray-scale map, and extracts key frame hand profile; Helpers comprises ball, mushroom nail, cylinder, chess piece, building blocks and knocks hammer etc.
The profile diagram of the static resting state of key frame hand profile and hand is carried out similarity contrast by Shape-based interpolation method for registering, if both are similar, identifies that current gesture is the rest position of hand;
If identify rest position in one's hands, then utilize Camshift algorithm to follow the trail of this gesture, synchronous extraction hand profile, and calculate dynamic gesture feature, comprise gesture motion direction, gesture geometric center in the position of game identification plate, speed, and hand profile variations amount;
If t
imoment target gesture center of gravity is (x
i, y
i), t
jmoment target gesture center of gravity (x
j, y
j), then:
Velocity magnitude
Direction of motion
If hand profile changes, then hand profile and the rehabilitation gesture nominal contour of specifying are carried out similarity registration, if similarity exceedes defined threshold, then think that current gesture is consistent with plan rehabilitation gesture, then in conjunction with dynamic gesture feature; Neural net method is utilized to judge gesture-type further, by the velocity magnitude of preceding aim gesture, direction, hand-type feature as input signal, carry out the training of BP neural network learning, prediction gesture-type, and send actuating signal controls Virtual Space object in display screen motion by control processor thus;
(4) during virtual game training, the image of staff, helpers and game identification plate is captured by camera, each two field picture is converted to gray-scale map by control processor, and extracts identification plate rectangle or circular contour according to Threshold Segmentation Algorithm, and the profile of hand or utility appliance; Identify the peripheral rectangular edges angle point of game identification plate, and according to rectangle size and angular coordinate Calculation Plane homography matrix; The profile of real-time follow-up staff or helpers, identify concrete gesture-type and hand, helpers center coordinate in the picture, if all there is not significant change in hand position and direction within the 5s time, think static gesture, processor starts to carry out gesture identification, and has determined whether virtual game task according to gesture identification result; If there is significant change in hand position or direction in 5s, think dynamic gesture, while identification hand-type, in conjunction with homography matrix and hand size variation relation, determine hand or the helpers volume coordinate relative to the game identification plate lower left corner, thus determine virtual hand, the helpers position relationship relative to virtual world coordinate system, give virtual hand or the virtual rehabilitation equipment of hand by the coordinate calculated simultaneously, when virtual hand or the virtual rehabilitation equipment of hand move to the position that virtual game specifies, complete virtual task.
Such as: interpolation class gaming example, as shown in Figure 8 a, the pattern that patient's complexity of first specifying according to virtual game is different, mobile mushroom nail moves on identification plate, then processor identification mushroom nail position, until patient's interpolation goes out the pattern that virtual game is specified, task completes.
Labyrinth class gaming example, as shown in Figure 8 b, patient first according to the labyrinth scene in virtual scene on identification plate according to appointment hand-type (as spherical grasp) mobile bead, then processor identification hand-type and little ball position, until patient walks out labyrinth, task completes.
Chess game example, as shown in Figure 8 c, select quintet game, the chess piece that first patient occurs according to virtual chessboard places chess piece according to appointment hand-type (as two refer to that folder is pinched), then processor identification hand-type and chess piece position, and whether real-time judge patient wins the game.
Hand trip class gaming example, as shown in figure 8d, first patient according to game requirement, produces different rehabilitation hand-types, then processor identification hand-type, if gesture is correct, task completes.
Qin class gaming example, as figure 8 e shows, patient, first according to game presentation, dials with finger or presses key, then processor identification hand-type and finger position, if gesture and position are all correct, task completes.
Above-described embodiment is only be described the preferred embodiment of the present invention; not scope of the present invention is limited; under not departing from the present invention and designing the prerequisite of spirit; the various distortion that those of ordinary skill in the art make technical scheme of the present invention and improvement, all should fall in protection domain that claims of the present invention determines.
Claims (6)
1. the virtual rehabilitation experimental box of the portable hand based on gesture interaction, mainly comprise upper box and lower box, it is characterized in that: upper box and the hinged composition one of lower box, at upper box inner surface, display screen is installed, display screen drive plate is embedded in upper box inside, inside the folding limit of upper box, movable installation two sections of rotary type body of rod supports, install camera in cantilever tip; In lower box, lower box is divided into two casings in left and right by installed vertical dividing plate, inlays install game identification plate at the case mouth place of lower box; Installation audio amplifier is inlayed respectively in the front side board of two lower boxes in left and right; One piece of back up pad is embedded bottom the lower box of right side, control processor, power module and radiator fan are installed on the supporting plate, the side plate of right side lower box installs USB interface, cable interface, power switch, reset key, power interface being connected with control processor respectively, and on side plate, louvre is offered in corresponding radiator fan position;
The input port of described control processor is connected by data line with camera, power switch, reset key; The output port of control processor is connected with audio amplifier, separately has output port to be connected with display screen after display screen drive plate by data line; Control processor and USB interface, cable interface are connected; Power interface is connected with power module, and power module is connected with control processor again powers.
2. the virtual rehabilitation experimental box of a kind of portable hand based on gesture interaction according to claim 1, it is characterized in that: in cantilever tip, camera is installed, at a camera other installation illumination servicing unit, illumination servicing unit is made up of light activated switch and LED lamp holder, serial battery.
3. the virtual rehabilitation experimental box of a kind of portable hand based on gesture interaction according to claim 1, is characterized in that: the arm support of top fixed installation arc plate body shape on front side of lower box.
4. the virtual rehabilitation experimental box of a kind of portable hand based on gesture interaction according to claim 1, it is characterized in that: described game identification plate is rectangle background board, wooden or hard plastic or steel material can be selected to make, and the case mouth size of size and lower box is supporting.
5. the virtual rehabilitation experimental box of a kind of portable hand based on gesture interaction according to claim 1 or 4, it is characterized in that, described game identification plate is divided into following four kinds of background boards:
(1) the first background board, front is black, and edge is provided with a circle white frame, the surface of black part is offered the circular hole varied in size, circular hole can assign mushroom-shaped plug pin;
(2) the second background board, front is black, and edge is provided with a circle white frame, and the surface of black part is depicted as gridiron pattern by line anyhow;
(3) the third background board, front is black, and edge is provided with a circle white frame;
(4) the 4th kinds of background boards, are stepped back up pad, each cascaded surface are drawn key or knocks key or musical string.
6. rely on a hand method of rehabilitation for claim 1, it is characterized in that, described method step is as follows:
(1) by Gesture Recognition Algorithm, control processor is programmed, make control processor possess gesture recognition process function; By virtual reality fusion algorithm, control processor is programmed, the function making control processor possess virtual reality to combine alternately; By programming setting virtual game scene in control processor; Prestore rehabilitation gesture graph in control processor;
(2) select the kind of game identification plate as required, game identification plate is arranged on the case mouth place of lower box, adjustment camera height and angle, enable the complete panorama of catching game identification plate clearly of camera;
(3) staff can directly place or hold helpers be placed on game identification plate on, keep the static resting state of hand, staff video is captured by camera, import gesture video into control processor, control processor gathers a two field picture as key frame every 40ms, then key frame images is converted to gray-scale map, and extracts key frame hand profile;
The profile diagram of the static resting state of key frame hand profile and hand is carried out similarity contrast by Shape-based interpolation method for registering, if both are similar, identifies that current gesture is the rest position of hand;
If identify rest position in one's hands, target tracking algorism is then utilized to follow the trail of this gesture, synchronous extraction hand profile, and calculate dynamic gesture feature, comprise gesture principal direction, direction of motion, gesture geometric center in the position of game identification plate, speed and hand profile variations amount;
If hand profile changes, then hand profile and the rehabilitation gesture nominal contour of specifying are carried out similarity registration, if similarity exceedes defined threshold, then think that current gesture is consistent with plan rehabilitation gesture, then in conjunction with dynamic gesture feature; Utilize neural net method to judge gesture-type further, and send actuating signal controls Virtual Space object in display screen motion by control processor thus;
(4), during virtual game training, captured the image of staff, helpers and game identification plate by camera, image is converted to gray-scale map, and carries out binary conversion treatment, identify the peripheral rectangular edges angle point of game identification plate, Calculation Plane homography matrix, the profile of real-time follow-up staff or helpers, identify concrete gesture-type and hand, helpers center coordinate in the picture, in conjunction with homography matrix and hand size variation relation, determine hand or the helpers volume coordinate relative to the game identification plate lower left corner, thus determine virtual hand, helpers is relative to the position relationship of virtual world coordinate system, the corresponding object in virtual game scene is given further by this position, realize dummy object to move, give dummy object corresponding suitable response according to gesture-type, complete the virtual rehabilitation training of hand.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510267751.4A CN104898837A (en) | 2015-05-22 | 2015-05-22 | Portable hand virtual rehabilitation experiment box based on gesture interaction and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510267751.4A CN104898837A (en) | 2015-05-22 | 2015-05-22 | Portable hand virtual rehabilitation experiment box based on gesture interaction and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104898837A true CN104898837A (en) | 2015-09-09 |
Family
ID=54031541
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510267751.4A Pending CN104898837A (en) | 2015-05-22 | 2015-05-22 | Portable hand virtual rehabilitation experiment box based on gesture interaction and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104898837A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106028160A (en) * | 2016-06-03 | 2016-10-12 | 腾讯科技（深圳）有限公司 | Image data processing method and device |
| CN106178427A (en) * | 2016-08-29 | 2016-12-07 | 常州市钱璟康复股份有限公司 | A kind of hands functional training based on the mutual virtual reality of many people and assessment system |
| CN106295612A (en) * | 2016-08-23 | 2017-01-04 | 广西科技大学 | A kind of visual monitoring method of finger motion in hand rehabilitation training |
| CN107295234A (en) * | 2017-08-03 | 2017-10-24 | 刘勇 | A kind of equal proportion filming apparatus |
| CN107485826A (en) * | 2017-07-28 | 2017-12-19 | 长春理工大学 | Hand rehabilitation training system based on virtual reality technology |
| CN109559593A (en) * | 2018-12-29 | 2019-04-02 | 深圳市掌网科技股份有限公司 | A kind of piano training system and method based on mixed reality |
| CN110478860A (en) * | 2019-09-02 | 2019-11-22 | 燕山大学 | The virtual rehabilitation system of hand function obstacle based on hand object natural interaction |
| WO2020016033A1 (en) * | 2018-07-18 | 2020-01-23 | Koninklijke Philips N. V. | A rehabilitation device and a method of monitoring hand movement |
| EP3626221A1 (en) * | 2018-09-20 | 2020-03-25 | Koninklijke Philips N.V. | A rehabilitation device and a method of monitoring hand movement |
| CN112790948A (en) * | 2021-01-27 | 2021-05-14 | 常州市钱璟康复股份有限公司 | Upper limb function combination training device and training method |
| CN113117295A (en) * | 2021-04-22 | 2021-07-16 | 合肥工业大学 | Brain cognitive rehabilitation robot system based on hand recognition and training method |
| CN113491621A (en) * | 2021-07-27 | 2021-10-12 | 河北省人民医院 | Hand rehabilitation training device for hemiplegic patient |
| CN113814986A (en) * | 2021-11-23 | 2021-12-21 | 广东隆崎机器人有限公司 | Method and system for controlling SCARA robot based on machine vision |
| CN114504777A (en) * | 2022-04-19 | 2022-05-17 | 西南石油大学 | Exercise intensity calculation system and method based on neural network and fuzzy comprehensive evaluation |
| CN114796991A (en) * | 2022-04-19 | 2022-07-29 | 重庆市江津区中心医院 | Exercise device of prevention low limbs thrombus |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101021948A (en) * | 2007-03-14 | 2007-08-22 | 华南理工大学 | Automatic identifying device and method for joint in human body symmetric motion image |
| US20100023857A1 (en) * | 2008-07-23 | 2010-01-28 | General Electric Company | Intelligent user interface using on-screen force feedback and method of use |
| CN202355789U (en) * | 2011-10-18 | 2012-08-01 | 昆山市工业技术研究院有限责任公司 | Multifunctional hand rehabilitation training device |
-
2015
- 2015-05-22 CN CN201510267751.4A patent/CN104898837A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101021948A (en) * | 2007-03-14 | 2007-08-22 | 华南理工大学 | Automatic identifying device and method for joint in human body symmetric motion image |
| US20100023857A1 (en) * | 2008-07-23 | 2010-01-28 | General Electric Company | Intelligent user interface using on-screen force feedback and method of use |
| CN202355789U (en) * | 2011-10-18 | 2012-08-01 | 昆山市工业技术研究院有限责任公司 | Multifunctional hand rehabilitation training device |
Non-Patent Citations (1)
| Title |
|---|
| 丁伟利等: "基于视觉交互的手部运动虚拟康复系统", 《系统仿真学报》 * |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106028160A (en) * | 2016-06-03 | 2016-10-12 | 腾讯科技（深圳）有限公司 | Image data processing method and device |
| CN106295612A (en) * | 2016-08-23 | 2017-01-04 | 广西科技大学 | A kind of visual monitoring method of finger motion in hand rehabilitation training |
| CN106178427A (en) * | 2016-08-29 | 2016-12-07 | 常州市钱璟康复股份有限公司 | A kind of hands functional training based on the mutual virtual reality of many people and assessment system |
| CN107485826A (en) * | 2017-07-28 | 2017-12-19 | 长春理工大学 | Hand rehabilitation training system based on virtual reality technology |
| CN107295234B (en) * | 2017-08-03 | 2023-02-03 | 刘勇 | Equal-proportion shooting device |
| CN107295234A (en) * | 2017-08-03 | 2017-10-24 | 刘勇 | A kind of equal proportion filming apparatus |
| WO2020016033A1 (en) * | 2018-07-18 | 2020-01-23 | Koninklijke Philips N. V. | A rehabilitation device and a method of monitoring hand movement |
| CN112423718A (en) * | 2018-07-18 | 2021-02-26 | 皇家飞利浦有限公司 | Rehabilitation device and method for monitoring hand movement |
| EP3626221A1 (en) * | 2018-09-20 | 2020-03-25 | Koninklijke Philips N.V. | A rehabilitation device and a method of monitoring hand movement |
| CN109559593A (en) * | 2018-12-29 | 2019-04-02 | 深圳市掌网科技股份有限公司 | A kind of piano training system and method based on mixed reality |
| CN109559593B (en) * | 2018-12-29 | 2021-06-04 | 深圳市掌网科技股份有限公司 | Piano training system and method based on mixed reality |
| CN110478860A (en) * | 2019-09-02 | 2019-11-22 | 燕山大学 | The virtual rehabilitation system of hand function obstacle based on hand object natural interaction |
| CN112790948A (en) * | 2021-01-27 | 2021-05-14 | 常州市钱璟康复股份有限公司 | Upper limb function combination training device and training method |
| CN113117295B (en) * | 2021-04-22 | 2022-07-15 | 合肥工业大学 | Brain cognitive rehabilitation robot system based on hand recognition and training method |
| CN113117295A (en) * | 2021-04-22 | 2021-07-16 | 合肥工业大学 | Brain cognitive rehabilitation robot system based on hand recognition and training method |
| CN113491621A (en) * | 2021-07-27 | 2021-10-12 | 河北省人民医院 | Hand rehabilitation training device for hemiplegic patient |
| CN113491621B (en) * | 2021-07-27 | 2022-11-01 | 河北省人民医院 | Hand rehabilitation training device for hemiplegic patient |
| CN113814986A (en) * | 2021-11-23 | 2021-12-21 | 广东隆崎机器人有限公司 | Method and system for controlling SCARA robot based on machine vision |
| CN114504777A (en) * | 2022-04-19 | 2022-05-17 | 西南石油大学 | Exercise intensity calculation system and method based on neural network and fuzzy comprehensive evaluation |
| CN114504777B (en) * | 2022-04-19 | 2022-07-15 | 西南石油大学 | Exercise intensity calculation system and method based on neural network and fuzzy comprehensive evaluation |
| CN114796991A (en) * | 2022-04-19 | 2022-07-29 | 重庆市江津区中心医院 | Exercise device of prevention low limbs thrombus |
| CN114796991B (en) * | 2022-04-19 | 2023-08-08 | 重庆市江津区中心医院 | Exercise device for preventing lower limb thrombosis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104898837A (en) | Portable hand virtual rehabilitation experiment box based on gesture interaction and method | |
| CN109350923B (en) | Upper limb rehabilitation training system based on VR and multi-position sensors | |
| Araujo et al. | Snake charmer: Physically enabling virtual objects | |
| CN103105929B (en) | Virtural agriculture garden Interactive Design and experiential method and system | |
| CN102908772B (en) | Upper limb rehabilitation training system by using augmented reality technology | |
| CN109192272A (en) | Based on the Leap Motion healing hand function training system combined with VR and its implementation | |
| CN107340871A (en) | The devices and methods therefor and purposes of integrated gesture identification and ultrasonic wave touch feedback | |
| CN206577357U (en) | A kind of Chinese chess Chinese-chess robot | |
| CN104932698A (en) | Handheld interactive device and projection interactive method thereof | |
| CN202150897U (en) | Body feeling control game television set | |
| CN109421052A (en) | A kind of quintet game Chinese-chess robot based on artificial intelligence | |
| CN102270276A (en) | Caloric burn determination from body movement | |
| CN106020478A (en) | Intelligent terminal manipulation method, intelligent terminal manipulation apparatus and intelligent terminal | |
| CN206639168U (en) | A kind of transformer station's three-dimension visible sysem based on VR technologies | |
| CN206209206U (en) | 3D glasses with fixed sample point and the virtual reality system of Portable multi-person interaction | |
| CN106293099A (en) | Gesture identification method and system | |
| CN110743160B (en) | Real-time pace tracking system and pace generation method based on somatosensory capture device | |
| TW201409279A (en) | Interactive augmented reality system and portable communication device and interaction method thereof | |
| US20080071481A1 (en) | Motion tracking apparatus and technique | |
| CN105630134A (en) | Operation event identification method and apparatus | |
| CN201871223U (en) | Virtual chessboard utilizing projector and image collecting device | |
| CN202548757U (en) | Virtual page turning device | |
| CN105204613B (en) | A kind of information processing method and wearable device | |
| CN106020468A (en) | Glove controlled augmented reality system | |
| CN115454256A (en) | Digital oath word tombstone device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150909 |