WO2017109205A1 - System and method for hand rehabilitation - Google Patents

System and method for hand rehabilitation Download PDF

Info

Publication number
WO2017109205A1
WO2017109205A1 PCT/EP2016/082609 EP2016082609W WO2017109205A1 WO 2017109205 A1 WO2017109205 A1 WO 2017109205A1 EP 2016082609 W EP2016082609 W EP 2016082609W WO 2017109205 A1 WO2017109205 A1 WO 2017109205A1
Authority
WO
WIPO (PCT)
Prior art keywords
finger sleeve
markers
finger
marker
loo
Prior art date
Application number
PCT/EP2016/082609
Other languages
French (fr)
Inventor
Sheng Jin
Jin Wang
Cheng Chen
Original Assignee
Koninklijke Philips N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips N.V. filed Critical Koninklijke Philips N.V.
Publication of WO2017109205A1 publication Critical patent/WO2017109205A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802Sensor mounted on worn items
    • A61B5/6804Garments; Clothes
    • A61B5/6806Gloves
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1124Determining motor skills
    • A61B5/1125Grasping motions of hands
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1126Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
    • A61B5/1127Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using markers
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813Specially adapted to be attached to a specific body part
    • A61B5/6824Arm or wrist
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813Specially adapted to be attached to a specific body part
    • A61B5/6825Hand
    • A61B5/6826Finger
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304Detection arrangements using opto-electronic means
    • G06F3/0308Detection arrangements using opto-electronic means comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813Specially adapted to be attached to a specific body part
    • A61B5/6825Hand

Definitions

  • the invention relates to the field of rehabilitation, in particular to a system, a device and a method for hand rehabilitation.
  • a camera-marker based rehabilitation system is a common solution, which is used by these patients to do rehabilitation exercises by themselves according to guidance provided by doctors/therapists.
  • Hand related exercises are typical exercises for these patients to recover or partially recover their hand functions.
  • a hand rehabilitation device helps patients to do movement therapy, which is well accepted for motor-relearning. Such devices generally guide, monitor and measure how patients conduct exercises, give feedback on exercise quality and motivate patients to comply with the training plan.
  • a glove-camera based solution is widely used for hand rehabilitation.
  • the camera detects markers on the glove to recognize hand gesture normally.
  • a fully closed glove with separate fingers is a common solution. However, it is very hard for patients to wear this kind of gloves, due to spasm and hand incapacity, for example, for post-stroke patients. For severe patients, the fully closed gloves don't work at all.
  • the patent CN104887238 discloses a hand positioning system for rehabilitation, using multiple cameras in combination with LEDs placed on all fingertips and joints of a hand of a person. The respective positions of the hand are determined accordingly. Due to incapacity of the patients' hands, it is difficult for the patients to attach the LEDs to the hand by the patients themselves. Moreover, wearing LEDs on all these fingertips and joints of the hand is troublesome for patients.
  • an object of the invention to provide a system, a device and a method which provide a more user- friendly hand rehabilitation solution.
  • the object is achieved by an device 100 for hand rehabilitation comprising a pluralities of finger sleeve markers 101, wherein each finger sleeve marker 101 comprises a layer of flexible material in a shape corresponding to part of a finger, the finger sleeve marker 101 is arranged to be in direct contact with one of a plurality of fingertips of fingers of a hand of a user and the flexible material is detectable by camera, wherein the device 100 does not fully cover the palm.
  • the device provides the possibility that the plurality of finger sleeve markers are disconnected from each other, where the device does not cover the palm of the hand.
  • the finger sleeve markers are enabled to be put on patients' fingertips or taken off from patients' fingertips separately, which is very convenient and user friendly.
  • the device overcomes the requirement for the patients to forcibly stretch their hands during putting on or taking off the whole glove, which is challenging for patients incapable of doing that due to their hand problem.
  • the patient needs first to get the whole hand into glove through the opening and then to aim their fingers to at the inlet for each finger to get into the finger sheaths.
  • the glove Since the glove is a sealed one piece apparatus with only one opening, where the finger sheaths and palm section are connected in a fixed way, it requires a high agility of the finger movements to successfully probing into the finger sheaths. This will be painful and time consuming for the hand rehabilitation patient, who is planning to take the rehabilitation exercise to restore or improve their control over the hand including the fingers. Furthermore, the glove is a flat piece with limited flexibility. It will be troublesome to get the hand through the opening at the first step for the patients who have difficulty with hand control.
  • the device provides a solution which does not require complex steps to wear markers or cables on the hands. The lack of the palm section make the finger sleeve markers separate and movable relative to each other. The device is also not a sealed piece anymore. The finger sleeve markers can be put onto the target position of the hand one by one in a more user friendly way, which bothers the patients least. In a sense, the system according to the invention provides the patients with a more user-friendly hand rehabilitation solution.
  • the plurality of finger sleeve markers are intended to surround the fingertips.
  • the plurality of finger sleeve markers are detectable on the basis of a camera image.
  • a finger sleeve marker thus is visible at any position around the fingertip.
  • the device allows the finger sleeve markers to be visible to a certain camera when the fingertips assume all kinds of postures in three-dimensional space. Therefore, the determining of the positions of the fingertip sleeves requires only one camera.
  • each of the plurality of finger sleeve markers comprises a pattern, said patterns of the plurality of finger sleeve markers being different.
  • the patterns of the plurality of finger sleeve markers comprise color patterns, graphic patterns, or shape patterns.
  • Using a color pattern-based marker or a graphic pattern-based marker allows more easily to identify the position of a finger sleeve marker from the background on the basis of acquired images.
  • the color pattern-based marker and the graphic pattern-based marker are also practical finger sleeve marker solutions.
  • the device further comprises a connector configured to connect the plurality of finger sleeve markers to each other, the connector being configured to leave finger joints uncovered.
  • the connector connects the plurality of finger sleeve markers together, thereby avoiding losing a finger sleeve marker during the motion of the rehabilitation exercise. This feature allows providing a more convenient solution for a user.
  • the device further comprises a wrist band marker intended to surround a wrist of the user, the wrist band marker being detectable on the basis of a camera image.
  • the starting points may be the same and the end points may also be the same.
  • the paths of the movements are different. Only using the plurality of finger sleeve markers, the two specific movements of the finger cannot be distinguished. Using, together with the plurality of finger sleeve markers, the wrist band marker, the specific movements of the finger can be more accurately distinguished. Meanwhile, wearing the wrist band marker remains equally user friendly.
  • each of the plurality of finger sleeve markers comprises a pattern
  • the wrist band marker comprises a different pattern than the patterns of the plurality of finger sleeve markers.
  • the wrist band marker has a different pattern than the patterns of the plurality of finger sleeve markers. This feature allows distinguishing it from the plurality of finger sleeve makers more easily in terms of less complexity and higher reliability.
  • a finger sleeve marker of the plurality of finger sleeve markers is configured to cover a fingertip region, the fingertip region is smaller than a given region, extending from a fingertip to a corresponding distal phalanx.
  • the object is achieved by a system for hand rehabilitation, comprising a device and a detection subsystem.
  • the device comprises a plurality of finger sleeve markers intended to surround a plurality of fingertips of fingers of a hand of a user, the plurality of finger sleeve markers being detectable on the basis of a camera image.
  • the detection subsystem comprises a camera configured to acquire a sequence of images of the apparatus.
  • the detection subsystem also comprises a processor configured to receive the sequence of images acquired from the camera and derive position information of the plurality of finger sleeve markers on the basis of the information of the plurality of finger sleeve markers contained in the sequence of images.
  • the device used in the system according to the invention employs the plurality of finger sleeve markers intended to surround the fingertips of the user. Fingertip movement tracking is crucial for hand rehabilitation as compared to movements of other parts, for example, hand joints. Therefore, by using a plurality of finger sleeve markers placed at fingertip positions, the device enables tracking the crucial movement for hand rehabilitation.
  • the detection subsystem further comprises a user interface configured to receive position information from the processor and present the position information.
  • the user interface allows interactions with a user when using the system for hand rehabilitation.
  • the user interface is further configured to receive instructions on intended use of the plurality of finger sleeve markers. This feature allows setting the parameters of the plurality of finger sleeve markers according to the intended use of the plurality of finger sleeve markers by a user.
  • the user interface is further configured to present instructions on the predetermined use of the plurality of finger sleeve markers. This feature allows guiding the user to put on the plurality of finger sleeves according to predetermined instructions.
  • the detection subsystem comprises a single camera
  • the processor is arranged to derive position information of the plurality of finger sleeve markers on the basis of the acquired sequence of images, and the fingertip movement is in three dimensional space. This features allows the three-dimensional movement of the fingertips to be captured by using a single camera.
  • a device for use in the system comprises a plurality of finger sleeve markers, each of the plurality of finger sleeve markers being intended to surround one of a plurality of fingertips of fingers of a hand of a user and being detectable on the basis of a camera image.
  • a method for hand rehabilitation comprises a step of acquiring a sequence of images of an apparatus.
  • the method also comprises a step of deriving position information of the plurality of finger sleeve markers on the basis of the sequence of images acquired in the step of acquiring.
  • An embodiment of the method according to the invention further comprises a step of receiving the position information.
  • the method further comprises a step of presenting the position information.
  • Fig.l schematically depicts an embodiment of the system according to the present invention
  • Fig.2 illustrates an example of the device according to embodiments of the present invention
  • Figs.3A, 3B illustrate examples of finger sleeve markers according to embodiments of the present invention
  • Figs.4A, 4B and 4C illustrate three examples of different finger sleeve marker device according to embodiments of the present invention
  • Fig.5 illustrates examples of graphic patterns according to embodiments of the present invention
  • Fig.6 illustrates examples of shape patterns according to embodiments of the present invention
  • Fig.7 illustrates an example of a connector according to an embodiment of the present invention
  • Fig.8 illustrates an example of a wrist band marker according to an embodiment of the present invention
  • Fig.9 illustrates an example of a finger sleeve marker size according to embodiments of the present invention
  • Fig.10 illustrates an example of the presentation of information on the user interface according to an embodiment of the present invention
  • Fig.l 1 illustrates an example of received instructions on the user interface according to embodiments of the present invention
  • Fig.12 illustrates an example of predetermined use presented on the user interface according to embodiments of the present invention
  • Fig.13 illustrates an example of using a single camera according to embodiments of the present invention.
  • Fig.14 schematically depicts a flowchart of a method 300 according to the present invention.
  • Fig. 1 schematically depicts an embodiment of a system 200 for hand rehabilitation according to the present invention.
  • the system 200 comprises a device 100 and a detection subsystem 210.
  • the device 100 comprises a plurality of finger sleeve markers 101 intended to surround a plurality of fingertips of fingers of a hand of a user, the plurality of finger sleeve markers 101 being detectable on the basis of a camera image.
  • Fig.2 illustrates an example of the device 100 according to embodiments of the present invention.
  • the detection subsystem 210 comprises a camera 201 configured to acquire a sequence of images of the devicelOO.
  • the detection subsystem 210 also comprises a processor 202 configured to receive the sequence of images acquired from the camera 201 and derive position
  • the plurality of finger sleeve markers 101 are intended to surround a plurality of fingertips of the user.
  • a finger sleeve marker 121 is illustrated in Fig.3A.
  • the finger sleeve marker 121 is sleeve shaped and comprises a first open end 122 and a second open end 123.
  • the finger sleeve marker 121 is arranged to enable a fingertip 124 to pass through and surround the fingertip 124.
  • An alternative embodiment of the finger sleeve marker 121 is illustrated in Fig.3B.
  • the finger sleeve marker 121 comprises a closed end 125 and the first open end 122.
  • the finger sleeve marker 121 is arranged to cover the fingertip 124.
  • Each finger sleeve marker is intended to be used for one fingertip.
  • a finger sleeve marker is intended to be used for identifying the corresponding fingertip.
  • the device 100 enables the plurality of finger sleeve markers 101 to be disconnected from each other. Each finger sleeve marker is used separately. The user can select a finger sleeve marker for a fingertip in a convenient manner.
  • the plurality of finger sleeve markers 101 is distinguishable from the background in an acquired camera image, which makes the plurality of finger sleeve markers 101 detectable on the basis of a camera image.
  • the camera 201 acquires a sequence of images of the device 100.
  • the sequence of images contain information of the hand of the user and the plurality of finger sleeve markers 101.
  • the processor 202 is electrically connected to the camera 201.
  • the sequence of images acquired by the camera 201 are transmitted to the processor 202.
  • the plurality of finger sleeve markers 101 are detected and position information of the sleeves is determined by the processor 202 on the basis of the camera images. This can be done with known image/pattern recognition software.
  • each of the plurality of finger sleeve markers 101 comprises a pattern, and the patterns of the plurality of finger sleeve markers 101 are different.
  • Figs.4A, 4B and 4C illustrate three examples of finger sleeve markers with different patterns which are placed at different positions on the hand. Five different patterns of finger sleeve markers are illustrated as 131, 132, 133, 134 and 135.
  • Fig.4A illustrates a first example.
  • each finger sleeve marker has a unique pattern which is different from the pattern of any other finger sleeve marker.
  • there are five finger sleeve markers which comprise five patterns. These five patterns are illustrated as 131, 132, 133, 134, and 135, respectively. Each of these five patterns is unique and different from others.
  • Fig.4B illustrates a second example.
  • each finger sleeve marker has a pattern different from the pattern(s) of the adjacent finger sleeve marker(s).
  • the pattern on the index finger is 132.
  • the adjacent patterns on both the thumb and the middle finger are 131, which is different with 132.
  • the pattern on the middle finger is 131.
  • the adjacent patterns on both the index finger and the little finger are 131, which is different from 132.
  • Fig.4C illustrates a third example.
  • a finger sleeve marker has a pattern different from one (of the) other pattern(s).
  • the pattern on the index finger is 131.
  • the pattern on the ring finger is 132, which is different from 131.
  • the patterns of the plurality of finger sleeve markers 101 comprise color patterns, graphic patterns, or shape patterns.
  • Color patterns, graphic patterns and shape patterns are typical embodiments of patterns of the plurality of finger sleeve markers 101.
  • a finger sleeve marker may comprise a color pattern.
  • a color pattern may be a pattern with only one specific color.
  • the color pattern may also be a pattern with multiple colors.
  • the color may comprise a gray level.
  • a color pattern may be a pattern with one specific gray level.
  • the color pattern may also be a pattern with multiple gray levels.
  • the color pattern may also be a pattern with mixed color(s) and gray level(s).
  • a finger sleeve marker may comprise a graphic pattern.
  • Fig.5 illustrates examples of graphic patterns according to embodiments of the present invention.
  • Fig.5 there are typical examples of graphic patterns.
  • the left one is a graphic pattern with circles.
  • the middle one is a graphic pattern with lines.
  • the right one is a graphic pattern with triangles.
  • the graphic patterns are not limited to the examples illustrated in Fig.5.
  • a finger sleeve marker may comprise a shape pattern.
  • Fig.6 illustrates examples of shape patterns according to embodiments of the present invention. At least one open end of a finger sleeve marker comprises a shape pattern. In Fig.6, there are three typical examples of shape patterns. The shape patterns are not limited to the examples illustrated in Fig.6.
  • the patterns of the plurality of finger sleeve markers 101 may also comprise a mixture of color patterns, graphic patterns and shape patterns.
  • the device 100 further comprises a connector 103 configured to connect the plurality of finger sleeve markers 101 to each other, the connector 103 being configured to leave finger joints uncovered.
  • Fig.7 illustrates an example of the connector 103 according to embodiments of the present invention.
  • the plurality of finger sleeve markers 101 are connected to each other by means of the connector 103.
  • the connector 103 is belt shaped, which is only one embodiment of the connector 103.
  • the shape and size of the connector 103 are not limited to this embodiment.
  • the connector 103 is configured to leave finger joints of the user uncovered. This enables free movement of the finger joints when the plurality of finger sleeve markers 101 are arranged to surround fingertips of the user.
  • the device 100 further comprises a wrist band marker 104 intended to surround a wrist of the user, the wrist band marker 104 being detectable on the basis of a camera image.
  • the wrist band marker 104 is arranged to surround the wrist of the user.
  • An embodiment of the wrist band marker 104 is illustrated in Fig.8.
  • the wrist band marker 104 is sleeve shaped and comprises a first open end 141 and a second open end 142.
  • the wrist band marker 104 is arranged to enable a wrist 143 to pass through and to surround the wrist 143.
  • the wrist band marker 104 is distinguishable from the background in an acquired camera image, which makes the wrist band marker 104 detectable on the basis of a camera image.
  • each of the plurality of finger sleeve markers 101 comprises a pattern
  • the wrist band marker 104 comprises a different pattern than the patterns of the plurality of finger sleeve markers 101.
  • the pattern of the wrist band marker 104 may be a color pattern which is different than the patterns of the plurality of finger sleeve markers 101.
  • the pattern of the wrist band marker 104 may also be a graphic pattern which is different than the patterns of the plurality of finger sleeve markers 104.
  • the pattern of the wrist band marker 104 may also be a shape pattern which is different than the patterns of the plurality of finger sleeve markers 104.
  • the pattern of the wrist band marker 104 may also be chosen from other alternative patterns which are different than the patterns of the plurality of finger sleeve markers 101.
  • the pattern of the wrist band marker 104 may be unique and different from any one of the plurality of finger sleeve markers 101.
  • the pattern of the wrist band marker 104 may be different from at least one of the plurality of finger sleeve markers 101.
  • a finger sleeve marker of the plurality of finger sleeve markers 101 is configured to cover a fingertip region, the fingertip region being smaller than a given region, extending from a fingertip to a corresponding distal phalanx.
  • a finger sleeve marker covers the fingertip region and does not cover the corresponding distal phalanx DP of the finger. Therefore, the joint movement of the fingertip region comprising the corresponding distal phalanx DR is the same as the joint movement of this fingertip region without the plurality of finger sleeve markers 101.
  • each of the plurality of finger sleeve markers 101 is detectable on the basis of a camera image, captured from at least two opposite viewing directions.
  • Images are captured by a camera from at least two opposite viewing directions.
  • Each of the plurality of finger sleeve markers 101 is detectable by the camera from at least two opposite viewing directions.
  • the detection subsystem 210 further comprises a user interface 203 configured to receive position information from the processor 202 and present the position information, as illustrated in Fig.1.
  • Fig.10 illustrates an example of a way in which the user interface 203 according to an embodiment of the present invention presents feedback information on a screen.
  • the positions of the plurality of finger sleeve markers 101 are indicated by spot areas illustrated as 141, 142, 143, 144 and 145 on a screen of the user interface 203.
  • the spot areas 141, 142, 143, 144 and 145 are visualized on the basis of the position information received from the processor 202.
  • An alternative presentation is to show the position information as numbers directly on the user interface 203.
  • the position information may be presented on the user interface 203 immediately for each image of the sequence of images acquired. Therefore, the movement paths of the plurality of finger sleeve markers 101 are presented on the user interface.
  • the user interface 203 is further configured to receive instructions on intended use of the plurality of finger sleeve markers 101.
  • the plurality of finger sleeve markers 101 are used for a plurality of fingertips of the user.
  • the user interface 203 is configured to receive instructions on how to use the plurality of finger sleeve markers 101, which are the basis of the use of the plurality of finger sleeve markers 101 by the user.
  • Fig. l 1 illustrates an example of received instructions on the user interface 203 according to embodiments of the present invention. The user may first put the plurality of finger sleeve markers 101 on his/her fingertips and then provide inputs to the user interface 203 on the basis of the use of the plurality of finger sleeve markers 101.
  • a finger sleeve marker pattern is selected, in this case designated as red, for the fingertip of the thumb.
  • the selection may be implemented by a common selecting box on the user interface 203. If a fingertip has no finger sleeve marker, then the user selects N/A from the list. The user needs to do the selection for all five fingertips.
  • the user interface 203 is further configured to present instructions on predetermined use of the plurality of finger sleeve markers 101.
  • the predetermined use of the plurality of finger sleeve markers 101 is used for guiding the user in using the plurality of finger sleeve markers 101.
  • the user is guided to put on the finger sleeve markers in a prescribed manner.
  • Fig.12 illustrates an example of predetermined use presented on the user interface 203 according to embodiments of the present invention.
  • finger sleeve markers to be used for fingertips are shown respectively for thumb, index finger, ring finger and little finger.
  • No finger sleeve marker is intended to be used for the middle finger in this example.
  • the instructions are used for guiding the user in putting the plurality of finger sleeve markers 101 on his/her fingertips.
  • the detection subsystem 210 comprises a single camera 201, and the processor 202 is arranged to derive position information of the plurality of finger sleeve markers 101 on the basis of the acquired sequence of images, the fingertip movement being in three dimensional space.
  • the camera 201 is arranged above the plurality of finger sleeve markers 101 to capture images of the movement of the plurality of finger sleeve markers 101. Images of the upper part of the plurality of finger sleeve markers 101 are captured by the camera 201, which means that the position information of the plurality of finger sleeve markers 101 is contained in the images captured by the camera 201.
  • a finger sleeve marker performs rotation movements along the central axis X of the finger sleeve marker, images of other parts of the finger sleeve marker can still be captured by the camera 201 as the finger sleeve marker is sleeve shaped.
  • the camera 201 captures the sequence of images of the movement of the plurality of finger sleeve markers 101 in three dimensional space. Consequently, the processor 202 derives the position information of the plurality of finger sleeve markers 101 on the basis of the acquired sequence of images.
  • Fig.2 illustrates an example of the devicelOO according to embodiments of the present invention.
  • the devicelOO for use in the system 200 comprises a plurality of finger sleeve markers 101, each of the plurality of finger sleeve markers 101 being intended to surround one of a plurality of fingertips of fingers of a hand of a user and being detectable on the basis of a camera image.
  • Fig.14 schematically depicts a flowchart of a method 300 of hand rehabilitation according to the present invention.
  • the method 300 comprises a step of acquiring 301 a sequence of images of an devicelOO.
  • the method 300 also comprises a step of deriving 302 position information of the plurality of finger sleeve markers 101 on the basis of the sequence of images acquired from the step of acquiring 301.
  • the method 300 further comprises a step of receiving 303 the position information and a step of presenting 304 the position information, as illustrated in Fig.14.
  • the method 300 further comprises a step of receiving instructions on intended use of a plurality of finger sleeve markers 101 (not shown).

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physiology (AREA)
  • Dentistry (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Rehabilitation Tools (AREA)

Abstract

The invention relates to a system (200) for hand rehabilitation. The system (200) comprises an device (1OO) and a detection subsystem (210). The device (1OO) comprises a plurality of finger sleeve markers (101), each of the plurality of finger sleeve markers (101) being intended to surround one of a plurality of fingertips of fingers of a hand of a user and being detectable on the basis of a camera image. The detection subsystem (210) comprises a camera (201) configured to acquire a sequence of images of the device (1OO). The detection subsystem (210) also comprises a processor (202) configured to receive the sequence of images acquired from the camera (201) and derive position information of the plurality of finger sleeve markers (101) on the basis of the information of the plurality of finger sleeve markers (101) contained in the sequence of images. The invention also relates to an device for use in the system (200) and a corresponding method of hand rehabilitation.

Description

System and method for hand rehabilitation
FIELD OF THE INVENTION
The invention relates to the field of rehabilitation, in particular to a system, a device and a method for hand rehabilitation. BACKGROUND OF THE INVENTION
Rehabilitation exercises are important for patients who suffered brain injury, for example, post-stroke patients. A camera-marker based rehabilitation system is a common solution, which is used by these patients to do rehabilitation exercises by themselves according to guidance provided by doctors/therapists.
Hand related exercises are typical exercises for these patients to recover or partially recover their hand functions. A hand rehabilitation device helps patients to do movement therapy, which is well accepted for motor-relearning. Such devices generally guide, monitor and measure how patients conduct exercises, give feedback on exercise quality and motivate patients to comply with the training plan.
A glove-camera based solution is widely used for hand rehabilitation. The camera detects markers on the glove to recognize hand gesture normally. A fully closed glove with separate fingers is a common solution. However, it is very hard for patients to wear this kind of gloves, due to spasm and hand incapacity, for example, for post-stroke patients. For severe patients, the fully closed gloves don't work at all.
The patent CN104887238 discloses a hand positioning system for rehabilitation, using multiple cameras in combination with LEDs placed on all fingertips and joints of a hand of a person. The respective positions of the hand are determined accordingly. Due to incapacity of the patients' hands, it is difficult for the patients to attach the LEDs to the hand by the patients themselves. Moreover, wearing LEDs on all these fingertips and joints of the hand is troublesome for patients.
SUMMARY OF THE INVENTION
It is an object of the invention to provide a system, a device and a method which provide a more user- friendly hand rehabilitation solution. According to a first aspect of the present invention, the object is achieved by an device 100 for hand rehabilitation comprising a pluralities of finger sleeve markers 101, wherein each finger sleeve marker 101 comprises a layer of flexible material in a shape corresponding to part of a finger, the finger sleeve marker 101 is arranged to be in direct contact with one of a plurality of fingertips of fingers of a hand of a user and the flexible material is detectable by camera, wherein the device 100 does not fully cover the palm.
The device provides the possibility that the plurality of finger sleeve markers are disconnected from each other, where the device does not cover the palm of the hand. The finger sleeve markers are enabled to be put on patients' fingertips or taken off from patients' fingertips separately, which is very convenient and user friendly. Compared with the glove- based solution, the device overcomes the requirement for the patients to forcibly stretch their hands during putting on or taking off the whole glove, which is challenging for patients incapable of doing that due to their hand problem. As for the glove based solution, the patient needs first to get the whole hand into glove through the opening and then to aim their fingers to at the inlet for each finger to get into the finger sheaths. Since the glove is a sealed one piece apparatus with only one opening, where the finger sheaths and palm section are connected in a fixed way, it requires a high agility of the finger movements to successfully probing into the finger sheaths. This will be painful and time consuming for the hand rehabilitation patient, who is planning to take the rehabilitation exercise to restore or improve their control over the hand including the fingers. Furthermore, the glove is a flat piece with limited flexibility. It will be troublesome to get the hand through the opening at the first step for the patients who have difficulty with hand control. The device provides a solution which does not require complex steps to wear markers or cables on the hands. The lack of the palm section make the finger sleeve markers separate and movable relative to each other. The device is also not a sealed piece anymore. The finger sleeve markers can be put onto the target position of the hand one by one in a more user friendly way, which bothers the patients least. In a sense, the system according to the invention provides the patients with a more user-friendly hand rehabilitation solution.
The plurality of finger sleeve markers are intended to surround the fingertips. The plurality of finger sleeve markers are detectable on the basis of a camera image. A finger sleeve marker thus is visible at any position around the fingertip. The device allows the finger sleeve markers to be visible to a certain camera when the fingertips assume all kinds of postures in three-dimensional space. Therefore, the determining of the positions of the fingertip sleeves requires only one camera. In an embodiment of the device according to the invention, each of the plurality of finger sleeve markers comprises a pattern, said patterns of the plurality of finger sleeve markers being different. To identify the position of a finger sleeve marker of the plurality of finger sleeve markers, using different marker patterns is easier than using the same marker patterns. This feature allows distinguishing between finger sleeve markers more easily in terms of less complexity and higher reliability.
In an embodiment of the device according to the invention, the patterns of the plurality of finger sleeve markers comprise color patterns, graphic patterns, or shape patterns. Using a color pattern-based marker or a graphic pattern-based marker allows more easily to identify the position of a finger sleeve marker from the background on the basis of acquired images. The color pattern-based marker and the graphic pattern-based marker are also practical finger sleeve marker solutions.
In an embodiment of the device according to the invention, the device further comprises a connector configured to connect the plurality of finger sleeve markers to each other, the connector being configured to leave finger joints uncovered. The connector connects the plurality of finger sleeve markers together, thereby avoiding losing a finger sleeve marker during the motion of the rehabilitation exercise. This feature allows providing a more convenient solution for a user.
In an embodiment of the apparatus according to the invention, the device further comprises a wrist band marker intended to surround a wrist of the user, the wrist band marker being detectable on the basis of a camera image. In some cases, for specific movements of a finger, the starting points may be the same and the end points may also be the same. However, the paths of the movements are different. Only using the plurality of finger sleeve markers, the two specific movements of the finger cannot be distinguished. Using, together with the plurality of finger sleeve markers, the wrist band marker, the specific movements of the finger can be more accurately distinguished. Meanwhile, wearing the wrist band marker remains equally user friendly.
In an embodiment of the device according to the invention, each of the plurality of finger sleeve markers comprises a pattern, the wrist band marker comprises a different pattern than the patterns of the plurality of finger sleeve markers. To facilitate identifying the position of a wrist band marker, the wrist band marker has a different pattern than the patterns of the plurality of finger sleeve markers.. This feature allows distinguishing it from the plurality of finger sleeve makers more easily in terms of less complexity and higher reliability. In an embodiment of the device according to the invention, a finger sleeve marker of the plurality of finger sleeve markers is configured to cover a fingertip region, the fingertip region is smaller than a given region, extending from a fingertip to a corresponding distal phalanx. This feature allows presenting movements of a fingertip, while avoiding the finger sleeve marker to interfere with movements of the corresponding distal interphalangeal.
According to another aspect of the present invention, the object is achieved by a system for hand rehabilitation, comprising a device and a detection subsystem. The device comprises a plurality of finger sleeve markers intended to surround a plurality of fingertips of fingers of a hand of a user, the plurality of finger sleeve markers being detectable on the basis of a camera image. The detection subsystem comprises a camera configured to acquire a sequence of images of the apparatus. The detection subsystem also comprises a processor configured to receive the sequence of images acquired from the camera and derive position information of the plurality of finger sleeve markers on the basis of the information of the plurality of finger sleeve markers contained in the sequence of images.
The device used in the system according to the invention employs the plurality of finger sleeve markers intended to surround the fingertips of the user. Fingertip movement tracking is crucial for hand rehabilitation as compared to movements of other parts, for example, hand joints. Therefore, by using a plurality of finger sleeve markers placed at fingertip positions, the device enables tracking the crucial movement for hand rehabilitation.
In an embodiment of the system according to the invention, the detection subsystem further comprises a user interface configured to receive position information from the processor and present the position information. The user interface allows interactions with a user when using the system for hand rehabilitation.
In an embodiment of the system according to the invention, the user interface is further configured to receive instructions on intended use of the plurality of finger sleeve markers. This feature allows setting the parameters of the plurality of finger sleeve markers according to the intended use of the plurality of finger sleeve markers by a user.
In an embodiment of the system according to the invention, the user interface is further configured to present instructions on the predetermined use of the plurality of finger sleeve markers. This feature allows guiding the user to put on the plurality of finger sleeves according to predetermined instructions.
In an embodiment of the system according to the invention, the detection subsystem comprises a single camera, the processor is arranged to derive position information of the plurality of finger sleeve markers on the basis of the acquired sequence of images, and the fingertip movement is in three dimensional space. This features allows the three-dimensional movement of the fingertips to be captured by using a single camera.
According to a second aspect of the present invention, a device for use in the system is provided. The device according to the invention comprises a plurality of finger sleeve markers, each of the plurality of finger sleeve markers being intended to surround one of a plurality of fingertips of fingers of a hand of a user and being detectable on the basis of a camera image.
According to a third aspect of the present invention, a method for hand rehabilitation is provided. The method according to the invention comprises a step of acquiring a sequence of images of an apparatus. The method also comprises a step of deriving position information of the plurality of finger sleeve markers on the basis of the sequence of images acquired in the step of acquiring.
An embodiment of the method according to the invention further comprises a step of receiving the position information. The method further comprises a step of presenting the position information.
Detailed explanations and other aspects of the invention will be given below.
BRIEF DESCRIPTION OF THE DRAWINGS
Particular aspects of the invention will now be explained with reference to the embodiments described hereinafter and considered in connection with the accompanying drawings, in which identical parts or sub-steps are designated in the same manner:
Fig.l schematically depicts an embodiment of the system according to the present invention;
Fig.2 illustrates an example of the device according to embodiments of the present invention;
Figs.3A, 3B illustrate examples of finger sleeve markers according to embodiments of the present invention;
Figs.4A, 4B and 4C illustrate three examples of different finger sleeve marker device according to embodiments of the present invention;
Fig.5 illustrates examples of graphic patterns according to embodiments of the present invention;
Fig.6 illustrates examples of shape patterns according to embodiments of the present invention; Fig.7 illustrates an example of a connector according to an embodiment of the present invention;
Fig.8 illustrates an example of a wrist band marker according to an embodiment of the present invention;
Fig.9 illustrates an example of a finger sleeve marker size according to embodiments of the present invention;
Fig.10 illustrates an example of the presentation of information on the user interface according to an embodiment of the present invention;
Fig.l 1 illustrates an example of received instructions on the user interface according to embodiments of the present invention;
Fig.12 illustrates an example of predetermined use presented on the user interface according to embodiments of the present invention;
Fig.13 illustrates an example of using a single camera according to embodiments of the present invention; and
Fig.14 schematically depicts a flowchart of a method 300 according to the present invention.
DETAILED DESCRIPTION OF THE INVENTION
The present invention will be described with respect to particular embodiments and with reference to the drawings, but the invention is not limited thereto but only by the claims. The drawings described are only schematic and are non-limiting. In the drawings, the size of some of the elements may be exaggerated and not drawn to scale for illustrative purposes.
Fig. 1 schematically depicts an embodiment of a system 200 for hand rehabilitation according to the present invention. The system 200 comprises a device 100 and a detection subsystem 210. The device 100 comprises a plurality of finger sleeve markers 101 intended to surround a plurality of fingertips of fingers of a hand of a user, the plurality of finger sleeve markers 101 being detectable on the basis of a camera image. Fig.2 illustrates an example of the device 100 according to embodiments of the present invention. The detection subsystem 210 comprises a camera 201 configured to acquire a sequence of images of the devicelOO. The detection subsystem 210 also comprises a processor 202 configured to receive the sequence of images acquired from the camera 201 and derive position
information of the plurality of finger sleeve markers 101 on the basis of the information of the plurality of finger sleeve markers 101 contained in the sequence of images. The plurality of finger sleeve markers 101 are intended to surround a plurality of fingertips of the user. One embodiment of a finger sleeve marker 121 is illustrated in Fig.3A. The finger sleeve marker 121 is sleeve shaped and comprises a first open end 122 and a second open end 123. The finger sleeve marker 121 is arranged to enable a fingertip 124 to pass through and surround the fingertip 124. An alternative embodiment of the finger sleeve marker 121 is illustrated in Fig.3B. The finger sleeve marker 121 comprises a closed end 125 and the first open end 122. The finger sleeve marker 121 is arranged to cover the fingertip 124.
Each finger sleeve marker is intended to be used for one fingertip.
Consequently, a finger sleeve marker is intended to be used for identifying the corresponding fingertip.
The device 100 enables the plurality of finger sleeve markers 101 to be disconnected from each other. Each finger sleeve marker is used separately. The user can select a finger sleeve marker for a fingertip in a convenient manner.
The plurality of finger sleeve markers 101 is distinguishable from the background in an acquired camera image, which makes the plurality of finger sleeve markers 101 detectable on the basis of a camera image.
The camera 201 acquires a sequence of images of the device 100. The sequence of images contain information of the hand of the user and the plurality of finger sleeve markers 101. The processor 202 is electrically connected to the camera 201. The sequence of images acquired by the camera 201 are transmitted to the processor 202. The plurality of finger sleeve markers 101 are detected and position information of the sleeves is determined by the processor 202 on the basis of the camera images. This can be done with known image/pattern recognition software.
In another embodiment, each of the plurality of finger sleeve markers 101 comprises a pattern, and the patterns of the plurality of finger sleeve markers 101 are different.
Figs.4A, 4B and 4C illustrate three examples of finger sleeve markers with different patterns which are placed at different positions on the hand. Five different patterns of finger sleeve markers are illustrated as 131, 132, 133, 134 and 135.
Fig.4A illustrates a first example. In this example, each finger sleeve marker has a unique pattern which is different from the pattern of any other finger sleeve marker. In Fig.4A, there are five finger sleeve markers which comprise five patterns. These five patterns are illustrated as 131, 132, 133, 134, and 135, respectively. Each of these five patterns is unique and different from others.
Fig.4B illustrates a second example. In this example, each finger sleeve marker has a pattern different from the pattern(s) of the adjacent finger sleeve marker(s). In Fig.4B, there are four finger sleeve markers. These four finger sleeve markers comprise two types of patterns which are illustrated as 131 and 132. For example, the pattern on the index finger is 132. The adjacent patterns on both the thumb and the middle finger are 131, which is different with 132. Meanwhile, the pattern on the middle finger is 131. The adjacent patterns on both the index finger and the little finger are 131, which is different from 132.
Fig.4C illustrates a third example. In this example, a finger sleeve marker has a pattern different from one (of the) other pattern(s). In Fig.4C, there are four finger sleeve markers. These four finger sleeve markers comprise two types of patterns which are illustrated as 131 and 132. For example, the pattern on the index finger is 131. The pattern on the ring finger is 132, which is different from 131.
In one embodiment, the patterns of the plurality of finger sleeve markers 101 comprise color patterns, graphic patterns, or shape patterns.
Color patterns, graphic patterns and shape patterns are typical embodiments of patterns of the plurality of finger sleeve markers 101.
A finger sleeve marker may comprise a color pattern. A color pattern may be a pattern with only one specific color. The color pattern may also be a pattern with multiple colors. The color may comprise a gray level. A color pattern may be a pattern with one specific gray level. The color pattern may also be a pattern with multiple gray levels. The color pattern may also be a pattern with mixed color(s) and gray level(s).
A finger sleeve marker may comprise a graphic pattern. Fig.5 illustrates examples of graphic patterns according to embodiments of the present invention.
In Fig.5, there are typical examples of graphic patterns. The left one is a graphic pattern with circles. The middle one is a graphic pattern with lines. The right one is a graphic pattern with triangles. The graphic patterns are not limited to the examples illustrated in Fig.5.
A finger sleeve marker may comprise a shape pattern. Fig.6 illustrates examples of shape patterns according to embodiments of the present invention. At least one open end of a finger sleeve marker comprises a shape pattern. In Fig.6, there are three typical examples of shape patterns. The shape patterns are not limited to the examples illustrated in Fig.6. The patterns of the plurality of finger sleeve markers 101 may also comprise a mixture of color patterns, graphic patterns and shape patterns.
In another embodiment, the device 100 further comprises a connector 103 configured to connect the plurality of finger sleeve markers 101 to each other, the connector 103 being configured to leave finger joints uncovered.
Fig.7 illustrates an example of the connector 103 according to embodiments of the present invention. The plurality of finger sleeve markers 101 are connected to each other by means of the connector 103. In Fig.7, the connector 103 is belt shaped, which is only one embodiment of the connector 103. The shape and size of the connector 103 are not limited to this embodiment.
The connector 103 is configured to leave finger joints of the user uncovered. This enables free movement of the finger joints when the plurality of finger sleeve markers 101 are arranged to surround fingertips of the user.
In another embodiment, the device 100 further comprises a wrist band marker 104 intended to surround a wrist of the user, the wrist band marker 104 being detectable on the basis of a camera image.
As illustrated in Fig.l, the wrist band marker 104 is arranged to surround the wrist of the user. An embodiment of the wrist band marker 104 is illustrated in Fig.8. The wrist band marker 104 is sleeve shaped and comprises a first open end 141 and a second open end 142. The wrist band marker 104 is arranged to enable a wrist 143 to pass through and to surround the wrist 143.
The wrist band marker 104 is distinguishable from the background in an acquired camera image, which makes the wrist band marker 104 detectable on the basis of a camera image.
In another embodiment, each of the plurality of finger sleeve markers 101 comprises a pattern, and the wrist band marker 104 comprises a different pattern than the patterns of the plurality of finger sleeve markers 101.
The pattern of the wrist band marker 104 may be a color pattern which is different than the patterns of the plurality of finger sleeve markers 101. The pattern of the wrist band marker 104 may also be a graphic pattern which is different than the patterns of the plurality of finger sleeve markers 104. The pattern of the wrist band marker 104 may also be a shape pattern which is different than the patterns of the plurality of finger sleeve markers 104. The pattern of the wrist band marker 104 may also be chosen from other alternative patterns which are different than the patterns of the plurality of finger sleeve markers 101. The pattern of the wrist band marker 104 may be unique and different from any one of the plurality of finger sleeve markers 101. The pattern of the wrist band marker 104 may be different from at least one of the plurality of finger sleeve markers 101.
In another embodiment, a finger sleeve marker of the plurality of finger sleeve markers 101 is configured to cover a fingertip region, the fingertip region being smaller than a given region, extending from a fingertip to a corresponding distal phalanx.
As illustrated in Fig.9, a finger sleeve marker covers the fingertip region and does not cover the corresponding distal phalanx DP of the finger. Therefore, the joint movement of the fingertip region comprising the corresponding distal phalanx DR is the same as the joint movement of this fingertip region without the plurality of finger sleeve markers 101.
In another embodiment, each of the plurality of finger sleeve markers 101 is detectable on the basis of a camera image, captured from at least two opposite viewing directions.
Images are captured by a camera from at least two opposite viewing directions.
Each of the plurality of finger sleeve markers 101 is detectable by the camera from at least two opposite viewing directions.
In an embodiment, the detection subsystem 210 further comprises a user interface 203 configured to receive position information from the processor 202 and present the position information, as illustrated in Fig.1.
Fig.10 illustrates an example of a way in which the user interface 203 according to an embodiment of the present invention presents feedback information on a screen. In this example, the positions of the plurality of finger sleeve markers 101 are indicated by spot areas illustrated as 141, 142, 143, 144 and 145 on a screen of the user interface 203. The spot areas 141, 142, 143, 144 and 145 are visualized on the basis of the position information received from the processor 202.
An alternative presentation is to show the position information as numbers directly on the user interface 203.
The position information may be presented on the user interface 203 immediately for each image of the sequence of images acquired. Therefore, the movement paths of the plurality of finger sleeve markers 101 are presented on the user interface.
In another embodiment, the user interface 203 is further configured to receive instructions on intended use of the plurality of finger sleeve markers 101. The plurality of finger sleeve markers 101 are used for a plurality of fingertips of the user. The user interface 203 is configured to receive instructions on how to use the plurality of finger sleeve markers 101, which are the basis of the use of the plurality of finger sleeve markers 101 by the user. Fig. l 1 illustrates an example of received instructions on the user interface 203 according to embodiments of the present invention. The user may first put the plurality of finger sleeve markers 101 on his/her fingertips and then provide inputs to the user interface 203 on the basis of the use of the plurality of finger sleeve markers 101. In this embodiment, for each finger sleeve marker placed on a fingertip, the user inputs the color of the finger sleeve marker on the respective fingertip. In Fig.l 1, a finger sleeve marker pattern is selected, in this case designated as red, for the fingertip of the thumb. The selection may be implemented by a common selecting box on the user interface 203. If a fingertip has no finger sleeve marker, then the user selects N/A from the list. The user needs to do the selection for all five fingertips.
In another embodiment, the user interface 203 is further configured to present instructions on predetermined use of the plurality of finger sleeve markers 101.
The predetermined use of the plurality of finger sleeve markers 101 is used for guiding the user in using the plurality of finger sleeve markers 101. In this embodiment, the user is guided to put on the finger sleeve markers in a prescribed manner. Fig.12 illustrates an example of predetermined use presented on the user interface 203 according to embodiments of the present invention. In Fig.12, finger sleeve markers to be used for fingertips are shown respectively for thumb, index finger, ring finger and little finger. No finger sleeve marker is intended to be used for the middle finger in this example. The instructions are used for guiding the user in putting the plurality of finger sleeve markers 101 on his/her fingertips.
In another embodiment, the detection subsystem 210 comprises a single camera 201, and the processor 202 is arranged to derive position information of the plurality of finger sleeve markers 101 on the basis of the acquired sequence of images, the fingertip movement being in three dimensional space.
As illustrated in Fig.13, the camera 201 is arranged above the plurality of finger sleeve markers 101 to capture images of the movement of the plurality of finger sleeve markers 101. Images of the upper part of the plurality of finger sleeve markers 101 are captured by the camera 201, which means that the position information of the plurality of finger sleeve markers 101 is contained in the images captured by the camera 201. When a finger sleeve marker performs rotation movements along the central axis X of the finger sleeve marker, images of other parts of the finger sleeve marker can still be captured by the camera 201 as the finger sleeve marker is sleeve shaped. Therefore, the camera 201 captures the sequence of images of the movement of the plurality of finger sleeve markers 101 in three dimensional space. Consequently, the processor 202 derives the position information of the plurality of finger sleeve markers 101 on the basis of the acquired sequence of images.
Fig.2 illustrates an example of the devicelOO according to embodiments of the present invention. The devicelOO for use in the system 200 comprises a plurality of finger sleeve markers 101, each of the plurality of finger sleeve markers 101 being intended to surround one of a plurality of fingertips of fingers of a hand of a user and being detectable on the basis of a camera image.
Fig.14 schematically depicts a flowchart of a method 300 of hand rehabilitation according to the present invention.
The method 300 comprises a step of acquiring 301 a sequence of images of an devicelOO. The method 300 also comprises a step of deriving 302 position information of the plurality of finger sleeve markers 101 on the basis of the sequence of images acquired from the step of acquiring 301.
In an embodiment, the method 300 further comprises a step of receiving 303 the position information and a step of presenting 304 the position information, as illustrated in Fig.14.
In another embodiment, the method 300 further comprises a step of receiving instructions on intended use of a plurality of finger sleeve markers 101 (not shown).
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing a claimed invention, from a study of the drawings, the disclosure, and the dependent claims.
In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfil the functions of several items re-cited in the claims. The mere fact that certain measures are re-cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:
1. A device (100) for hand rehabilitation comprising a pluralities of finger sleeve markers (101), wherein each finger sleeve marker (101) comprises a layer of material in a shape corresponding to part of a finger and the finger sleeve marker (101) is arranged to surround one of a plurality of fingertips of fingers of a hand of a user;
wherein the device(lOO) does not cover the palm.
2. A device(lOO) as claimed in claim 1, wherein each of the plurality of finger sleeve markers (101) comprises a pattern or color, wherein patterns or colors of the plurality of finger sleeve markers (101) are different, .
3. A device(lOO) as claimed in claim 2, wherein the patterns of the plurality of finger sleeve markers (101) comprise color patterns, graphic patterns, or shape patterns.
4. A device(lOO) as claimed in claim 1, 2, or 3, wherein the device(lOO) further comprises a connector (103) configured to connect the plurality of finger sleeve markers
(101) to each other, wherein the connector (103) is configured to leave finger joints uncovered.
5. A device(lOO) as claimed in claim 1, wherein the device(lOO) further comprises a wrist band marker (104) intended to surround a wrist of the user, wherein the wrist band marker (104) is detectable on the basis of a camera image.
6. A device(lOO) as claimed in claim 5, wherein each of the plurality of finger sleeve markers (101) comprises a pattern, wherein the wrist band marker (104) comprises a different pattern than patterns of the plurality of finger sleeve markers (101).
7. A device(lOO) as claimed in any one of claims 1 to 6, wherein a finger sleeve marker of the plurality of finger sleeve markers (101) is configured to cover a fingertip region, wherein the fingertip region is part of a given region, extending from a fingertip to a corresponding distal phalanx.
8. A system (200) for hand rehabilitation comprising an device(lOO) and a detection subsystem (210), the device(lOO) comprising:
a plurality of finger sleeve markers (101),
the detection subsystem (210) comprising:
a camera (201) configured to acquire a sequence of images of the device(lOO); and
a processor (202) configured to receive the sequence of images acquired from the camera (201) and derive position information of the plurality of finger sleeve markers (101) on the basis of the information of the plurality of finger sleeve markers (101) contained in the sequence of images;
wherein each finger sleeve marker (101) comprises a layer of material in a shape
corresponding to part of a finger, the finger sleeve marker (101) is arranged to surround one of a plurality of fingertips of fingers of a hand of a user and the material is detectable by camera;
wherein the device(lOO) does not cover the palm.
9. A system as claimed in any one of claims 8, wherein each of the plurality of finger sleeve markers (101) is detectable on the basis of a camera image.
10. A system as claimed in any one of claims 8 to 9, wherein the detection subsystem (210) further comprises a user interface (203) configured to receive the position information from the processor (202) and present the position information.
11. A system as claimed in claim 10, wherein the user interface (203) is further configured to receive instructions on intended use of the plurality of finger sleeve markers (101).
12. A system as claimed in claim 10, wherein the user interface (203) is further configured to present instructions on predetermined use of the plurality of finger sleeve markers (101).
13. A system as claimed in any one of claims 9 to 12, wherein the detection subsystem (210) comprises a single camera (201), and wherein the processor (202) is arranged to derive the position information of the plurality of finger sleeve markers (101) on the basis of the acquired sequence of images, wherein the fingertip movement is in three dimensional space.
14. A method (300) of hand rehabilitation, comprising the steps of:
acquiring (301) a sequence of images of the device(lOO); and deriving (302) position information of the plurality of finger sleeve markers (101) on the basis of the sequence of images acquired from the step of acquiring (301), wherein each finger sleeve marker (101) comprises a layer of material in a shape corresponding to part of a finger, the finger sleeve marker (101) is arranged to surround one of a plurality of fingertips of fingers of a hand of a user and the material is detectable by camera;
wherein the device(lOO) does not cover the palm.
15. A method as claimed in claim 14, further comprising the steps of:
receiving (303) the position information; and
presenting (304) the position information.
PCT/EP2016/082609 2015-12-24 2016-12-23 System and method for hand rehabilitation WO2017109205A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CNPCT/CN2015/098688 2015-12-24
CN2015098688 2015-12-24
EP16162376.4 2016-03-24
EP16162376 2016-03-24

Publications (1)

Publication Number Publication Date
WO2017109205A1 true WO2017109205A1 (en) 2017-06-29

Family

ID=57681598

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2016/082609 WO2017109205A1 (en) 2015-12-24 2016-12-23 System and method for hand rehabilitation

Country Status (1)

Country Link
WO (1) WO2017109205A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100199232A1 (en) * 2009-02-03 2010-08-05 Massachusetts Institute Of Technology Wearable Gestural Interface
US20130060166A1 (en) * 2011-09-01 2013-03-07 The Regents Of The University Of California Device and method for providing hand rehabilitation and assessment of hand function
WO2015139750A1 (en) * 2014-03-20 2015-09-24 Telecom Italia S.P.A. System and method for motion capture
WO2015161194A1 (en) * 2014-04-17 2015-10-22 Flint Rehabilitation Devices, Llc. Systems and methods for rehabilitating the hand

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100199232A1 (en) * 2009-02-03 2010-08-05 Massachusetts Institute Of Technology Wearable Gestural Interface
US20130060166A1 (en) * 2011-09-01 2013-03-07 The Regents Of The University Of California Device and method for providing hand rehabilitation and assessment of hand function
WO2015139750A1 (en) * 2014-03-20 2015-09-24 Telecom Italia S.P.A. System and method for motion capture
WO2015161194A1 (en) * 2014-04-17 2015-10-22 Flint Rehabilitation Devices, Llc. Systems and methods for rehabilitating the hand

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
FRANCESCA CORDELLA ET AL: "Patient performance evaluation using Kinect and Monte Carlo-based finger tracking", BIOMEDICAL ROBOTICS AND BIOMECHATRONICS (BIOROB), 2012 4TH IEEE RAS&EMBS INTERNATIONAL CONFERENCE ON, IEEE, 24 June 2012 (2012-06-24), pages 1967 - 1972, XP032234058, ISBN: 978-1-4577-1199-2, DOI: 10.1109/BIOROB.2012.6290794 *
JONAS FREDRIKSSON ET AL: "Real-time 3D hand-computer interaction", PROCEEDINGS OF THE 5TH NORDIC CONFERENCE ON HUMAN-COMPUTER INTERACTION BUILDING BRIDGES, NORDICHI '08, ACM PRESS, NEW YORK, NEW YORK, USA, 20 October 2008 (2008-10-20), pages 133 - 141, XP058114738, ISBN: 978-1-59593-704-9, DOI: 10.1145/1463160.1463175 *
LUIGI LAMBERTI ET AL: "Handy: A real-time three color glove-based gesture recognizer with learning vector quantization", EXPERT SYSTEMS WITH APPLICATIONS, vol. 39, no. 12, 1 February 2012 (2012-02-01), pages 10489 - 10494, XP028487577, ISSN: 0957-4174, [retrieved on 20120306], DOI: 10.1016/J.ESWA.2012.02.081 *

Similar Documents

Publication Publication Date Title
US20120077163A1 (en) 3d monocular visual tracking therapy system for the rehabilitation of human upper limbs
JP7116696B2 (en) Learning support system and program
JP5609973B2 (en) Method and system for examining or training eye movements and body movements, and method for examining or training visual ability and intention
KR102064795B1 (en) Posture training system and method of control thereof
CN111091732B (en) Cardiopulmonary resuscitation (CPR) instructor based on AR technology and guiding method
EP3232930B1 (en) Assessment of an attentional deficit
US10849532B1 (en) Computer-vision-based clinical assessment of upper extremity function
CN104274183A (en) Motion information processing apparatus
KR20140015144A (en) Method and system for hand presence detection in a minimally invasive surgical system
KR20130027006A (en) Method and apparatus for hand gesture control in a minimally invasive surgical system
US11426099B2 (en) Mobile device avatar generation for biofeedback to customize movement control
US11134866B2 (en) Garments, systems and methods for sports training
TWI496027B (en) Motion guidance prompting method, system thereof and motion guiding prompting device
Sziladi et al. The analysis of hand gesture based cursor position control during solve an IT related task
KR100452917B1 (en) Method and System for sensing Three-Dimensional body motion using color marker
KR20170010157A (en) Method for guiding actions of an user in interactive exercise programs and the apparatus thereof
Vidrios-Serrano et al. Development of a haptic interface for motor rehabilitation therapy using augmented reality
Gauthier et al. Human movement quantification using Kinect for in-home physical exercise monitoring
WO2013098791A1 (en) A method and apparatus for tracking hand and/or wrist rotation of a user performing exercise
Krabben et al. How wide should you view to fight? Establishing the size of the visual field necessary for grip fighting in judo
JP6310255B2 (en) Method and apparatus for presenting options
CN104023634A (en) A method and apparatus for tracking hand and/or wrist rotation of a user performing exercise
WO2017109205A1 (en) System and method for hand rehabilitation
KR20180099399A (en) Fitness Center and Sports Facility System Using a Augmented reality virtual trainer
WO2023135602A1 (en) System and method for hand rehabilitation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16819939

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16819939

Country of ref document: EP

Kind code of ref document: A1