KR20200082423A - Virtual reality-based hand rehabilitation system with haptic feedback - Google Patents

Abstract

The present invention relates to a tactile feedback-enabled virtual reality-based hand rehabilitation system which receives real-time motion through the movement of a finger using a bending sensor and a 3-axis movement of the hand through an IMU sensor to express a virtual hand image in content, and performs rehabilitation training by giving a tactile feedback using a vibration motor when an object in the image corresponds to the virtual hand image.

Description

Virtual reality-based hand rehabilitation system with haptic feedback

The present invention relates to a hand rehabilitation system, and in detail, receives a real-time motion through the movement of a finger using a bending sensor and a 3-axis movement of a hand through an IMU sensor to express a virtual hand image on content, and objects and virtual images It relates to a virtual reality-based hand rehabilitation system that enables tactile feedback to perform rehabilitation training by providing a tactile feedback through a vibration motor when responding to a hand image.

Hand rehabilitation is a treatment method that restores the normal function of the hand by applying proper stimulation to the hand or exercising for the movement of the hand, and is essential training for normal daily life.

The hand has an anatomical structure formed of numerous delicate tissues, has a very complex movement mechanism, and has many functions that can be performed. For example, it is possible to perform a variety of sophisticated movements by accommodating not only the grabbing function but also the detailed sense.

Hand rehabilitation focuses on restoring the natural movement of the hand, and is mainly repeated training to improve the strength and accuracy of hand movement.

Currently, such hand rehabilitation can be applied to various methods such as robot rehabilitation, mirror therapy, and functional electrical simulation (FES), and it is usually inconvenient to visit an institution equipped with complex facilities or under the guidance of a rehabilitation specialist. It is pointed out that the problem of low motivation is caused by repetition of ships and simple movements.

In particular, in recent years, trauma of the upper limb is lost or decreased due to traumatic brain injury due to various factors such as stroke, traffic accidents, or rheumatoid patients, or peripheral nerve damage. Rehabilitation treatments such as artificial exercise for the desired joint, such as the wrist and metacarpophalangeal joints, are essential because they may fall into greater difficulty due to decreased elasticity.

Accordingly, there is a need for techniques capable of conveniently performing hand rehabilitation even at home, which is performed by medical equipment specialized in rehabilitation and specialists, which is limited in place and time and creates a lot of economic burden on patients or guardians.

Republic of Korea Registered Patent No. 10-1546405 (2015.08.17)

The present invention was created to solve the above problems, and an object of the present invention is to receive real-time hand motion through the movement of a finger using a flexural sensor and a 3-axis movement of a hand through an IMU sensor to simulate virtual hands on content. A virtual reality-based hand rehabilitation system capable of tactile feedback that expresses an image and provides tactile feedback when responding to an object in the image and a virtual hand image and supports effective rehabilitation training

For the above object, in the hand rehabilitation system, the present invention is worn on the subject's head, and outputs the direction detection unit measuring the movement of the subject's head and the applied image content by reflecting the measurement results of the direction detection unit. HMD unit having a display; A body in the form of a glove that is worn on the user's hand and has an exoskeleton structure formed across the user's back in the lateral direction; One end portion is installed on the exoskeleton structure through a hinge portion that can be rotated left and right, a bending detection unit that is formed for each finger portion and can measure the degree of bending of the finger; An angle sensing unit capable of measuring the rotation angle of the hinge unit; A vibration stimulation unit installed at each fingertip to generate vibration of a pattern set according to an applied signal; A motion detection unit that senses 3-axis movement of the body by combining an acceleration sensor and a gyro sensor; A virtual hand image while reflecting the detection results of the bending detection unit, the angle detection unit and the motion detection unit to the applied image content, and the storage unit storing the image content instructing the user's hand to move, and applying the image content to the display. Control unit having a synchronization unit for reflecting the image content, and a feedback unit for operating the vibration stimulation unit in the set detection results; It is characterized by consisting of.

In this case, the image content may include a virtual cube image, and the feedback unit may be configured to operate the vibrating stimulation unit when the virtual hand picks up the cube image.

In addition, the video content includes a map image that guides the user to take a specific hand gesture, and the feedback unit may be configured to operate the vibration stimulator when the virtual hand takes the same posture as the guided hand gesture.

In addition, the bending detection unit may be configured to detect bending of each joint by being located above the joint portion of the finger and hand.

Through the present invention, the user can perform hand rehabilitation training in a moving form by mapping the hand image corresponding to the movement of the user's hand of the VR content to the objects while wearing the HMD and the glove-shaped body. Through the dynamic change and vibration feedback of the displayed image object, it is possible to induce interest and motivation according to repeated training.

In particular, it is possible to conduct rehabilitation training with only simple equipment at home without a visit to a hospital or a specialized institution, providing excellent convenience regardless of time and place.

In addition, the rehabilitation exercise can be conducted in a game manner, thereby promoting concentration and fun, and an efficient rehabilitation exercise can be performed.

1 is a conceptual diagram of the present invention,
Figure 2 is a structural diagram showing the structure of the body according to an embodiment of the present invention,
Figure 3 is a block diagram showing the configuration and connection relationship according to an embodiment of the present invention,
4 is a first exemplary view showing a video content according to an embodiment of the present invention,
5 is a second exemplary view showing image content according to an embodiment of the present invention.

Hereinafter, a virtual reality based hand rehabilitation system capable of tactile feedback of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram of the present invention, the present invention is to enable an effective rehabilitation exercise for the hand part, but based on virtual reality through an input device to receive the motion and movement of the hand during the rehabilitation training game to be used for rehabilitation content As such, it supports the knee exercise through the promotion of motivation and interest in the home or in various indoor environments without the assistance of other specialized equipment and experts.

Figure 2 is a structural diagram showing the structure of a body according to an embodiment of the present invention, Figure 3 is a block diagram showing the configuration and connection relationship according to an embodiment of the present invention, the present invention is the HMD unit 110 and the armored body The main components of the 120, the bending detection unit 130, the angle detection unit 140, the vibration stimulation unit 150, the motion detection unit 160, and the control unit 170 will be provided.

The HMD unit 110 is a head mounted display for a virtual reality or augmented reality experience, and is worn on a subject's head to output an image. The detailed configuration includes a direction detecting unit 111 and a display 112. To be equipped.

The direction detecting unit 111 is configured to recognize the direction in which the subject views the subject's head by measuring the movement of the subject's head, that is, it is implemented through various sensors capable of measuring three-axis movement.

The display 112 is located in front of the eyes of the target inside the HMD unit 110 to output the authorized image content, that is, a VR image, but reflects the measurement result of the direction detecting unit 111 to display the image corresponding to the subject's gaze. Output and build a VR environment.

The body 120 is a structure in the form of a glove worn on the user's hand, and is provided with an exoskeleton structure 121 formed across the user's back in the lateral direction.

This exoskeleton structure 121, unlike the prior art that measures only the bending of the finger, allows the installation of the hinge portion 122 to be described later in a configuration for variously implementing the actual operation of the finger.

The bending detection unit 130 is installed for each finger is a sensor (FlexSensor) for measuring the degree of bending of the finger, one end is installed through the hinge portion 122 that can be rotated left and right to the exoskeleton structure 121, As the resistance value changes according to the degree of bending, the degree of bending of each finger is measured. At this time, the hinge portion 122 is installed to support the left and right rotation, not the rotation of the front and rear of the finger, that is, bending and bending.

At this time, by installing one bending detection unit 130 on each finger, it is possible to detect the overall bending of the finger, but as one finger is made of multiple joints, the bending detection unit is located above the joints of the fingers and hands. It can be configured to detect the bending of each joint. That is, a plurality of bending detection units may be installed in correspondence with each joint of the finger to detect detailed bending of the finger.

The angle detection unit 140 is installed on the hinge unit 122 is an angle sensor that can measure the left and right rotation angle of each bending detection unit 130, to detect the left and right movement of the finger as shown in Figure 5 attached do.

This is intended to measure the degree of flaring, not the bending of the finger, and is not a technology that is implemented in a product using a conventional flexible sensor. Through this, more colorful hand movements and movements can be achieved.

The vibrating stimulation unit 150 is installed for each fingertip and is configured to generate a vibration of a set pattern according to an applied signal. This is a vibration motor for tactile feedback to a subject, mounted on a fingertip When interacting with an object in the game, a voltage is applied to the user to feel vibration.

That is, it is possible to provide tactile feedback for each finger that acts on the subject as a result of the content through a vibration motor, not a sensor, as in the prior art.

As will be described later, when the cube is picked up by the virtual hand and the real-time visual information 1 displayed by the virtual hand on the display 112, that is, when the finger position matches the cube and the thumb and index finger are in contact with the cube, the picking motion and Likewise, by moving the cube along the hand, it can be operated as an input device capable of controlling motion reflected on the display of the content from the subject.

In some conventional techniques, the intensity of the pinch operation is transmitted from the subject to the content using a pressure sensor, but in the present invention, interaction may be made through feedback from the content to the subject. At this time, the vibration stimulation unit 150 is mounted on the tip of each finger, and provides feedback by vibrating under a specific condition in response to each finger action.

The motion detection unit 160 is a configuration that detects the 3-axis movement of the body by combining an accelerometer sensor and a gyro sensor, and measures overall hand posture. In an embodiment of the present invention, an acceleration sensor and a gyro sensor are combined. You will use the old IMU (Inertia Measurement Unit) sensor.

The acceleration sensor constituting this measures the acceleration of how fast or slow the object is, and it is possible to measure how much it is tilted with respect to the Earth's surface through gravitational acceleration, i.e. 3 of the x, y, and z axes. The axis can be measured.

The gyro sensor measures the rotational speed of the three axes of the x, y, and z axes. When the hand movement is to be accurately measured as in the present invention, the acceleration sensor alone cannot compensate for the direction of movement to compensate for the direction and movement. The accuracy is measured more accurately.

In addition, unlike the acceleration sensor, the gyro sensor is not affected by gravity, so precise motion measurement is performed by combining it with the acceleration sensor. Similarly, measurement is performed on the 3 axes by x, y, z axis or roll, pitch, and yaw. To perform.

The control unit 170 communicates signals and data through communication with the HMD unit 110, the bending detection unit 130 and the angle detection unit 140, the vibration stimulation unit 150 and the motion detection unit 160. It is a configuration that performs a rehabilitation exercise while exchanging, and includes a detailed configuration of a storage unit 171, a synchronization unit 172, and a feedback unit 153 as a detailed configuration.

To this end, the HMD unit 110, the bending detection unit 130 and the angle detection unit 140, the vibration stimulation unit 150 and the motion detection unit 160, and movement through a short range wireless communication module represented by Bluetooth It is possible to increase the ease of use and realism while reducing the inconvenience of, and a detailed description is omitted in order to prevent the purpose of the invention from being blurred by using such communication means.

The storage unit 171 stores image content instructing the user to move the hand.

The synchronization unit 172 is configured to apply the video content according to the mode selection to the display 112, the detection result of the bending detection unit 130 and the angle detection unit 140 and the motion detection unit 160 Will be reflected in the authorized video content.

To this end, basically, the bending detection unit 130 and the angle detection unit 140 and an engine capable of systematically reproducing the movement of the hand through the data of the detection results of the motion detection unit 160 are provided. Through this, it is determined whether the subject takes a hand gesture, and accordingly, the VR image is also synchronized so that it can be changed correspondingly.

The feedback unit 173 provides feedback by operating the vibration stimulation unit 150 as the set detection result, that is, the subject takes a specific hand gesture.

That is, the motion of each finger and the motion of the hand through the IMU sensor of the motion detection unit 160 from the bending detection unit 130 and the angle detection unit 140 are inputted in real time, and virtual hands are added to the content. It can be expressed as an image, and a tactile feedback can be given by vibrating when an object in the image seen by the patient is touched or bumped.

4 is a first exemplary view showing image content according to an embodiment of the present invention, wherein the image content includes a virtual cube image, and the feedback unit 173 is operated when a virtual hand picks up the cube image It may be configured to operate the vibration stimulation unit 150.

In addition, the video content includes a map image that guides the user to take a specific hand gesture, and the feedback unit 173 operates the vibration stimulator 150 when the virtual hand takes the same posture as the guided hand gesture. It can be configured to.

5 is a second exemplary view showing image content according to an embodiment of the present invention, and shows a state reflected by a virtual hand when a user spreads or closes a finger from side to side.

The rights of the present invention are not limited to the embodiments described above, but are defined by the claims, and those skilled in the art can make various modifications and adaptations within the scope of the claims. It is self-evident.

110: HMD unit 111: direction detection unit
112: display 120: body
121: exoskeleton structure 122: hinge portion
130: bending detection unit 140: angle detection unit
150: vibration stimulation unit 160: motion detection unit
170: control unit 171: storage unit
172: synchronization unit 173: feedback unit

Claims (4)

In the hand rehabilitation system,
An HMD unit equipped with a direction sensing unit 111 that is worn on the subject's head and measures the movement of the subject's head, and a display 112 that outputs the applied image content by reflecting the measurement results of the direction detection unit 111 (110);
It is worn on the user's hand, the body 120 in the form of a glove having an exoskeleton structure 121 formed across the user's back in the lateral direction;
One end portion is installed on the exoskeleton structure 121 through a hinge portion 122 that can be rotated left and right, is formed for each finger portion and the bending detection unit 130 capable of measuring the degree of bending of the finger;
An angle sensing unit 140 capable of measuring the rotation angle of the hinge unit 122;
A vibration stimulation unit 150 installed at each fingertip and generating vibration of a pattern set according to an applied signal;
A motion detection unit 160 that senses three-axis movement of the body 120 by combining an acceleration sensor and a gyro sensor;
The storage unit 171 in which the image content instructing the user to move the hand is stored, and the image content is applied to the display 112, but the bending detection unit 130 and the angle detection unit 140 and the motion detection unit 160 ) Is provided with a synchronization unit 172 that reflects the detection result of the applied image content and reflects a virtual hand image on the image content, and a feedback unit 173 that operates the vibration stimulation unit 150 in the set detection result. The control unit 170; Hand rehabilitation system based on virtual reality capable of tactile feedback, characterized in that consisting of.
According to claim 1,
The image content includes a virtual cube image,
The feedback unit 173 is a virtual reality-based hand rehabilitation system capable of tactile feedback, characterized in that the virtual hand is configured to operate the vibration stimulation unit 150 during the operation of picking up the cube image.
According to claim 1,
The video content includes a map image that guides the user to take a specific hand gesture,
The feedback unit 173 is a virtual reality-based hand rehabilitation system capable of tactile feedback, characterized in that configured to operate the vibration stimulation unit 150 when the virtual hand takes the same posture as the guided hand movement.
According to claim 1,
The bending detection unit 130 is located on the upper part of the joint of the finger and hand, and configured to detect bending of each joint, a virtual reality-based hand rehabilitation system capable of tactile feedback.

Images