KR102253353B1 - Method and apparatus for providing rehabilitation content using virtual transplant hand based on vr/ar - Google Patents

Abstract

According to an embodiment of the present invention, a method for providing rehabilitation content comprises the operations of: storing rehabilitation content including a map composed of a plurality of fields through which a character in a virtual space passes from a starting point to a destination; outputting the plurality of fields to the virtual space as the rehabilitation content is played; acquiring the shape and movement of a second hand according to the movement of a patient from a wearable device worn on the second hand, which is a body part symmetrical with a first hand corresponding to an amputated or paralyzed body part of the patient; generating a virtual body image having the shape and movement of the second hand and the inverted shape and movement in a virtual space based on the obtained information; outputting a list of coding blocks that control an operation of the character to the virtual space, and generating a coding block selected by the patient; arranging the coding block in any one of the plurality of fields based on the movement of the virtual body image that moves in reverse with respect to the movement of the second hand; and controlling, as the arrangement of the coding block for the plurality of fields is completed, to perform an operation according to coding information mapped to a field through which the character passes from the starting point. According to the present invention, it is possible to conduct treatment which has the same concept as mirror therapy.

Description

Method and device for providing rehabilitation contents using VR/AR-based virtual implanted hands {METHOD AND APPARATUS FOR PROVIDING REHABILITATION CONTENT USING VIRTUAL TRANSPLANT HAND BASED ON VR/AR}

The present invention relates to a method and apparatus for providing rehabilitation content using a virtual implanted hand based on VR/AR.

The number of patients with amputation of an arm or leg is increasing every year due to diseases related to modern society such as diabetes and Burger's disease. 50% to 80% of limb amputation patients with such amputation of an arm or leg suffer from phantom pain. Fantastic pain is a disease in which a certain part of the body feels as though it is physically absent due to an accident or disease and suffers pain.

A representative treatment for phantom pain is mirror therapy. Mirror therapy is a treatment method that relieves phantom pain by performing hand movements on a patient whose arm is amputated by using a mirror to illuminate the other arm as if they had two arms. As a method of reducing the phantom pain felt at the amputated area by causing the visual illusion that the body in the cut area moves without pain in amputated patients, the effectiveness of this treatment has been proven in the medical field.

However, mirror therapy is inconvenient to apply because it must be performed in a posture that accurately aligns the angle of the patient's gaze to the mirror, and because it allows only a limited range of movements right next to the mirror, it differs from various movements used in real life. There was a limit that it could only be achieved.

In addition, the treatment method using a mirror is difficult to apply to patients with amputated or paralyzed lower limbs because it is difficult to realistically match the angle between the patient's gaze and the mirror in the case of a patient with a leg amputated.

Korean Patent Publication No. 10-200112: Motion recognition device using gyro sensor and its motion recognition method

The problem to be solved in the embodiments of the present invention is that the patient maintains the treatment concept of mirror therapy that reduces the phantom pain felt at the cut site by causing the patient to make an illusion with the visual effect that the body at the cut part moves without pain. By implementing a VR/AR technology combination method that can perform various movements without a mirror, it provides a technology that contributes to enhancing the effect of treatment and rehabilitation training to alleviate phantom pain of limb amputation patients.

In addition, by providing content that utilizes the features of VR/AR technology in addition to the above-described treatment technology, VR/AR content that allows patients to feel interest while receiving the feeling that they use their arms and legs realistically in the treatment process is provided. I want to provide.

However, the technical problems to be achieved by the embodiments of the present invention are not limited to the above-mentioned problems, and various technical problems can be derived from the contents to be described below within a range that is obvious to a person skilled in the art.

A method of providing rehabilitation content performed by an apparatus for providing rehabilitation content according to an embodiment of the present invention includes storing rehabilitation content including a map consisting of a plurality of fields that a character in a virtual space passes through to go from a starting point to a destination; Outputting the plurality of fields to a virtual space as the rehabilitation content is played; Acquiring the shape and movement of the second hand according to the movement of the patient from the wearable device worn on the second hand, which is a body part symmetric with the first hand corresponding to the patient's amputated or paralyzed body part; Generating, in the virtual space, a virtual body image having a shape and movement of the second hand and a shape and movement in which the left and right sides are reversed based on the obtained information; Outputting a list of coding blocks that control the motion of the character to the virtual space, and generating a coding block selected by the patient; Arranging the coding block in any one of the plurality of fields based on the motion of the virtual body image; And an operation of controlling the character to perform an operation according to coding information mapped to a field passed from the starting point upon completion of the arrangement of the coding blocks for the plurality of fields.

In addition, the generating of the coding block may include outputting a list including a plurality of coding blocks to the virtual space; And generating, in the virtual body image, a coding block selected by the grabbing operation of the virtual body image with respect to the list.

In addition, the placing in the field may include detecting a motion of the virtual body image placing or throwing the coding block; Calculating a direction and speed at which the coding block is thrown based on a direction and speed of the virtual body image, and outputting a motion of the coding block in the virtual space; And when it is determined that the coding block has reached any one of the plurality of fields according to the movement of the coding block, mapping coding information for an operation corresponding to the coding block to the one of the fields. It may include.

In addition, the plurality of fields may include a first field through which a character can pass; A second field in which the game is over when the character passes; And a third field in which a monster exists, and the operation of outputting the plurality of fields to the virtual space may include selecting at least two or more fields from among the first field, the second field, and the third field; And generating the map so that at least one other selected field is connected to one side of one of the selected fields and outputting the map to the virtual space.

In addition, the coding block includes first coding information for advancing the character; Second coding information for jumping the character to jump over a field in front; Third coding information for rotating the character to the left; Fourth coding information for rotating the character to the right; Fifth coding information for removing a monster in front of the character by performing an attack; And sixth coding information for changing an item mapping rule for the one field so that a plurality of coding blocks can be mapped to one field, and the controlling operation is performed by changing the mapping rule according to the sixth coding information. When the character passes through a field in which a plurality of coding blocks are mapped to one field, an operation of controlling the operation of the character in the order in which the plurality of coding blocks are mapped may be included.

In addition, the method for providing rehabilitation content includes, after the operation of placing in the field, the code progress information on the order in which the coding information mapped to the plurality of fields is executed according to the arrangement of the coding blocks for the plurality of fields. The operation of outputting to a virtual space may be further included.

In addition, the wearable device is configured in a shape corresponding to the second hand such that a flex film capable of sensing the shape of the surface through a resistance value that changes as the surface is increased or decreased, surrounds the second hand. , The information on the shape and movement of the second hand sensed by the flex film surrounding the second hand may be transmitted to the rehabilitation content providing device through wired or wireless communication.

In addition, the operation of generating the virtual body image in the virtual space may include generating a plurality of 3D location information constituting the shape and motion information of the second hand based on information obtained from the wearable device; Generating a plurality of virtual location information obtained by making each of the plurality of 3D location information plane-symmetric; And generating the virtual body image by 3D rendering the plurality of virtual location information.

An apparatus for providing rehabilitation content according to an embodiment of the present invention includes: a display unit for outputting an image; One or more memories storing instructions for performing a predetermined operation; And one or more processors connected to be operable with the one or more memories and configured to execute the instructions, wherein the one or more processors include a map consisting of a plurality of fields through which a character in a virtual space passes from a starting point to a destination. Storing the rehabilitation content; Outputting the plurality of fields to a virtual space as the rehabilitation content is played; Acquiring the shape and movement of the second hand according to the movement of the patient from the wearable device worn on the second hand, which is a body part symmetric with the first hand corresponding to the patient's amputated or paralyzed body part; Generating, in the virtual space, a virtual body image having a shape and movement of the second hand and a shape and movement in which the left and right sides are reversed based on the obtained information; Outputting a list of coding blocks that control the motion of the character to the virtual space, and generating a coding block selected by the patient; Arranging the coding block in any one of the plurality of fields based on the motion of the virtual body image that is inverted and moved with respect to the motion of the second hand; And an operation of controlling the character to perform an operation according to coding information mapped to a field passed from the starting point upon completion of the arrangement of the coding blocks for the plurality of fields.

According to an embodiment of the present invention, a virtual body may be implemented at a part of the patient's body amputated, and the implemented virtual body may be implemented to have a movement symmetrical to the movement of the uncut body. Accordingly, the patient consciously considers that both hands or both feet perform the same motion, and visually recognizes the movement of the virtual body implemented through virtual reality, so that the treatment of a concept such as mirror therapy can be performed.

Accordingly, the patient can move the virtual body like an actual arm and leg through the recognition that the behavior of both bodies moves equally, and can visually perceive the movement of the body in a realistic sense, thereby alleviating the phantom pain. And the effectiveness of rehabilitation training can be improved.

In addition, since the patient perceives a virtual body image in front of his/her eyes at the same viewpoint as the real life through the HMD (Head Mounted Display) device, the patient can perceive it vividly from a natural viewpoint like a real body.

In addition, the movement of the patient can be implemented so that the wearable device composed of the FLEX film senses the detailed movement of the actual patient's arm or leg, so that the movement of the virtual body more accurately reflects the movement of the actual body.

In addition, the patient uses hands, arms, legs, etc., which are virtually created using virtual body images displayed in a virtual space through the HMD device, and in each field the character passes through until the given task is completed. By arranging the required coding blocks to solve the problem, it is possible to provide treatment content that combines the elements of interest to the patient during the treatment process.

In addition to this, various effects that are directly or indirectly identified through this document can be provided.

1 is a block diagram of a system for providing rehabilitation content according to an embodiment of the present invention.
2 is a functional block diagram of an HMD device according to an embodiment of the present invention.
3 is a flowchart of a method of providing rehabilitation content using a virtual implanted hand performed by an HMD device according to an embodiment of the present invention.
4 is an exemplary diagram of a map composed of a plurality of fields output to a virtual space according to an embodiment of the present invention.
5 is an exemplary view showing a state of a patient wearing an HMD device according to an embodiment of the present invention.
6 is an exemplary diagram showing an operation of using an HMD device according to an embodiment of the present invention.
7 is a diagram illustrating an example of wearing a wearable device on a hand according to an embodiment of the present invention.
8 is an exemplary diagram of a list including a plurality of coding blocks output to a virtual space according to an embodiment of the present invention.
9 is an exemplary diagram of an operation of selecting a coding block by outputting the position and movement of a virtual body image in a virtual space according to an embodiment of the present invention.
10 is an exemplary diagram of an operation in which a virtual body image places or throws a coding block in a field by reflecting a moving direction and speed of a virtual body image according to an embodiment of the present invention.
11 is an exemplary diagram of an operation in which coding information of a coding block is mapped to a field when a coded block placed or thrown reaches a field according to an embodiment of the present invention.
12 is an illustration of an operation in which a character attacks a monster as an example in the middle of performing an operation according to coding information arranged by a patient according to an embodiment of the present invention.
13 is a flowchart of a method for providing rehabilitation content using a virtual implanted lower body performed by an HMD device according to an embodiment of the present invention.
14 is a diagram illustrating an example of wearing a wearable device on a lower body according to an embodiment of the present invention.

Advantages and features of the present invention, 　, and a method of achieving them will become apparent with reference to the embodiments described later in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms. It is provided to fully inform the person of the scope of the invention, and the scope of the invention is only defined by the claims.

In describing the embodiments of the present invention, detailed descriptions of known functions or configurations will be omitted except when actually necessary. In addition, terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to the intention or custom of users or operators. Therefore, the definition should be made based on the contents throughout the present specification.

The functional blocks shown in the drawings and described below are only examples of possible implementations. In other implementations, other functional blocks may be used without departing from the spirit and scope of the detailed description. In addition, although one or more functional blocks of the present invention are represented as individual blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

In addition, the expression to include certain elements is an open expression, merely refers to the existence of the corresponding elements, and should not be understood as excluding additional elements.

Furthermore, when a component is connected to or is referred to as being connected to another component, it should be understood that it may be directly connected or connected to the other component, but other components may exist in the middle.

In addition, expressions such as'first, second', etc. are used only for distinguishing a plurality of elements, and do not limit the order or other features between the elements.

The rehabilitation content providing system 10 according to the embodiment of the present invention is a treatment concept of mirror therapy that reduces the illusion of pain felt at the cut area by causing the patient to make an illusion with a visual effect that the body of the cut area moves without pain. While maintaining, by implementing a treatment method that allows the patient to perform various movements without a mirror, it provides a technology that contributes to enhancing the effect of treatment and rehabilitation training to relieve the phantom pain of a limb amputation patient. In addition, by providing content that utilizes the features of VR/AR technology in addition to the above-described treatment technology, VR/AR content that allows patients to feel interest while receiving the feeling that they use their arms and legs realistically in the treatment process is provided. to provide.

The rehabilitation content providing system according to an embodiment of the present invention implements a virtual body image on the part where the patient's body is cut through VR (Virtual Reality) or AR (Augmented Reality) technology, and the implemented virtual body image is It is implemented to have a movement that is symmetrical to the movement of an uncut body, so that VR/AR contents can be played. To this end, components included in the rehabilitation content providing system according to an embodiment of the present invention and operations of each component will be described together with the drawings.

1 is a block diagram of a system for providing rehabilitation content according to an embodiment of the present invention.

Referring to FIG. 1, a system for providing rehabilitation content according to an embodiment of the present invention includes a wearable device 100, a rehabilitation content providing device 200 (ex. HMD device 200 ), and a tracker 300. ) Can be included. In addition, the system according to an embodiment of the present invention may further include a depth camera 400 or an embodiment in which the wearable device 100 is replaced with the depth camera 400.

The rehabilitation content providing device 200 may be implemented through the HMD device 200. The HMD device 200 may be worn like glasses around the patient's eye, and a device that outputs an image including virtual information to the patient. to be. In the following description, for convenience of understanding, the apparatus 200 for providing rehabilitation content will be referred to as “HMD device”. However, the implementation of the rehabilitation content providing apparatus 200 is not necessarily limited to the form of an "HMD device", and may be implemented as various types of devices including a configuration to be described with reference to FIG. 2.

The wearable device 100 is a device that is attached or worn on a body to perform wired or wireless communication with an external device. The wearable device 100 may include motion capture gloves, VR smart gloves, and motion recognition gloves, and may be worn on parts of the body such as hands, arms, feet, and ankles of a patient. For example, the wearable device 100 is a body part (hereinafter referred to as "second body 12") that is symmetrical with a part of the patient's amputated or paralyzed body (hereinafter, referred to as "first body 11"). ), and the wearable device 100 may generate information that senses the shape and movement of the second body 12 according to the movement of the patient.

For example, the wearable device 100 may include a flex film capable of sensing a change in shape of a surface through a resistance value that changes as the surface increases or decreases. When the second body 12 corresponds to a "hand or arm", the flex film is configured in a shape corresponding to the second body 12 so as to surround the second body 12 (ex. FIG. 7 reference numerals) 110), and the wearable device 100 generates information on the shape and movement of the second body 12 sensed by the flex film surrounding the second body 12, and the HMD device ( 200).

In addition, when the second body 12 corresponds to the “lower body”, referring to FIG. 14, the wearable device 100 has both ends connected to the upper and lower sides of the joint portion 12a of the second body 12. 2 The soft sensor 101 whose resistance value changes according to the movement of the joint part 12a of the body 12, the adhesive part 102 and the adhesive part 102 provided at both ends of the soft sensor 101 and adhered to the patient's body ) May include an actuator 103 that is provided on one side of the second body 12 and applies vibration to the second body 12. Accordingly, when the patient moves the joint portion 12a of the second body 12, information on the movement of the joint portion 12a is generated through a change in the resistance value of the soft sensor 101, and the HMD device through wired or wireless communication. It can be transmitted to (200).

The tracker 300 is a physical mark or a wearable band displayed so that the HMD device 200 can specify one end of the patient's first body 11 (ex. (Wearable Band) may be included. The wearable band 300 is worn on one end of the patient's first body 11 (ex. a wrist, wrist, ankle, knee, etc. connected to the cut body part), and a ligament that changes when the patient tries to move the cut body. The movement of the ligament is sensed to generate information on the movement of the ligament, and the movement of the cut part intended by the patient may be determined based on the information on the movement of the ligament.

The depth camera 400 is a camera that includes a plurality of camera modules and is capable of generating 3D depth information of an object by calculating a depth value of an image pixel obtained from an object. The depth camera 400 may photograph a patient's hand, arm, foot, ankle, and the like. For example, the depth camera 400 may photograph the second body 12, which is a body part symmetrical with the first body 11 corresponding to the severed or paralyzed body part of the patient, and 2 It is possible to generate information that senses the shape and movement of the body 12.

2 is a functional block diagram of an HMD device 200 according to an embodiment of the present invention. Referring to FIG. 2, the HMD device 200 according to the embodiment of the present invention may include a memory 210, a processor 220, an input interface 230, a display unit 240, and a communication interface 250. have.

The memory 210 may store information and commands for performing the embodiments, including the rehabilitation content DB 211 and the command DB 215.

The rehabilitation content DB 211 may store rehabilitation content including a map composed of a plurality of fields that a character in the virtual space passes through to go from a starting point to a destination. The rehabilitation content DB 211 may store a previously generated map in the form of a file, and the map is a 3D virtual space or 3D that is output on the display unit 240 as the rehabilitation content according to an embodiment of the present invention is played. As an object, an example thereof is shown in FIG. 4.

The command DB 215 may store a command for performing an operation of the processor 120 to be described later, or a computer code for performing an operation corresponding to the operations of the processor 220 to be described later.

The processor 220 may control components included in the HMD device 200, the memory 210, the input interface 230, the display unit 240, and the overall operation of the communication interface 250. The processor 220 may include a content module 221, a sensing module 223, an operation module 225, and a control module 227. The processor 220 may drive the content module 221, the sensing module 223, the operation module 225, and the control module 227 by executing instructions stored in the memory 210. An operation performed by the content module 221, the sensing module 223, the operation module 225, and the control module 227 may be understood as an operation performed by the processor 220.

The content module 221 may load data stored in the rehabilitation content DB 211 and output it to the display unit 240. For example, the content module 221 may configure a map composed of a plurality of fields stored in the rehabilitation content DB 211 in a virtual space and output it to the display unit 240.

The sensing module 223 may refine and manage data so that the data acquired from an external device (eg, the wearable device 100, the tracker 300, the depth camera 400, etc.) can be used according to the purpose.

The calculation module 225 provides a control according to the movement of the patient from the wearable device 100 worn on the second body 12, which is a body part symmetrical to the first body 11 corresponding to the patient's amputated or paralyzed body part. 2 Using the sensed information on the shape and movement of the body 12, a virtual body image 31 having the shape and movement of the second body 12 and the shape and movement in which the left and right are inverted is generated in a virtual space. It can be output to the display unit 240.

The control module 227 may control the play of the rehabilitation content according to the patient's input or the movement of the virtual body image 31.

The input interface 230 may acquire a patient's input (eg, physical input, gesture input, or voice input). The input interface 130 may include a hardware module such as a touch sensor, a voice sensor, and a camera.

The display unit 240 may include a hardware configuration that outputs a virtual body image 31 including a display panel and an image including a virtual space (eg, FIGS. 9 to 12 ).

The communication interface 250 communicates with an external device (eg, the wearable device 100, the tracker 300, the depth camera 400, etc.) to transmit and receive information. To this end, the communication interface 250 may include a wireless communication module or a wired communication module.

Hereinafter, a flow of operations for performing the embodiments by interlocking the components of the rehabilitation content providing apparatus 200 according to an embodiment of the present invention will be described with reference to FIGS. 3 to 14.

3 is a flowchart of a method of providing rehabilitation content using a virtual implanted hand performed by the HMD device 200 according to an embodiment of the present invention. Each step of the method for providing rehabilitation content according to FIG. 3 may be performed by the configurations of the HMD device 200 described with reference to FIG. 2, and each step will be described below.

The rehabilitation content DB 211 may store rehabilitation content including a map composed of a plurality of fields that a character in the virtual space passes through to go from a starting point to a destination (S310).

As the rehabilitation content is loaded, the content module 221 may generate a map composed of a plurality of fields that a character in the virtual space passes through to go from a starting point to a destination and output it to the display unit 240 (S320). The map is a 3D virtual space or a 3D object that is output to the display unit as rehabilitation content according to an embodiment of the present invention is played, and an example thereof is shown in FIG. 4.

4 is an exemplary diagram of a map composed of a plurality of fields output to a virtual space according to an embodiment of the present invention.

Referring to FIG. 4, a field constituting a map may be configured in various forms so that a character must perform a specific action to pass through. For example, the type of field is the starting point field from which the character departs, the first field that can be passed through by correctly setting the direction of the character, the second field where the game is over when the character passes, and the third field with monsters. , And a destination field, which is a destination to which the character should arrive.

The content module 221 may output a pre-stored map, or at least two fields of the same type or different types among various types of fields (ex. the first field, the second field, and the third field). You can create a map by selecting more than one field. In this case, the field may have a rectangular shape, for example, and a map may be generated by configuring a field connection such that at least one other selected field is connected to one side of one of the selected fields. For example, the content module 221 may connect the "departure field-first field-second field-third field-destination field" in a straight line like the map configuration illustrated in FIG. 4. In addition, the content module 221 may generate a map such that some fields are connected in a form that is bent by 90 degrees, like the map configuration illustrated by reference numeral 51 of FIG. 12.

As the rehabilitation content is loaded, the content module 221 forms a 3D virtual space and arranges the map stored in the rehabilitation content DB 211 as an object in the virtual space. Image information including) may be generated and output to the display unit 240. The content module 221 may arrange a virtual camera that generates image information viewed from a point in the 3D virtual space as a first-person view or a third-person view at a preset point in the virtual space. The virtual camera may generate image information of a first-person view or a third-person view that looks in one direction at a predetermined angle of view from a point arranged in a virtual space. The content module 221 may output image information (eg, FIGS. 9 to 12) generated by controlling a focal length or a projection method of the virtual camera to the display unit 240.

The sensing module 223 is a patient from the wearable device 100 worn on the second hand 12, which is a body part symmetrical to the first hand 11 corresponding to the severed or paralyzed body part of the patient playing rehabilitation content. The shape and movement of the second hand 12 according to the movement of may be acquired (S330).

5 is an exemplary view showing a state of a patient wearing the HMD device 200 according to an embodiment of the present invention.

Referring to FIG. 5, the patient is in a state in which the first hand 11 is amputated. Accordingly, in order to play the rehabilitation content according to an embodiment of the present invention, the patient may wear the wearable device 100 on the second hand 12. In addition, the patient may wear the HMD device 200 around the eyeball. In order to specify the location of the cut part of the body, the patient may mark a mark as a tracker 300 near the distal end of the first hand 11 (near 300 in FIG. 5 ), or wear a wearable band 300. I can.

The calculation module 225 generates a virtual body image 31 having a shape and movement of the second hand 12 and a shape and movement in which the left and right are inverted based on the information obtained from the wearable device 100 in a virtual space. It can be done (S340).

6 is an exemplary diagram showing an operation of using the HMD device 200 according to an embodiment of the present invention.

Referring to FIG. 6, the HMD apparatus 200 may generate a virtual body image 31 having a body part that is symmetrical with a cut body part of a patient and a movement in which the left and right sides are inverted.

According to an embodiment, the sensing module 223 may generate a plurality of 3D location information constituting the shape and motion information of the second hand 12 based on information acquired from the wearable device 100. . Accordingly, the calculation module 225 may generate a plurality of virtual location information obtained by inverting the left and right of the plurality of 3D location information. For example, the calculation module 225 may generate a plurality of virtual location information in which the left and right sides are reversed by plane-symmetrical each of a plurality of 3D location information based on a predetermined 3D surface. Alternatively, when the plurality of three-dimensional position information is composed of three-dimensional coordinates (x, y, z), the three-dimensional coordinates (-x, y, z) obtained by inverting the sign of any one of the three-dimensional information ), you can create virtual location information. Thereafter, the calculation module 225 may generate a virtual body image 31 that can be visually output to a VR or AR image by 3D rendering the plurality of virtual location information.

Referring to Step 1 to Step 3 of FIG. 6, the HMD device 200 may output a virtual body image 31 at the distal end of the patient's first body 11 together with a VR image or an AR image.

According to an embodiment, the calculation module 225 may connect the virtual body image 31 to the patient's body. To this end, the calculation module 225 may specify a first point where the tracker 300 worn or displayed at a predetermined point among the first hand 11 of the patient is located through wired/wireless communication of the communication interface 250. Thereafter, in the virtual body image 31, the calculation module 225 determines the boundary between the worn body part (12 in FIG. 5) and the unworn body part (13 in FIG. 5) in the virtual body image 31. The position of the second point corresponding to the point (reference numeral a in FIG. 5) can be specified. Accordingly, the processor 220 may output the virtual body image 31 to the display unit 240 so that the second point of the virtual body image 31 is connected to the first point of the patient.

In addition, when the tracker 300 is the wearable band 300, the sensing module 223 may determine a motion intended by the patient to move the severed body based on the information acquired from the wearable band 300. To this end, the wearable band 300 outputs a corresponding resistance value according to the shape deformation of the patient's wrist or ankle ligament along the body part surrounding the wrist or wrist of the patient, and the body part contacting the body. A sensor unit including a plurality of strain gauges, and a processor configured to determine a shape of a motion intended by a patient based on patterns of resistance values of the plurality of strain gauges recognized by the sensor unit. The calculation module 225 may reflect and output a motion corresponding to a virtual body image based on the shape of the motion determined by the wearable band 300. Accordingly, the patient can visually check the movement of the virtual body image 31 according to the movement of the ligaments even when the body is amputated.

In addition, as an embodiment, the operation module 225 is a virtual body image 31 that is symmetric with the area of the body on which the wearable device 100 is worn, in a form that is not connected to the body. Only the body image 31 may be generated and output in a virtual space.

The content module 221 may output a list of coding blocks that control a character's motion to a virtual space, and generate a coding block selected by a patient (S350). In this case, the coding block may be selected based on a motion of the patient's voice or a virtual body image (eg, a pressing motion, a grabbing motion, etc.).

8 is an exemplary diagram of a list 10 including a plurality of coding blocks output to a virtual space according to an embodiment of the present invention.

Referring to FIG. 8, the coding block may include coding information for causing a character to perform a predetermined operation. For example, the coding information includes first coding information 11 that advances the character to the field in front, second coding information 12 that jumps over the field in front by jumping the character, and third coding that rotates the character to the left. It may include information 13, fourth coding information 14 for rotating the character to the right, and fifth coding information 15 for removing obstacles in front of the character by attacking a monster in front. In addition, the sixth coding information 16 for changing the item mapping rule for a field so that a plurality of coding blocks can be mapped to one field while there is a rule that basically one coding block is arranged in one field. It may include. In addition, the list 10 including the coding blocks may include a map switch button 17, a restart button 18, and a code execution button 19 according to completion of the arrangement of the coding blocks.

The control module 227 may arrange the coding block in any one of the plurality of fields based on the motion of the virtual body image 31.

9 is an exemplary diagram of an operation of selecting the coding block 21 by outputting the position and movement of a virtual body image 31 in a virtual space according to an embodiment of the present invention.

Referring to FIG. 9, the control module 227 may generate a designated coding block 21 at a location near the virtual body image 31 according to a patient's selection for the list 10. For example, the control module 227 outputs a list 10 including a plurality of coding blocks to a part of the virtual space, and selects the patient's voice or virtual body image 31 for the list 10 A coding block selected by the /grab motion may be generated at a location near the virtual body image 31. Since the virtual body image 31 moves symmetrically with the movement of the first hand 11 wearing the wearable device 100, the selection motion or the grabbing motion is the second hand 12 wearing the wearable device 100 Can be identified according to the movement of the

The control module 227 may place the coding block in any one of the plurality of fields based on the motion of the virtual body image 31 (S360).

10 is an example of an operation in which the virtual body image 31 places or throws the coding block 21 on the field 41 by reflecting the moving direction and speed of the virtual body image 31 according to an embodiment of the present invention. It is a degree.

Referring to FIG. 10, the sensing module 223 acquires information on the direction and speed of movement of the patient's hand from the wearable device 100, and the calculation module 225 The direction in which the coding block 21 should move as the motion of the body image 31 is generated in a virtual space to implement a motion of placing or throwing the coding block 21, and as it is determined that the virtual body image has taken a throwing or releasing motion. And by calculating the speed, the coding block 21 may implement and output a form that moves in a virtual space.

According to the above process, when it is determined that the virtual body image 31 has taken a throwing action, or when it is determined that it is taking an action of opening and releasing a hand in a specific position in a virtual space, or a hand that is grasped while taking a throwing action When it is determined that the shape has changed to the shape of an open hand, the calculation module 225 determines that the patient has taken a motion of throwing or releasing the coding block 21, and the sensing module 223 determines the direction and speed of the patient's hand. , And coordinates on a virtual space terminated by performing a throwing or releasing motion by the patient (=location information in the virtual space where the coding block started to leave the virtual body image) may be obtained. The arithmetic module 225 performs a motion in which the patient throws or releases the coding block 21, so that the coding block 21 moves from the coordinates where the patient's hand is separated, toward the direction in the virtual space in which the virtual body image has moved. With the speed in the virtual space proportional to the speed, the coding block 21 is calculated to move by receiving gravity and air resistance according to the physical laws (ex.gravity acceleration set in the virtual space, air resistance set in the virtual space), The movement of the inner coding block 21 can be implemented.

The control module 227 compares the coordinate information of the coding block 21 according to the movement of the coding block 21 in the virtual space with the coordinate information in which a plurality of fields are arranged in the virtual space, and the information of the coding block 21 You can perform the following operation of mapping to a field.

11 is an exemplary diagram of an operation in which the coding information of the coding block 21 is mapped to the field 41 when the coded block 21 placed or thrown reaches the field 41 according to an embodiment of the present invention.

Referring to FIG. 11, when it is determined that the coding block 21 has reached the position of any one of a plurality of fields according to the movement of the coding block 21, the control module 227 Coding information included in the coding block 21 may be mapped to one field 41.

According to an embodiment of the present invention, basically, one coding block may be arranged in one field. Accordingly, when a character reaches a specific field, the character performs an operation according to one coding block included in the field. Meanwhile, the sixth coding information described above basically changes a basic rule for a field that one coding block can be arranged in one field, so that a plurality of coding blocks can be mapped to one field. That is, the sixth coding information may change the above-described basic rule by performing a role similar to the conditional statement.

12 is an exemplary diagram of an operation in which a plurality of coding blocks are mapped to one field and a character executes a plurality of commands in one field according to an embodiment of the present invention.

Referring to FIG. 12, when determining that the coding block including the sixth coding information is disposed in a specific field, the control module 227 changes the state so that a plurality of coding blocks can be mapped to the corresponding field, and the patient May map information on a plurality of coding blocks to a corresponding field so that the plurality of coding blocks are executed in the order in which they are input, as a plurality of coding blocks are thrown into the corresponding field. Accordingly, when a character passes through the corresponding field, the operation of the character in one field may be controlled in the order in which a plurality of coding blocks are mapped. (if input of field A = 6th coding block, output of field A = all execution when characters reach field A in the order of input of a plurality of coding blocks input after the 6th coding block)

Among the fields of the code progress information 51 of FIG. 12, there are fields to which two coding information are mapped, and this means that after the patient maps the coding block according to the sixth coding information to the corresponding field, a plurality of coding blocks are sequentially performed. It was mapped. Accordingly, in the field in which "Turn left, Forward" is mapped among the fields of FIG. 12, the character may enter the field bent by 90 degrees by performing an operation of "turning to the left and moving forward". In addition, in the field where "Attack, Forward" is mapped, the character performs an action of "attacking and moving forward" to defeat the monster and enter the next field.

That is, the sixth coding information may basically play a role similar to a conditional statement for changing a basic rule for a field in which one coding block can be arranged in one field.

The control module 227 may control to perform an operation according to the coding information mapped to the field that the character passes through from the starting point according to the completion of the arrangement of the coding blocks for the plurality of fields (S370).

The control module 227 may output code progress information (eg, reference numeral 51 of FIGS. 9 to 12) about an execution order of coding information mapped to a plurality of fields to a virtual space. Referring to the code progress information 51 of FIG. 12, a plurality of fields constituting a map may refer to functions, and an order in which the fields are connected may refer to an order in which functions are executed. In addition, the coding information inside each field refers to an instruction statement to be executed inside the function, and the position of a character may refer to an order in which the code is executed and a position in which the code is executed.

On the other hand, in FIGS. 3 to 12 described above, an embodiment of using rehabilitation content based on a patient whose upper limb is paralyzed or amputated has been described, but according to the embodiments of FIGS. 13 and 14, a patient whose lower body is amputated or paralyzed has been described. Rehabilitation content can also be applied.

13 is a flowchart of a method of providing rehabilitation content using a virtual implanted lower body performed by the HMD device 200 according to an embodiment of the present invention. Each step of the method for providing rehabilitation content according to FIG. 13 may be performed by the configurations of the HMD device 200 described with reference to FIG. 2, and each step will be described below.

The content DB 211 may store and load rehabilitation content including a map composed of a plurality of fields that a character in the virtual space passes through to go from a starting point to a destination (S1310).

As the rehabilitation content is played, the content module 221 may generate a map composed of a plurality of fields that a character in the virtual space passes through to go from a starting point to a destination and output it to the display unit 240 (S1320). The sensing module 223 is a patient from the wearable device 100 worn on the second lower body 12, which is a body part symmetrical to the first lower body 11 corresponding to the severed or paralyzed body part of the patient playing rehabilitation content. The shape and movement of the second lower body 12 according to the movement of may be acquired (S1330).

14 is a view showing an example of wearing the wearable device 100 on the lower body according to an embodiment of the present invention.

Referring to FIG. 14, the wearable device 100 has both ends connected to the upper and lower joints 12a of the second lower body 12 and resists according to the movement of the joints 12a of the second lower body 12. The soft sensor 101, which has a variable value, is provided at both ends of the soft sensor 101 and is attached to the patient's body, and is provided on one side of the adhesive part 102 and the second lower body 12 is applied with vibration. It may include an actuator 103. Accordingly, when the patient moves the joint portion 12a of the second lower body 12, the HMD device generates information on the movement of the joint portion 12a through a change in the resistance value of the soft sensor 101 It can be transmitted to (200).

The calculation module 225 generates a virtual body image 31 having a shape and movement of the second lower body 12 and a shape and movement in which the left and right are inverted based on the information obtained from the wearable device 100 in a virtual space. It can be done (S1340).

Thereafter, the content module 221 may output a list of coding blocks that control the motion of the character to the virtual space, and generate a coding block selected by the patient (S1350). In this case, the coding block may be generated on the floor of the virtual space.

The control module 227 may place the coding block in any one of the plurality of fields based on the motion of the virtual body image 31 (S1360). At this time, the patient can kick the coding block while wearing the wearable device 100, and accordingly, the movement symmetrical with the second lower body 12 is implemented as a virtual body image 31, An operation of kicking the coding block generated on the floor may be performed.

When it is determined that the virtual body image 31 has taken a kicking motion, the sensing module 223 uses the direction and speed of the patient's lower body to move, and the coordinates in the virtual space where the patient's body image and the coding block are in contact with each other (= coding. It is possible to obtain information on the location information of the point occupied by the patient by the block in the virtual space). The calculation module 225 is a virtual space in which the virtual body image 31 is moved from the coordinates where the coding block 21 starts to contact the virtual body image 31 by performing a motion in which the patient kicks the coding block 21. At the speed in the virtual space proportional to the moving speed toward the inside direction, the coding block 21 receives gravity and air resistance according to the laws of physics (ex.gravity acceleration set in the virtual space, air resistance set in the virtual space). By calculating to move, it is possible to implement the movement of the coding block 21 in the virtual space. The control module 227 compares the coordinate information of the coding block 21 according to the movement of the coding block 21 in the virtual space with the coordinate information in which a plurality of fields are arranged in the virtual space, and the information of the coding block 21 Mapping to a field can be performed.

The control module 227 may control an operation according to coding information mapped to a field that the character passes through from the starting point according to the completion of the arrangement of the coding blocks for the plurality of fields (S1370).

Meanwhile, in the embodiment of FIG. 13, the contents of the operation different from the implementation in the embodiment of FIG. 2 are described, and implementations other than the operation described in FIG. 13 overlap with the contents described in the description of FIGS. 3 to 12. Duplicate content will be omitted.

According to the above-described embodiment, a virtual body may be implemented at a part of the patient's body amputated, and the implemented virtual body may be implemented to have a movement symmetrical to a movement of an uncut body. Accordingly, the patient consciously considers that both hands or both feet perform the same motion, and visually recognizes the movement of the virtual body implemented through virtual reality, so that the treatment of a concept such as mirror therapy can be performed.

Accordingly, the patient can move the virtual body like an actual arm and leg through the recognition that the behavior of both bodies moves equally, and can visually perceive the movement of the body in a realistic sense, thereby alleviating the phantom pain. And the effectiveness of rehabilitation training can be improved.

In addition, since the patient perceives a virtual body image in front of his/her eyes at the same time point as in real life through the HMD device 200, the patient can perceive it vividly from a natural point of view like a real body.

In addition, the movement of the patient is implemented so that the wearable device 100 composed of the FLEX film senses the detailed movement of the actual patient's arm or leg, so that the movement of the virtual body more accurately reflects the movement of the actual body. I can.

In addition, the patient completes a given task while using a virtual hand, arm, leg, etc. in a realistic way by using a virtual body image (ex. hand or lower body) displayed in a virtual space through the HMD device 200. By arranging the coding blocks required in each field passed by the character to solve the problem, it is possible to provide treatment contents that combine the elements of interest to the patient in the treatment process.

The above-described embodiments of the present invention can be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of implementation by hardware, the method according to embodiments of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), and Programmable Logic Devices (PLDs). , Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the function or operation described above. A computer program in which a software code or the like is recorded may be stored in a computer-readable recording medium or a memory unit and driven by a processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor through various known means.

Also, combinations of each block of the block diagram and each step of the flowchart attached to the present invention may be performed by computer program instructions. Since these computer program instructions can be mounted on the encoding processor of a general purpose computer, special purpose computer or other programmable data processing equipment, the instructions executed by the encoding processor of the computer or other programmable data processing equipment are each block of the block diagram or Each step in the flow chart will create a means to perform the functions described. These computer program instructions can also be stored in computer-usable or computer-readable memory that can be directed to a computer or other programmable data processing equipment to implement a function in a particular way, so that the computer-usable or computer-readable memory It is also possible to produce an article of manufacture that includes instruction means for performing the functions described in each block of the block diagram or each step of the flowchart. Since computer program instructions can also be mounted on a computer or other programmable data processing equipment, a series of operating steps are performed on a computer or other programmable data processing equipment to create a computer-executable process to create a computer or other programmable data processing equipment. It is also possible for the instructions to perform the processing equipment to provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

In addition, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function. It should also be noted that in some alternative embodiments, functions mentioned in blocks or steps may occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially simultaneously, or the blocks or steps may sometimes be performed in the reverse order depending on the corresponding function.

As such, those skilled in the art to which the present invention pertains will be able to understand that the present invention can be implemented in other specific forms without changing the technical spirit or essential features thereof. Therefore, the embodiments described above are illustrative in all respects and should be understood as non-limiting. The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. .

Claims (10)

In the rehabilitation content providing method performed by the rehabilitation content providing device,
Storing rehabilitation content including a map composed of a plurality of fields that a character in the virtual space passes through to go from a starting point to a destination;
Outputting the plurality of fields to a virtual space as the rehabilitation content is played;
Acquiring the shape and movement of the second hand according to the movement of the patient from the wearable device worn on the second hand, which is a body part symmetrical to the first hand corresponding to the patient's amputated or paralyzed body part;
Generating, in the virtual space, a virtual body image having a shape and movement of the second hand and a shape and movement in which the left and right sides are reversed based on the obtained information;
Outputting a list of coding blocks that control the motion of the character to the virtual space and generating a coding block selected by the patient;
Arranging the coding block in any one of the plurality of fields based on the motion of the virtual body image; And
Comprising an operation of controlling the character to perform an operation according to the coding information mapped to the field to be passed through from the starting point according to the completion of the arrangement of the coding blocks for the plurality of fields,
How to provide rehabilitation content.
The method of claim 1,
The operation of generating the coding block,
Outputting a list including a plurality of coding blocks to the virtual space; And
Comprising an operation of generating a coding block selected by the grabbing operation of the virtual body image for the list in the virtual body image,
How to provide rehabilitation content.
The method of claim 1,
The operation to be placed in the field,
Detecting a motion of the virtual body image placing or throwing the coding block;
Calculating a direction and speed at which the coding block is thrown based on a direction and speed of the virtual body image, and outputting a motion of the coding block in the virtual space; And
When it is determined that the coding block has reached any one of the plurality of fields according to the movement of the coding block, an operation of mapping coding information on an operation corresponding to the coding block to the one of the fields is performed. Included,
How to provide rehabilitation content.
The method of claim 1,
The plurality of fields,
A first field through which a character can pass; A second field in which the game is over when the character passes; And a third field in which a monster is present,
The operation of outputting the plurality of fields to a virtual space,
Selecting at least two or more fields from among the first field, the second field, and the third field; And
Generating the map so that at least one other selected field is connected to one side of one of the selected fields and outputting the map to the virtual space,
How to provide rehabilitation content.
The method of claim 1,
The coding block,
First coding information for advancing the character; Second coding information for jumping the character to jump over a field in front; Third coding information for rotating the character to the left; Fourth coding information for rotating the character to the right; Fifth coding information for removing a monster in front of the character by performing an attack; And sixth coding information for changing an item mapping rule for one field so that a plurality of coding blocks can be mapped to one field,
The controlling operation,
When the character passes through a field in which a plurality of coding blocks are mapped to one field by changing a mapping rule according to the sixth coding information, the operation of the character is controlled in the order in which the plurality of coding blocks are mapped. Including the action,
How to provide rehabilitation content.
The method of claim 1,
The method of providing rehabilitation content,
After the operation of placing in the field, according to the arrangement of coding blocks for the plurality of fields, an operation of outputting code progress information on an order in which coding information mapped to the plurality of fields is executed to the virtual space is further performed. Included,
How to provide rehabilitation content.
The method of claim 1,
The wearable device,
A flex film capable of sensing the shape of the surface through a resistance value that changes as the surface increases or decreases is configured in a shape corresponding to the second hand so as to surround the second hand. Transmitting information on the shape and movement of the second hand sensed by the surrounding flex film to the rehabilitation content providing device through wired or wireless communication,
How to provide rehabilitation content.
The method of claim 7,
The operation of generating the virtual body image in the virtual space,
Generating a plurality of 3D location information constituting the shape and motion information of the second hand based on the information acquired from the wearable device;
Generating a plurality of virtual location information obtained by making each of the plurality of 3D location information plane-symmetric; And
3D rendering the plurality of virtual location information to generate the virtual body image,
How to provide rehabilitation content.
A computer program stored in a computer-readable recording medium for causing a processor to perform the method of providing rehabilitation content according to any one of claims 1 to 8.
In the rehabilitation content providing device,
A display unit that outputs an image; One or more memories storing instructions for performing a predetermined operation; And one or more processors connected to be operable with the one or more memories and configured to execute the instructions,
The one or more processors,
Loading rehabilitation content including a map composed of a plurality of fields that a character in the virtual space passes through to go from a starting point to a destination;
Outputting the plurality of fields to a virtual space according to loading of the rehabilitation content;
Acquiring the shape and movement of the second hand according to the movement of the patient from the wearable device worn on the second hand, which is a body part symmetrical to the first hand corresponding to the patient's amputated or paralyzed body part;
Generating, in the virtual space, a virtual body image having a shape and movement of the second hand and a shape and movement in which the left and right sides are reversed based on the obtained information;
Outputting a list of coding blocks that control the motion of the character to the virtual space and generating a coding block selected by the patient;
Arranging the coding block in any one of the plurality of fields based on the motion of the virtual body image that is inverted and moved with respect to the motion of the second hand; And
Comprising an operation of controlling the character to perform an operation according to the coding information mapped to the field to be passed through from the starting point according to the completion of the arrangement of the coding blocks for the plurality of fields,
Rehabilitation content providing device.

Images