KR20200105136A - Method for cognitive rehabilitation training and evaluating through roleplaying in story using virtual reality environment - Google Patents

Abstract

The present invention relates to a method for training and evaluating cognitive rehabilitation through role performance in a story using a virtual reality environment, which comprises the steps of: (a) preparing for cognitive rehabilitation training by role performance in a story using a virtual reality environment by a local computer; (b) performing cognitive rehabilitation training through role performance in the story through bidirectional communication with HMD and VR controller equipment which creates a virtual reality environment by the local computer as a user wears the same; (c) determining, by the local computer, whether or not a target is achieved according to the cognitive rehabilitation training; (d) by the local computer, providing the additional work selected in the step (a); (e) providing results by the local computer; and (f) by the local computer, confirming whether the user is terminated and terminating the content when the user selects the end, and then repeating the process after the step (a) when selecting the following, thereby improving participation by performing rehabilitation training with fun while understanding the story in the fairy tale, and also maximizing the efficiency of the rehabilitation training.

Description

Cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment {METHOD FOR COGNITIVE REHABILITATION TRAINING AND EVALUATING THROUGH ROLEPLAYING IN STORY USING VIRTUAL REALITY ENVIRONMENT}

The present invention relates to a cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment, and more specifically, extracting an item for training/evaluating the cognitive function execution ability from a story of a fairy tale. It is about cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment that is configured in and evaluates the result through the performance of a user wearing a virtual reality device.

The conventional cognitive rehabilitation training and evaluation method using virtual reality provides problems such as simple arithmetic operations to solve the problem by touching the place where the correct answer is started, or by providing a game such as catching a mole and rehabilitation through game execution. It has been done in a way of evaluating whether training is performed and whether the game score reaches the target score.

The conventional cognitive rehabilitation training and evaluation method using virtual reality as described above has a problem in that it is difficult to induce interest in the user because it contains somewhat learning content, and is thus easily abandoned.

Of course, the conventional cognitive rehabilitation training and evaluation method using virtual reality as described above also provides a game for cognitive rehabilitation training, but the content provided by the game is simple, so you cannot enjoy the game for a long time and lose interest. There is a problem that does not greatly help in training.

Republic of Korea Patent Publication No. 10-2017-0124978 (2017. 11. 13)

In order to solve the above-described problems, the present invention plays a role in the story while playing a traditional fairy tale story in a virtual reality space, thereby inducing the patient's comprehensive cognitive rehabilitation training. It aims to provide cognitive rehabilitation training and evaluation methods through performance.

Cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention for achieving the above object is: (a) A local computer performs cognitive rehabilitation training through performing a role in the story using a virtual reality environment. Preparing; (b) performing cognitive rehabilitation training by performing a role in a story through two-way communication with the HMD and VR controller equipment that creates a virtual reality environment as the user wears the local computer; (c) determining, by the local computer, whether a goal has been achieved according to the cognitive rehabilitation training; (d) providing, by the local computer, the additional task selected in step (a); (e) the local computer providing results; and (f) the local computer confirming whether the user is ended, ending the content when ending is selected, and then repeating the process after step (a) upon selection below.

Preferably, step (a) of the cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention to achieve the above object is (a-1) the local computer is the HMD and VR controller Confirming the connection of the equipment; (a-2) the local computer providing a content list to the connected HMD and receiving a user's content selection; (a-3) providing, by the local computer, a story proceeding method and a short proceeding method of the content selected by the user, and receiving a selected proceeding method according to an operation of the VR controller; And (a-4) when the user's selection of the progression method is the story progression method, the local computer providing precautions and progression methods.

Preferably, step (b) of the cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention for achieving the above object is (b-1) the content selected by the user by the local computer To start the story of; (b-2) the local computer presenting and confirming a goal to be performed in the process of proceeding with the story of the content selected by the user; (b-3) allowing the local computer to select the necessary tools to achieve the goal; (b-4) transmitting, by the local computer, an instruction to achieve the goal to a selected tool after moving the user to a location where an environment for achieving the goal is established; And (b-5) performing an action by the user to achieve a goal with the tool selected in step (b-3) using the VR controller.

Preferably, step (b) of the cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention for achieving the above object is (b-6) the situation determination generating unit of the local computer In the process of executing an action to achieve the goal, the step of directing a situation that may occur in the content to the user and allowing the user to determine the produced situation through a Q&A with the user; characterized by further comprising.

Preferably, step (b) of the cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention for achieving the above object is (b-7) the unexpected situation occurrence unit of the local computer The method further comprising: generating an unexpected situation that may occur in the content to the user in the process of executing an action to achieve the goal so that the user can overcome the unexpected situation by manipulation of the VR controller; do.

Preferably, the cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention to achieve the above object is the case where the selection of the user's progress method in the step (a-3) is the short progress method. It is characterized in that the local computer is provided to target and execute only actions or actions performed by characters in the content, regardless of the flow of the story.

Preferably, the cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention for achieving the above object is the process after step (a) according to the following selection in step (f). Prior to the repeating step, the user can change the target and the additional task in the selected content.

Cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention consists of a rehabilitation program for a specific person from a story in a fairy tale, and the rehabilitation therapist becomes the main character in the fairy tale By performing the rehabilitation exercise with fun while understanding the story in the fairy tale, there is an effect of improving participation and also maximizing the efficiency of the rehabilitation exercise.

1 is a diagram showing a cognitive rehabilitation training and evaluation system through performing a role in a story using a virtual reality environment according to the present invention.
2 is a block diagram of a local computer of a cognitive rehabilitation training and evaluation system by performing a role in a story using a virtual reality environment according to the present invention.
3 is a flow chart of a cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention.
Figure 4 is a flow chart embodied the preparatory steps of the cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention.
5 is a flow chart illustrating the steps of performing cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention.
6 to 9 are graphic diagrams for explaining a cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be described in detail with reference to the drawings. However, this is not intended to limit the present invention to a specific embodiment, it should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. The term and/or includes a combination of a plurality of related items or any of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected to or connected to the other component, but other components may exist in the middle. something to do. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

Throughout the specification and claims, when a certain part includes a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

Hereinafter, a method for evaluating cognitive and execution ability training using a virtual reality environment according to the present invention will be described in detail with reference to the accompanying drawings.

As shown in FIG. 1, the cognitive rehabilitation training and evaluation system by performing a role in a story using a virtual reality environment according to the present invention includes a head mounted display (HMD), a VR controller 200, and a local computer. 300, a communication network 400, and a content server 500.

The HMD 100 is a device worn by the user on the head and provides a virtual image screen.

The VR controller 200 interlocks with the HMD 100 and carries it in the user's hand to perform a task by selecting or using a tool on the screen provided by the HMD 100 according to manipulation.

The local computer 300 is an equipment having the same specifications as a desktop PC, and a story in a moving picture is reconstructed and stored as a rehabilitation program, and provides a virtual reality rehabilitation image to the HMD 100, which is a terminal worn by a user connected by wired or wirelessly. And display the work performed on the video.

In particular, the local computer 300 is a main control unit 310, an image providing unit 320, a communication unit 330, a voice providing unit 340, a target providing unit 350, as shown in FIG. It includes a study 360, a situation determination generation unit 370, an unexpected situation generation unit 380, and a database unit 390.

Hereinafter, as another embodiment, a cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment according to the present invention having the configuration as described above will be described in detail with reference to FIG. 3.

In particular, in order to help understanding in the detailed description of cognitive rehabilitation training and evaluation method through the role performance in the story using the virtual reality environment according to the present invention, a traditional fairy tale gold ax silver ax is described as an example.

First, the cognitive rehabilitation training and evaluation system by performing a role in a story using a virtual reality environment according to the present invention performs a step of preparing for talent rehabilitation (S100).

Referring to FIG. 4, the step (S100) will be described in more detail, the HMD 100 and the VR constituting the cognitive rehabilitation training and evaluation system through the role in the story using the virtual reality environment according to the present invention. When accessing the local computer 300 by wearing the controller 200, the local computer 300 performs a step of checking the connection of the devices (S110).

The control unit 310 of the local computer 300 provides a content list, such as gold ax, silver ax, Hansel and Gretel, stored in the content DB 391 of the database unit 390 to the connected HMD 100, and provides a user Performs the step of receiving the content selection (S120).

When the user selects content in the step (S120), the local computer 300 displays it as shown in FIG. 6A.

According to the manipulation of the VR controller 200 linked to the HMD 100, the content selected by the user is displayed together with a progress mode selection button, and a step of receiving a progress mode selection is performed (S130).

That is, as shown in FIG. 6B, the local computer 300 displays the menu buttons for story progress, short progress, and ending along with the content screen selected by the user on the HMD 100, and generates a sound effect when the button is selected.

In the case of a story progression after the step (S140) of determining whether the progress method selected by the user in the step (S130) is'talk progress' or'short progression', the main control unit 310 of the local computer 300 includes the selected content and In connection with it, a step of providing a description of precautions and a method of proceeding to the HMD 100 is performed (S150).

The above precautions may include training while sitting, organizing objects within a 1 meter radius around the user, prohibiting access, prohibiting excessive head rotation, and ending training when dizziness occurs.

As shown in Fig. 6c, the description of the method consists of selecting a menu or tool through a control operation in a virtual space, explaining the use of the tool, proceeding a story, and performing a short story. It includes an explanation to perform progress, an explanation to use a progress guide phrase when progress is difficult due to repeated errors, and an explanation that training can be applied continuously through short stories.

In the case where the story is not in progress in the step (S140), the local computer 300 performs a short progression (S150').

The short progression corresponds to training in which only specific actions or actions performed by the characters in the content are independently and fragmented, regardless of the flow of the story.

The short progression allows the user to train by intensively targeting and training specific actions or actions that were difficult to achieve goals after the user has previously performed the story process of the content, thereby reinforcing the actions or actions that the user was vulnerable to. have.

When the step (S100) of preparing human resource rehabilitation training is completed as described above, the local computer 300 performs a step of performing cognitive rehabilitation training (S200).

First, if the user checks the precautions and the progress method provided in the step (S150), the local computer 300 determines the user in (S120) as shown in FIG. 6D according to the progress method selected by the user in the step (S130). Performs the step of starting the story of the selected content (S210).

At this time, when the main control unit 310 of the local computer 300 extracts image data from the content DB 391, the image providing unit 320 transmits the image data to the HMD ( 100).

In addition, at this time, the main control unit 310 of the local computer 300 extracts voice data from the content DB 391, and the voice providing unit 340 transmits the voice data to the HMD through the communication unit 330. Provided to (100).

The local computer 300 performs a step of presenting and receiving confirmation of a goal that the user, who is a rehabilitation target, should perform today in the story in the process of proceeding with the story of the content selected by the user (S220).

That is, as shown in FIG. 7A, the target providing unit 250 of the local computer 300 provides the target stored in the target DB 392 of the database unit 390, and more specifically, the “gold ax” selected by the user As an example of the list of trees to be cut today in the "Silver Ax" content, the goal is to provide 4 oak trees, 4 silver trees, and 4 gold trees and receive confirmation.

The tool providing unit 360 of the local computer 300 provides tool data necessary for the user to achieve the goal suggested in the step (S220) from the tool DB 393 of the database unit 390, A step of receiving a user's selection is performed (S230).

That is, as shown in FIG. 7B, the user can select various tools other than an ax, which is a tool necessary for cutting trees with a virtual hand linked through manipulation of the VR controller 200, and the VR controller 200 With the control, you can change the position of the virtual hand to select the desired tool, and you can complete the tool selection by putting the virtual hand into the basket.

Of course, in the case of incorrect tool selection, the user can select another tool through manipulation of the VR controller 200.

Thereafter, while continuing the story, the local computer 300 presents a tool selection completion message so that the user who selects the tool can achieve the goal by using the tool, and then proceeds to the next screen moving to the forest.

At this time, the user performs a gait rehabilitation exercise in which at least one step or more is moved to move to the forest according to the screen change.

The local computer 300 performs a step of transmitting a tree-cutting command for achieving the target to the HMD 100 of the user (S240).

As shown in FIG. 7C, a cart is provided on the left, a forest (gold, silver, oak) on the front, and a pond on the right, thereby creating an environment in which an actual user can execute a task according to a tree cutting command.

The user who has received the tree-cutting command in step S240 performs the step of performing the tree-cutting with the tool selected in step S230 using the VR controller 200 (S250).

When the metal ax is swung horizontally to the side by using a metal ax among the tools selected in step (S230), the tree is cut.

The actual user performs a rehabilitation exercise by holding and swinging the VR controller 200 by hand.

In the present invention, an example of the content is a'gold ax silver ax', which applies the axing action to the rehabilitation exercise, and the VR controller according to the number and time of targets in the story according to the content type. Click (200) to perform the rehabilitation exercise, in the case of'look-ahead', the rehabilitation exercise is performed as a role-playing task that aims at the target point, and in the case of'move, go', the rehabilitation exercise is performed. Rehabilitation exercises are performed as a role performance task moving at a target speed.

At this time, the main control unit 310 of the local computer 300 scores for the accuracy of aiming in the case of'aiming at the target point' and moves to the'target point' including movement according to body movements such as head, arm, and leg. In the case of'Move', the accuracy of movement is scored, and in the case of'Move at the target speed', including the speed of movement of the body such as head, arms, and legs, the accuracy of the movement speed is scored, and a number of buttons on the controller In the case of'clicking the controller according to the number of targets and time', including duplicate or single use, it is desirable to evaluate the user's rehabilitation exercise by scoring the accuracy of the target number and time.

In this content, if you swing a total of 3 times, one tree is acquired while the sound effect is generated, and the display unit of the HMD 100 and the sound generated by shaking the tree, the touch controller, the vibration effect, etc. It is desirable to be able to implement it as a negative.

The user may perform a rehabilitation exercise in which the lower back is bent and stretched by additionally performing tree picking, etc., in which fallen trees are loaded on the left wagon while performing tree cutting in the step S250.

In addition, the situation determination generating unit 370 of the local computer 300 directs the situation in which the ax falls into the pond as shown in FIG. 7D in the process of the user cutting down the tree with an ax, and then makes the user determine the situation. Performs the step (S260).

That is, after directing the situation where the ax falls into the pond, as shown in Fig. 8A, as the activated pond is touched, a virtual mountain spirit appears, and the mountain spirit presents a question about the ax fallen into the pond and selects the tool selected according to the answer. Use the steps to command the re-execute tree cutting to achieve the goal.

It is preferable that the local computer 300 give an ax that fell into the pond and another ax if the answer is true according to the user's answer selection, and if it is false, give an ax that fell into the pond.

When the situation determination is completed, the sudden situation generation unit 380 of the local computer 300 provides a previous screen (forest screen) of cutting trees so that the tree cutting can be continued and the step of creating an unexpected situation is performed. Performed (S270).

Before that, it is desirable to notify the user in advance before the occurrence of the unexpected situation.As shown in FIG. , The user may swing the VR controller 200 to threaten and drive the thief, but if he does not recognize the appearance of the thief, the tree is stolen and it takes more time to achieve the goal.

The local computer 300 performs a step of determining whether the proposed target has been achieved by moving the screen to the step (S220) of presenting and confirming the target (S300).

In the step (S300), the local computer 300 switches the screen to the forest shown in FIG. 7C when the target is not reached according to the result of determining the target achievement, and repeats the process after step (S240). It is desirable to provide a congratulatory message.

When the target is achieved, the main control unit 310 of the local computer 300 provides an additional task that can be performed with the target achieved in the additional task stored in the additional task DB 393 of the database unit 390 It may be further performed (S400).

That is, as shown in FIG. 8D, the local computer 300 provides the creation of a chair consisting of a tree list, which is a target object acquired according to the goal achievement, as an additional task. More specifically, the user uses the VR controller 200 The chair assembly task is performed by controlling the position and rotation of the wood chips on the right, referring to the completed chair on the left.

When the creation of the chair, which is an additional task, is completed by the user, the local computer 300 sends an additional message, and provides an ending screen and a result graph as shown in FIG. 9B together with an ending comment for the ending shown in FIG. 9A. Perform (S500).

In the result graph, the execution time for planning, execution, and result is presented in the form of a bar graph, and the time performed in the planning, execution, and result steps is provided as a percentage.

When the user clicks the result confirmation button for the provided result as described above with the VR controller 200 held by the user, the local computer 300 performs a step of moving to the game end screen (S600).

As shown in FIG. 9C, the content of the'gold ax silver ax' selected at the end of the screen is ended, and then the story proceeds again when the following selection is made, and the local computer 300 is then Steps after step (S120) of step (S100) are repeatedly performed.

In this case, the user may control the VR controller 200 to change and select targets and additional tasks to be performed.

The content server 500, which is the last configuration of the cognitive rehabilitation training and evaluation system by performing a role in a story using a virtual reality environment according to the present invention, is connected to the local computer 300 and the communication network 400. The user's content execution result is stored, the user's personal information, including user ID and age, etc., is databased by date, and the user's content performance is analyzed.

In addition, the content server 500 provides an analysis result according to the content performance of the user and raw data of the analysis result according to the request of the local computer 300.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the technical field to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

100: HMD (Head Mounted Display)
200: VR controller
300: local computer
310: Main Fisherman
320: image providing unit
330: communication department
340: voice providing unit
350: Target provision unit
360: tool provision unit
370: situation judgment generating unit
380: unexpected situation occurrence part
390: database unit
400: communication network
500: content server

Claims (7)

(a) preparing cognitive rehabilitation training by performing a role in a story using a virtual reality environment by a local computer;
(b) performing cognitive rehabilitation training by performing a role in a story through two-way communication with the HMD and VR controller equipment that creates a virtual reality environment as the user wears the local computer;
(c) determining, by the local computer, whether a goal has been achieved according to the cognitive rehabilitation training;
(d) providing, by the local computer, the additional task selected in step (a);
(e) the local computer providing results;
(f) the local computer checks whether or not the user is finished, ends the content when selecting the end, and then repeats the process after step (a) when selecting the following; in a story using a virtual reality environment, Cognitive rehabilitation training and evaluation method through role performance.
The method of claim 1,
Step (a)
(a-1) verifying, by the local computer, the connection of the HMD and VR controller equipment;
(a-2) the local computer providing a content list to the connected HMD and receiving a user's content selection;
(a-3) providing, by the local computer, a story proceeding method and a short proceeding method of the content selected by the user, and receiving a selected proceeding method according to an operation of the VR controller; And
(a-4) When the user's selection of the progression method is the story progression method, the local computer provides precautions and progression methods; cognitive rehabilitation training through performing a role in a story using a virtual reality environment including And evaluation methods.
The method of claim 2,
Step (b)
(b-1) the local computer starting a story of the content selected by the user;
(b-2) the local computer presenting and confirming a goal to be performed in the process of proceeding with the story of the content selected by the user;
(b-3) allowing the local computer to select the necessary tools to achieve the goal;
(b-4) transmitting, by the local computer, an instruction to achieve the goal to a selected tool after moving the user to a location where an environment for achieving the goal is established; And
(b-5) using the VR controller, the step of executing an action for achieving the goal with the tool selected in step (b-3); through performing a role in the story using a virtual reality environment including Cognitive rehabilitation training and evaluation methods.
The method of claim 3,
Step (b)
(b-6) The situation determination generating unit of the local computer directs a situation that may occur in the content to the user in the process of executing an action to achieve the goal, and determines the situation produced through a question and answer with the user. Cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment that further includes.
The method of claim 3,
Step (b)
(b-7) In the process of executing an action to achieve the target by the local computer's emergency situation generating unit, the user generates an unexpected situation that may occur in the content, so that the user can operate the VR controller to determine the sudden situation. Cognitive rehabilitation training and evaluation method by performing a role in a story using a virtual reality environment that further includes a step to overcome.
The method of claim 5,
A virtual reality environment in which the local computer targets and executes only the actions or actions performed by the characters in the content regardless of the flow of the story when the user's selection of the progress method in the step (a-3) is the short progress mode. Cognitive rehabilitation training and evaluation method through the performance of the role in the story.
The method of claim 6,
In step (f) above
Before the step of repeating the process after step (a) according to the following selection, the user performs cognitive rehabilitation training through the role in the story using a virtual reality environment in which the target in the selected content and the additional tasks can be changed. And evaluation methods.

Images