KR20210144208A - Augmented reality based cognitive rehabilitation training system and method - Google Patents

Abstract

The present invention relates to an augmented reality cognitive rehabilitation system and a method thereof capable of performing a cognitive function test of the elderly or disabled through a test and rehabilitation platform based on augmented reality, and performing effective cognitive rehabilitation training by providing a cognitive rehabilitation training program based on a test result. An augmented reality based cognitive rehabilitation training system according to the present invention comprises: an evaluating and training unit which outputs an augmented reality based image, and performs cognitive ability evaluation and training of a trained person; a server which receives an evaluating result of the evaluating and training unit, and builds and provides a personalized training curriculum corresponding to the evaluating result; and a manager terminal which displays and manages the evaluating result transmitted to the server and a training aspect of the trained person.

Description

AR-based cognitive rehabilitation training system and method {AUGMENTED REALITY BASED COGNITIVE REHABILITATION TRAINING SYSTEM AND METHOD}

The present invention relates to an augmented reality-based cognitive rehabilitation training system and method, and more particularly, to perform a cognitive function test of the elderly or the disabled through an augmented reality-based inspection and rehabilitation platform, and cognitive rehabilitation based on the test result It relates to an augmented reality cognitive rehabilitation system and method that can perform effective cognitive rehabilitation training by providing a training program.

Recently, health has been improved due to the improvement of living standards in modern society and the development of medical technology. is increasing,

In the case of the number of patients with acquired mild cognitive impairment, according to the analysis of the National Health Insurance Corporation's "health insurance medical expenses payment data", the number of health insurance patients due to mild cognitive impairment is increasing at an average annual rate of 43.9%.

Currently, rehabilitation training for mild cognitive impairment in Korea is mainly conducted in centers and hospitals, and training methods include computerized cognitive training, social interaction training, task performance analysis, compensatory cognitive training, activity education for each cognitive level, and cognitive training through workbooks. Rehabilitation training is carried out in ways such as

However, most of the existing rehabilitation training for mild cognitive impairment is based on simple number matching, letter matching, and diary writing. It is difficult to conduct memory recovery training through the environment of daily life, and the effect is insignificant.

In the case of computerized cognitive rehabilitation training, it is difficult to apply rehabilitation training if there is no assistance or assistance from others, as training is performed using a keyboard, mouse, joystick, and monitor touch, which are difficult to operate. Therefore, there is a need for a rehabilitation training system with a new convenient interface that does not require other people or auxiliary helpers.

Due to these problems, we have developed an HMD-based virtual reality cognitive rehabilitation system that allows for immersion, interest, and connection with daily life, and a convenient interface that allows one-on-one rehabilitation training. In the case of disabled patients, when wearing an HMD, cyber distance symptoms such as headache, nausea, and motion sickness may occur, so a different system is required.

In addition, the method of testing for cognitive impairment is often written in the form of a paper pen or questionnaire, but since it uses multiple-choice questions that evaluate time and place orientation, immediate memory, attention, etc., it uses multiple-choice questions. There is also a problem in that accurate cognitive ability results cannot be derived as the test subjects who perform the test cannot concentrate.

Korean Patent Laid-Open Patent No. 10-2017-0140705: Cognitive ability rehabilitation training system and method using associative card matching game

The present invention was created to solve the above problems, and through an augmented reality system, a test for cognitive ability can be performed in an environment where the subject can be intuitive and more immersed in the situation, and the test value measured through this The purpose of the present invention is to provide an augmented reality-based cognitive rehabilitation training system and method that can increase the effectiveness of cognitive rehabilitation training by providing a cognitive rehabilitation program tailored to each individual and helping them to perform rehabilitation training.

The augmented reality-based cognitive rehabilitation training system according to the present invention outputs an augmented reality-based image, an evaluation and training unit capable of evaluating the cognitive ability of a training subject and performing training, and an evaluation result of the evaluation and training unit A server for receiving input and building and providing a customized training curriculum for each individual corresponding to the evaluation result, and a terminal for an administrator capable of displaying and managing the evaluation result transmitted to the server and the training aspect of the training target.

The evaluation and training unit includes a main body including a gesture unit formed so that a body part of the training subject can enter, and is installed on the main body and detects a state in which the body part of the training subject entering the gesture unit moves within the gesture unit. a motion recognition sensor, which is installed in the main body, and a photographing unit for photographing a body part of the training subject who has entered the gesture unit, and an image of the training subject's body part photographed by the photographing unit, cognitive ability evaluation or recognition It is preferable to include an image processing unit for transmitting an augmented reality image by synthesizing an image image for ability training, and a display unit for outputting the image output from the image processing unit so that the training target can see it.

The evaluation and training unit may further include AR glasses that the training target can wear, and an augmented reality training field for evaluation and training, and the augmented reality training field may be formed such that a structure for augmented reality evaluation and training is arranged. .

The terminal for the manager is preferably a web PC or a mobile terminal that the manager can access.

It may further include a data cloud for collecting, analyzing and processing big data for all training subjects by receiving evaluation information and training information from a plurality of evaluation and training units and each server.

The augmented reality-based cognitive rehabilitation training method of the present invention is analyzed in the cognitive ability evaluation step for evaluating the cognitive ability of the training target, the evaluation analysis step of collecting and analyzing the evaluation scores for the training target's cognitive ability, and the evaluation analysis step A training step of providing a training program to the training target based on the results obtained and performing cognitive rehabilitation training, and a re-evaluation step of reassessing the cognitive ability of the training target after cognitive rehabilitation training. An evaluation step of evaluating the cognitive ability And the training and re-evaluation steps are performed through an evaluation and training unit, wherein the evaluation and training unit includes a main body including a gesture unit formed so that a part of the training target's body can enter, and is installed in the main body and enters the gesture unit A motion recognition sensor for detecting a state in which a body part of a training subject moves within the gesture unit, a photographing unit installed in the main body, and a photographing unit for photographing a body part of the training subject entering the gesture unit, and the photographing unit An image processing unit that transmits an augmented reality image by synthesizing an image of a body part of the trained training target and an image image for cognitive ability evaluation or cognitive ability training, and outputting the image output from the image processing unit so that the training target can see It is configured to include a display unit.

After the cognitive ability evaluation step for the training target, further comprising a curriculum configuration step of composing a customized training curriculum according to the evaluation score or analysis result of the evaluated training target, for the progress of the evaluation and analysis step and the curriculum configuration step It is preferable that the evaluation analysis and training curriculum be provided in a server that is connected to enable data transmission with the evaluation and training unit and receives analysis information for evaluation from the evaluation and training unit, or provides information necessary for the evaluation or training do.

According to the augmented reality-based cognitive rehabilitation training system and method according to the present invention, since it is possible to examine and evaluate the cognitive ability of the training target in an intuitive and immersive environment, it is possible to accurately analyze the condition of the training target, and There is an advantage of maximizing the effect of cognitive rehabilitation training because it is possible to provide the training program necessary for the training target accordingly.

In addition, according to the augmented reality-based cognitive rehabilitation training system and method according to the present invention, it is safe because the examination and rehabilitation training can be performed using augmented reality without wearing an uncomfortable HMD for the elderly or the cognitively disabled who mainly perform cognitive rehabilitation training. There is an advantage that examination and training can proceed in a comfortable state.

1 is a conceptual diagram of an embodiment of an augmented reality-based cognitive rehabilitation training system according to the present invention;
Figure 2 is an exploded perspective view showing the evaluation and training unit of the augmented reality-based cognitive rehabilitation training system of Figure 1;
3 is a view showing a state in which cognitive ability evaluation or rehabilitation training is performed through the evaluation and training unit of FIG. 2;
4 is a conceptual diagram of another embodiment of an augmented reality-based cognitive rehabilitation training system including AR glasses and augmented reality training ground;
5 is a view showing each step of a method of conducting cognitive rehabilitation training using an augmented reality-based cognitive rehabilitation training system according to the present invention;
6 is a flowchart illustrating a process of evaluating and training cognitive ability using the augmented reality-based cognitive rehabilitation training system of the present invention.

Hereinafter, an augmented reality-based cognitive rehabilitation training system and method according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Since the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific disclosed form, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements. In the accompanying drawings, the dimensions of the structures are enlarged than the actual size for clarity of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that it does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Referring to the drawings, the augmented reality-based cognitive rehabilitation training system 10 according to the present invention includes an evaluation and training unit 100 configured to evaluate the cognitive ability of a trainee and perform training, and an evaluation and training unit 100 . A server 200 that receives and analyzes the evaluation result, configures a training curriculum and transmits it to the evaluation and training unit 100, a data cloud 300 that transmits and receives data with the server 200, and a data cloud 300 It includes a manager terminal 400 connected to.

As shown in FIGS. 2 and 3, the evaluation and training unit 100 is formed so that evaluation or training can proceed with an augmented reality image by entering the hand of the training target.

The evaluation and training unit 100 includes a main body 110 including a gesture unit 111 , a motion recognition sensor 120 formed on the main body 110 , a photographing unit 130 , and an image processing unit 140 . and a display unit 150 .

The main body 110 includes a base frame 112 and a support frame 113 extending upward from the base frame 112 to form a gesture unit 111 that allows the hand of the training subject to enter inside. and an upper frame 114 provided on an upper portion of the support frame 113 .

The hand of the trainee can enter the upper part of the base frame 112 , and the space into which the hand enters becomes the gesture unit 111 .

The motion recognition sensor 120 is provided on one side of the main body 110 to detect the hand motion of the training target. That is, the motion recognition sensor 120 recognizes a motion such as the direction of the user's hand, the shape of a finger, an action of opening or squeezing the hand, and an action of pointing to an instruction using a finger.

The motion recognition sensor 120 may be formed of various types of sensors, such as a lidar sensor, an acceleration sensor, an ultrasonic sensor, or an infrared sensor, capable of detecting a hand motion of a training subject, alone or in combination.

The photographing unit 130 is installed on the upper portion of the main body 110 to photograph the hand of the trainee who entered the gesture unit 111 . Since the image captured by the photographing unit 130 is output to the display unit 150 together with the augmented reality image through the image processing unit 140 to be described later, in the same direction as the direction that the training target sees through the display unit 150 . It is preferable to be installed to photograph the gesture unit 111 .

The image processing unit 140 generates an augmented reality image by synthesizing the image image for evaluation or training provided by the server 200 to be described later with the hand of the training subject photographed by the photographing unit 130 as described above, and send out

For example, the image processing unit 140 may synthesize the space of the gesture unit 111 as a card or a specific object image in addition to the user's hand image captured by the photographing unit 130 . The synthesized image is provided to the training target through the display unit 150, and the training target recognizes the space of the gesture unit 111 that is actually empty as a specific object or a specific space, so that evaluation or training can be performed.

The display unit 150 outputs the augmented reality image processed by the image processing unit 140 as described above, and in addition to the image output from the display unit 150, the speaker is provided so that voice guidance for evaluation or training of the training target can be made. can be embedded.

The server 200 evaluates the cognitive ability of the training target through the evaluation and training unit 100 and configures a training curriculum.

The server 200 evaluates the cognitive functions of eight domains for the evaluation of the cognitive ability of the trainee, that is, orientation, memory, attention, memory recall, name waiting, repeating, command execution, and understanding judgment fields. proceed In the evaluation process, the training target may input an answer by saying or touching the answer to the question on the display, or depending on the situation outputted through the augmented reality image while putting the hand into the gesture unit 111 to change the position or shape of and acquire the motion of the training target through the motion recognition sensor 120 .

The evaluation results for various fields are transmitted to the server 200, and the server 200 analyzes the cognitive ability of the training target based on the evaluation results.

The server 200 configures a training curriculum for improving the cognitive ability of the training target according to the cognitive ability analysis result. In other words, among the eight evaluated areas, a curriculum that can be intensively trained on the areas requiring reinforcement training for cognitive ability is constructed, and after the training curriculum is saved, the trainee conducts training to improve cognitive ability When it is done, it provides a training video corresponding to the configured training curriculum.

In this embodiment, the evaluation and training unit 100 for evaluating or training the cognitive ability of the training target is configured to include a main body 110 that can be used by entering the user's hand, but other than the user's hand. Cognitive ability evaluation and training can be made through body parts and augmented reality images.

In addition, as shown in FIG. 4, the evaluation and training unit 100 further includes an AR glass 160 formed so that the training target can see the augmented reality image by wearing it, and the augmented reality training ground 170 for evaluation and training. may include

The AR glasses 160 can be worn on the face of the training target, and can output virtual or augmented reality images or images to the training target. AR glass may be applied. In addition, the augmented reality training ground 170 may provide a space for the training target to enter and perform evaluation or training, and a motion sensor capable of sensing the entire movement of the training target may be applied. In addition, the augmented reality training ground 170 may be provided with a separate structure or article for evaluation or training.

The data cloud 300 is connected to receive evaluation information and training information for training subjects from several servers 200 . The data cloud 300 collects and analyzes big data for all training subjects while receiving information on a plurality of training subjects.

The manager's terminal is for a manager who manages evaluation and training of training subjects, and can receive and output information about each training target from the data cloud 300, and if necessary, the manager can perform evaluation or training You can input information about or correct or add a training curriculum built in the server 200 .

The manager terminal may be configured as a web PC 410 or a mobile terminal 420 such as a smart phone or tablet PC 410 .

The cognitive rehabilitation training method through the augmented reality-based cognitive rehabilitation training system 10 according to the present invention described above will be described as follows.

As shown in FIG. 5, the augmented reality-based cognitive rehabilitation training method of the present invention is a cognitive ability evaluation step for evaluating the cognitive ability of the training target, and the evaluation score for the training target's cognitive ability measured in the cognitive ability evaluation step The evaluation and analysis stage of collecting and analyzing, the curriculum construction stage of composing the training curriculum for the trainee based on the results analyzed in the evaluation and analysis stage, and the training stage of conducting training to improve the cognitive ability of the trainee according to the configured curriculum and a re-evaluation step of re-evaluating the cognitive ability of the training target after performing cognitive rehabilitation training.

The cognitive ability evaluation step proceeds through the evaluation and training unit 100 of the augmented reality-based cognitive rehabilitation training system 10, that is, eight areas of cognitive functions, that is, orientation, memory, attention, memory recall, name waiting. , follow-up, command execution, and comprehension judgment fields are checked, and the test result is transmitted to the server 200 .

The evaluation and analysis step is performed in the server 200, and the cognitive ability of the training target is evaluated based on the evaluation information measured in the cognitive ability evaluation step. In particular, an individual evaluation of eight cognitive functions is performed to classify the normal and vulnerable parts of the training target's cognitive abilities, and the degree of cognitive function in each field is analyzed.

The curriculum configuration step is performed in the server 200, and a customized training curriculum for the training target is configured based on the evaluation analysis result. In other words, the curriculum is structured so that intensive cognitive rehabilitation training can be carried out on the weak areas as a result of measuring the cognitive ability of the trainee.

The training step is performed through the evaluation and training unit 100, and the training target performs rehabilitation training for cognitive ability improvement in the same manner as in cognitive ability evaluation.

The re-evaluation step is to check whether the cognitive ability of the trainee has improved after going through the training step, to check the part that requires additional rehabilitation training, is evaluated through the evaluation and training system, and the evaluation analysis is performed in the server 200 will perform

Data on the evaluation analysis and training curriculum, rehabilitation training progress record, re-evaluation test execution and re-evaluation test results are transmitted and stored in the data cloud 300 , and the data cloud 300 is transmitted from a plurality of servers 200 . Information on each trained training target is divided and stored, and big data information is collected and information is analyzed.

The manager can check information about the training progress and measurement results of the trainee through his/her manager terminal, and can adjust the training curriculum or revise the data as needed.

According to the augmented reality-based cognitive rehabilitation training system 10 and method according to the present invention described above, cognitive rehabilitation training is performed by first evaluating the cognitive ability of a training target with reduced cognitive ability, and configuring a customized training curriculum for each training target. can be carried out, the effect on cognitive rehabilitation training can be maximized.

In addition, since cognitive ability evaluation or cognitive rehabilitation training based on augmented reality can be performed, the accuracy of cognitive ability evaluation can be increased by increasing the concentration on training subjects compared to the conventional cognitive ability evaluation or cognitive rehabilitation training method, and customized training You can provide a curriculum and maximize the training effect.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the invention. Thus, the present invention is not intended to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

10: Augmented Reality Based Cognitive Rehabilitation Training System
100: evaluation and training unit
110: body
111: gesture unit
112: base frame
113: support frame
114: upper frame
120: motion recognition sensor
130: filming unit
140: image processing unit
150: display unit
160: AR glass
170: augmented reality training ground
200: server
300: data cloud
400: administrator terminal
410: web PC
420: mobile terminal

Claims (8)

an evaluation and training unit that outputs an augmented reality-based image and is capable of evaluating and training the subject's cognitive ability;
a server that receives the evaluation and evaluation results of the training unit, builds and provides a customized training curriculum for each individual corresponding to the evaluation result;
Comprising a terminal for an administrator capable of displaying and managing the evaluation result transmitted to the server and the training aspect of the training target
Augmented reality based cognitive rehabilitation training system.
The method of claim 1,
The evaluation and training unit includes: a body including a gesture unit formed to allow a body part of a training target to enter;
a motion recognition sensor installed on the main body and detecting a state in which a body part of a trainee who has entered the gesture unit moves within the gesture unit;
a photographing unit installed on the main body and photographing a part of the body of the training target who has entered the gesture unit;
an image processing unit for transmitting an augmented reality image by synthesizing an image of a body part of the training target taken by the photographing unit and an image image for cognitive ability evaluation or cognitive ability training;
A display unit for outputting the image output from the image processing unit so that the training target can see it
Augmented reality based cognitive rehabilitation training system.
3. The method of claim 2,
The evaluation and training unit further comprises an AR glass that the training target can wear, and an augmented reality training ground for evaluation and training,
The augmented reality training ground, characterized in that the structure for the augmented reality evaluation and training is arranged
Augmented reality based cognitive rehabilitation training system.
3. The method of claim 2,
The manager terminal is a web PC or mobile terminal that the manager can access
Augmented reality based cognitive rehabilitation training system.
3. The method of claim 2,
A plurality of evaluation and training units and receiving evaluation information and training information from each server, characterized in that it further comprises a data cloud for collecting, analyzing and processing big data for all training subjects
Augmented reality based cognitive rehabilitation training system.
6. The method of claim 5,
The manager terminal is a web PC or mobile terminal capable of receiving information from the data cloud, outputting transmission data to the manager, and inputting the manager's management signal.
Augmented reality based cognitive rehabilitation training system.
a cognitive ability evaluation step for evaluating the cognitive ability of the trainee;
an evaluation and analysis step of collecting and analyzing evaluation scores on the cognitive ability of the trainee;
a training step of providing a training program to the trainees based on the results analyzed in the evaluation and analysis step and performing cognitive rehabilitation training;
Including a re-evaluation step of re-evaluating the cognitive ability of the training target after cognitive rehabilitation training,
The evaluation stage, the training stage, and the re-evaluation stage for evaluating the cognitive ability are made through the evaluation and training unit,
The evaluation and training unit includes: a body including a gesture unit formed to allow a body part of a training target to enter;
a motion recognition sensor installed on the main body and detecting a state in which a body part of a trainee who has entered the gesture unit moves within the gesture unit;
a photographing unit installed on the main body and photographing a part of the body of the training target who has entered the gesture unit;
an image processing unit for transmitting an augmented reality image by synthesizing an image of a body part of the training target taken by the photographing unit and an image image for cognitive ability evaluation or cognitive ability training;
and a display unit for outputting the image output from the image processing unit so that the training target can see it.
Augmented reality-based cognitive rehabilitation training method.
8. The method of claim 7,
After the cognitive ability evaluation step for the training target, further comprising a curriculum configuration step of configuring a customized training curriculum according to the evaluation score or analysis result of the evaluated training target,
For the progress of the evaluation and analysis step and the curriculum construction step, the evaluation and training unit and data transmission are possible, and the analysis information for evaluation is provided from the evaluation and training unit, or information necessary for the evaluation or training is provided. characterized in that evaluation analysis and training curriculum provision is made in the server
Augmented reality-based cognitive rehabilitation training method.

Images