KR20200074786A - Memory improvement training system based on augmented reality - Google Patents

Abstract

The present invention relates to a memory improvement training system based on augmented reality. According to one embodiment of the present invention, the memory improvement training system based on augmented reality comprises: an AR device displaying a mixture of real image and virtual image and including a camera; and a learning device providing content related to memory training. The AR device includes: an image storage unit in which at least one content image capable of enhancing memory enhancement is produced and stored in advance; an image providing unit including a scenario unit which matches data recognized from a camera unit with content images stored in the image storage unit, and providing the matched content images; and an image display unit displaying the content images matched by the image providing unit.

Description

Memory improvement training system based on augmented reality

The present invention relates to a training system and method for improving memory based on augmented reality. More specifically, the present invention relates to a training system and method for improving memory based on augmented reality for improving a user's memory by performing a memory improvement training process through an augmented reality device.

With the development of science and technology, human life expectancy is steadily increasing. As a result, aging is accelerating in modern society, and in Korea, the elderly population is close to 10 million. Therefore, maintaining the health of the elderly is a very important issue both nationally and socially.

Among the senile diseases, especially in the case of senile dementia, the probability of developing increases according to aging, and recovery is virtually impossible when symptoms progress to some extent. It is very difficult to solve in that the economic, social, and emotional stresses are very high.

In a recent study, it has been found that senile dementia can be prevented or recovered through rehabilitation training at the stage of mild symptoms. According to this background, the early detection of senile dementia and rehabilitation treatment at an early stage are increasingly requested not only by individuals with dementia, but also by their families and national society. However, it is still very difficult to detect the symptoms of senile dementia early, and in addition, a rehabilitation treatment device is in need of further development.

To solve this problem, (Patent Document 1) has been disclosed, and specifically described with reference to (Patent Document 1) is as follows.

1 is a block diagram showing a cognitive ability enhancement system according to the prior art and a cognitive ability training method using the same, according to this, in a system for providing a plurality of contents capable of promoting cognitive ability, predetermined data is The database unit 10 to be stored, the input unit 20 for providing a predetermined signal, the data stored in the database unit 10 and the output unit 30 for outputting a signal provided from the input unit 20 and A control unit 40 is included, and the data includes a plurality of problems classified according to predetermined criteria and answers to the problems, and the control unit 40 selects a plurality of problems and presents them to a user (42), the user's signal and the solution stored in the database unit (10) scoring unit for scoring, and a rating determining unit for determining the user's rating from the scoring result derived through the scoring (46), and the questionnaire 42 includes a cognitive ability enhancement system (1) characterized in that the questions are asked by combining the classified problems.

However, according to the above-mentioned (Patent Document 1), it consists of a system for questioning and matching the problem, lacking elements to attract the user's interest, and lacking the diversity of systems related to cognitive ability, which the user can use continuously. There is a problem that the probability is significantly lowered.

Republic of Korea Patent Publication No. 10-2013-0099714

An object of the present invention is to provide a memory enhancement training system and method based on augmented reality for improving a user's memory by performing a memory enhancement training process through an augmented reality device.

In order to solve the above problem,

A memory enhancement training system based on augmented reality according to an embodiment of the present invention includes an AR device including a real image and a virtual image and displaying them; And

A learning device that provides content related to memory training;

Including,

The AR device includes an image storage unit in which one or more content images capable of promoting memory enhancement are produced and stored in advance;

An image providing unit that includes a scenario unit matching data recognized by the camera and content images stored in the image storage unit, and provides a matched content image; And

An image display unit displaying the matched content image in the image providing unit;

It may include.

In addition, in the training system for improving memory based on augmented reality according to an embodiment of the present invention, the learning device is formed in a book or note shape, a groove is formed in each section inside, and provides a memory training program to the user Can.

In addition, in the memory enhancement training system based on augmented reality according to an embodiment of the present invention, the learning device comprises a plurality of assemblies made of the same shape as the groove; And

A first recognition unit formed at an edge of the learning device to recognize an exposed point of the learning device;

It may further include.

In addition, in the memory enhancement training system based on augmented reality according to an embodiment of the present invention, the assembly is formed on the border of one surface, a second recognition unit for giving a unique marker of the assembly; And

A third recognition unit formed inside the second recognition unit and displayed as an image or text to refer to the assembly;

It may further include.

In addition, in the memory enhancement training system based on augmented reality according to an embodiment of the present invention, the image storage unit is a tutorial unit for introducing and guiding the AR device or training program before performing memory training;

It may further include.

In addition, in the memory enhancement training system based on augmented reality according to an embodiment of the present invention, the image providing unit is a recognition unit receiving the image through the camera to recognize the situation being performed by the learning device;

A confirmation unit for determining an image recognized by the recognition unit and checking whether it is the same as the content image proceeding in the scenario unit;

A corresponding unit determining a correct answer or a wrong answer based on the data confirmed by the checking unit, and corresponding to the correct answer or incorrect answer; And

A calculation unit that accumulates data of correct or incorrect answers determined by the corresponding unit and calculates a final score;

It may further include.

In addition, in the memory enhancement training system based on augmented reality according to an embodiment of the present invention, the image display unit augmented image with a matched content image in the scenario unit to display the image enhancement unit; And

A pop-up unit displaying a content image played on the AR device and a result according to whether or not the learning device is operated;

It may further include.

In addition, in the training method for improving memory using augmented reality according to an embodiment of the present invention,

(a) the AR device recognizing the first recognition unit;

(b) matching the content image stored in the image storage unit based on the data recognized by the first recognition unit, and playing the matched content image;

(c) recognizing an assembly suitable for the content image to be reproduced, and determining a correct answer or incorrect answer;

(d) displaying an image augmentation or pop-up that matches the correct answer or incorrect answer; And

(e) calculating a score for the final result and displaying it;

It may include.

In addition, in the memory enhancement training method using augmented reality according to an embodiment of the present invention, step (c) is

(c-1) recognizing a second recognition unit configured in the assembly;

(c-2) determining whether the assembly is suitable for the content image;

(c-3) determining correct answers and incorrect answers based on the determined data;

It may include.

In addition, in the memory enhancement training method using augmented reality according to an embodiment of the present invention, if it is determined that step (c-3) is correct, the image may be augmented on the learning device through an image enhancement unit.

In addition, in the memory enhancement training method using augmented reality according to an embodiment of the present invention, if it is determined that the step (c-3) is incorrect, a pop-up related to the incorrect answer may be displayed through the pop-up unit.

In addition, in the memory enhancement training method using augmented reality according to an embodiment of the present invention, if the wrong answer is repeated three times, it is determined as the final wrong answer, and the next content image to be played can be reproduced.

In addition, in the training method for improving memory using augmented reality according to an embodiment of the present invention, scores may be differentially assigned according to the number of incorrect answers.

These solutions will become more apparent from the following detailed description of the invention based on the accompanying drawings.

Prior to this, the terms or words used in the specification and claims should not be interpreted in a conventional and lexical sense, and the inventor may appropriately define the concept of terms in order to best describe his or her invention. Based on the principle of being able to be interpreted as meanings and concepts consistent with the technical idea of the present invention.

According to an embodiment of the present invention, a memory enhancement training program may be performed through an AR device, and a user's interest may be induced to induce memory training.

Figure 1 is an overall block diagram showing the prior art.
Figure 2 is a usage example showing a memory enhancement training system based on augmented reality according to an embodiment of the present invention.
Figure 3 is a perspective view showing a learning device of a memory enhancement training system based on augmented reality according to an embodiment of the present invention.
Figure 4 is a perspective view showing an assembly of a memory enhancement training system based on augmented reality according to an embodiment of the present invention.
5 is a block diagram showing an AR device of a training system for improving memory based on augmented reality according to an embodiment of the present invention.
Figure 6 is a flow chart showing a memory improvement training method based on augmented reality according to an embodiment of the present invention.

The specific aspects and specific technical features of the present invention will become more apparent from the following specific contents and embodiments associated with the accompanying drawings. It should be noted that in this specification, when adding reference numerals to the components of each drawing, the same components have the same reference numerals as possible even though they are displayed on different drawings. In addition, in describing an embodiment of the present invention, when it is determined that detailed descriptions of related well-known structures or functions may obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

In addition, in describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the component from other components, and the nature, order, or order of the component is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected to or connected to the other component, but another component between each component It should be understood that elements may be "connected", "coupled" or "connected".

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

2 to 5, a memory enhancement training system based on augmented reality according to an embodiment of the present invention displays a mixture of reality-based images and processed images, such as virtual images, collected from the camera 101 It includes an AR device 100 and a learning device 200 that provides content related to memory training.

The AR device performs a function of augmenting an image based on reality by mixing and displaying a reality-based image collected through the camera 101 and a content image stored in advance, and is a device that provides a visual aesthetic to the user.

In the AR device 100, one or more content images capable of promoting memory enhancement are produced in advance and stored in a storage device provided therein, and data recognized by the camera 101 (or Image) and an image providing unit 130 that matches the content image stored in the image storage unit 110 to provide an image from which a result value is derived, and an image display unit 150 that displays the content image derived from the image providing unit 130. It can contain.

The image storage unit 110 stores a variety of previously produced content images, and a storage medium capable of storing these content images may be provided inside the AR device 100, but is not limited thereto, and is a separate server. After a wireless connection is established through a local area network (not shown) and the content image is stored in the server, a corresponding image may be requested and received from the server.

The image storage unit 110 may include a tutorial unit 111 that will guide how to use the AR device 100 and the learning device 200 before executing the memory training program.

The tutorial unit 111 is produced in the form of a guide image so that the user can easily use and operate the AR device 100 and the learning device 200, and the user can operate the AR device 100 even if the user does not set it separately. When this is done, it is configured to guide the tutorial in a pop-up form before executing the training program.

The image providing unit 130 recognizes a marker assigned to the learning device 200 and matches the content images stored in the image storage unit 110 based on the recognized data to provide an image from which the result value is derived. As to perform, it includes a scenario unit 131, a recognition unit 133, a confirmation unit 135, a correspondence unit 137, and a calculation unit 139.

The scenario unit 131, as described above, the learning device 200 matches the data recognized by the camera 101 and the content images stored in the image storage unit 110 to select a content image that matches the recognized data To perform.

The recognition unit 133 recognizes a section (or page) of the learning device 200 through a situation performed in the learning device 200, such as a camera 101, and a marker assigned to each assembly 230. It recognizes the learning device 200 and each assembly 230 such as a function of recognizing and performs a function of distinguishing them.

Here, the learning device 200 and each assembly 230 can be recognized through the first recognition unit 250, the second recognition unit 270, and the third recognition unit 290, which will be described later. Will be described later.

The verification unit 135 recognizes the learning device 200 and each assembly 230 recognized through the content image played by the scenario unit 131 and the camera 101, and the assembly suitable for the content image being played ( 230) It performs a function of checking whether or not it is.

In addition, the confirmation unit 135 may check whether the assembly 230 is inserted into the groove 210 formed in the learning device 200 to be described later, and when the assembly 230 is inserted into the groove 210 The correct answer or incorrect answer may be determined through the corresponding unit 137, and if the assembly 230 is not inserted into the groove 210, the timeout or incorrect answer may be displayed through the image display unit 150.

The correspondence unit 137 determines whether the assembly 230 suitable for the content image being played is properly inserted into the groove 210 formed in the learning device 200, and performs a corresponding function based on the determined data.

Specifically, the correspondence unit 137 determines whether the correct answer or incorrect answer is determined by determining whether the content image being played and the assembly 230 suitable for the content image are correct, and if it is the correct answer, augments the image through the image display unit 150 to relate to the correct answer. The animation is played, and in the case of an incorrect answer, a pop-up related to the incorrect answer may be displayed through the image display unit 150.

The calculation unit 139 performs a function of calculating the total score by synthesizing data that determines the correct or incorrect answer by the corresponding unit 137, and the number of times the content video is exhausted or the content video playback preset by the user is performed. Can be calculated after termination.

The image display unit 150 performs a function of displaying the matched content image in the image providing unit 130.

Also, the image display unit 150 augments the image together with the content image matched by the scenario unit 131 to display the image enhancement unit 151 and the AR device 100 to display the content image and the learning device 200 It may further include a pop-up unit 153 for displaying the results according to whether or not the operation.

The image augmentation unit 151 performs a function of augmenting the image of the corresponding content image to give the user a visual aesthetic point of view when the corresponding unit 137 determines that the answer is correct. No, the content image itself can be augmented, but is not limited thereto.

The pop-up unit 153 may be displayed in a pop-up form when the corresponding unit 137 determines that the answer is incorrect, the calculation unit 139 displays data about the result, or displays the AR device 100 settings. .

Meanwhile, the learning device 200 may be configured in various shapes, such as an object, a book, a note, a smart device, and an electronic device, through which a user can easily experience content. In the present invention, any one of books or notes is selected. Formed to provide content related to memory enhancement to the user.

In addition, when the learning device 200 is made of, for example, a book or a notebook, a groove 210 is formed inside the assembly 230 for each section (or page), and an assembly suitable for a section corresponding to a user's section It is possible to provide a memory-related training program to the user by having a structure in which the training for inserting 230 is performed.

Also, the learning device 200 may configure the first recognition unit 250 so that the AR device 100 can be recognized for each section (or page).

The first recognition unit 250 is configured so that the AR device 100 can easily recognize each section by configuring a unique marker of each section (or page), for example, a learning device 200 made of books or notes. In this case, it may be configured to recognize a currently unfolded section (or page) by giving a page number to a corner.

The assembly 230 is provided with different assemblies 230 to be inserted into the grooves 210 formed for each chapter of the learning device 200, and the AR device 100 can recognize the different assemblies 230. To provide a marker on one side, for example, a border of a surface corresponding to a drawing plane, an image is provided inside the second recognition unit 270 and the second recognition unit 270 to identify different assemblies 230. A third recognition unit 290 may be included.

The third recognition unit 290 is provided to allow each user to identify the different assemblies 230 by giving each image to the different assemblies 230, and is described by selecting at least one of text or images The AR device 100 may also include a function of identifying the assembly 230 through the camera 101.

That is, the second recognition unit 270 and the third recognition unit 290 provided on one surface of the assembly 230 are unique markers provided for the AR device 100 to identify, in particular, the third recognition unit 290 ) Is an image that can be identified by the AR device 100 and the user.

Meanwhile, as illustrated in FIG. 6, in the training method for improving memory based on augmented reality according to an embodiment of the present invention, the AR device 100 recognizes the first recognition unit 250 (S20) and On the basis of the data recognized by the first recognition unit 250, matching the content image stored in the image storage unit and reproducing the matched content image (S30), and recognizing the assembly 230 suitable for the content image, , Determining the correct or incorrect answer (S31, S34), displaying the image enhancement or pop-up for the correct or incorrect answer (S32, S35), and calculating the score for the final result and displaying it (S50) It can contain.

First, before performing the step (S20) of the AR device 100 to recognize the first recognition unit 250, the user may use the tutorial unit (in order to guide the use of the AR device 100 or the learning device 200). 111) can be selected (S10), and when the user selects a tutorial, the image storage 110 may play a previously stored tutorial-related image (S11).

The memory improvement training program allows the user to perform training through the AR device 100 and the learning device 200, and recognizes the learning device 200 through the camera 101 installed in the AR device 100, , In this way, the AR device 100 may recognize the first recognition unit 250 (S20 ).

The AR device 100 recognizing the first recognition unit 250 matches the content image stored in the image storage unit based on the recognized data, and the image through which the result value is derived is displayed through the image display unit 150. It is reproduced (S30).

After the content video is continuously played, a problem related to the content video is presented, and the user has a structure that can answer the problem through the learning device 200.

Specifically, when the assembly 230 is inserted into the groove 210 of the learning device 200 by fitting, the confirmation unit 135 formed in the assembly 230 recognizes the second recognition unit 270 (S31). ), it is determined whether the second recognition unit 270 is inserted into the groove 210.

For example, when the assembly 230 is inserted into the groove 210, a correct answer or a wrong answer is determined through the corresponding unit 137 (S34), for example, the assembly 230 is not inserted into the groove 210, or corresponding If the unit 137 determines that the answer is incorrect, a pop-up related to the incorrect message may be displayed (S32).

After displaying the pop-up related to the incorrect message (S32), the same problem is presented 3 times, and if the correct answer is corrected within 3 times (S33), it is recognized as the correct answer, but the score is differentially assigned, and the same problem is incorrectly 3 times If it is processed (S33), it is processed as the final incorrect answer to play the next content image.

On the other hand, when the assembly 230 is inserted into the groove 210 and judged as the correct answer through the counter 137, the image related to the correct answer is enhanced through the image enhancement unit 151 to enhance the visual aesthetics of the user. Can be given.

The content video is repeatedly played as many times as the user has set in advance, or the number of content video is played (S40), and after the content video is completely played, the score for the final result is displayed through the pop-up unit 153. do.

That is, in the memory improvement training system and method using augmented reality according to an embodiment of the present invention, a memory improvement training program can be executed through the AR device 100, and the user's memory training is caused by inducing the user's interest It has the effect of inducing.

The present invention has been described in detail by way of example, but this is for specifically describing the present invention, and the training system and method for improving memory based on augmented reality according to the present invention are not limited thereto. In addition, the terms "include", "compose", or "have", as described above, mean that the corresponding component can be inherent unless specifically stated to the contrary, excluding other components However, all terms, including technical or scientific terms, should be interpreted as being able to further include other components, and unless otherwise defined, generally understood by those skilled in the art to which the present invention pertains. It has the same meaning as being.

In addition, the above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains can make various modifications and modifications without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain the scope of the present invention, and the scope of the technical spirit of the present invention is not limited by the exemplary embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical spirits within the equivalent range should be interpreted as being included in the scope of the present invention.

100-AR device 101-Camera
110-Video storage 111-Tutorial
130-Image providing unit 131-Scenario unit
133-recognition unit 135-confirmation unit
137-Correspondence section 139-Calculation section
150-Video display unit 151-Image enhancement unit
153-Popup part 200-Learning device
210-home 230-assembly
250-first recognition unit 270-second recognition unit
290-Third recognition unit

Claims (13)

AR device including a real image and a virtual image, and including a camera; And a learning device that provides content related to memory training;
Including,
The AR device includes an image storage unit in which one or more content images capable of promoting memory enhancement are produced and stored in advance;
An image providing unit that includes a scenario unit matching data recognized by the camera and content images stored in the image storage unit, and provides a matched content image; And
An image display unit displaying the matched content image in the image providing unit;
Memory enhancement training system based on augmented reality, including.
The method according to claim 1,
The learning device is made of a book or note shape, a groove is formed for each section therein, a memory enhancement training system based on augmented reality that provides a memory training program to the user.
The method according to claim 1,
The learning device includes a plurality of assemblies having the same shape as the groove; And
A first recognition unit formed at an edge of the learning device to recognize an exposed point of the learning device;
Memory training system based on augmented reality further comprising a.
The method according to claim 1,
The assembly is formed on the edge of one surface, a second recognition unit for giving a unique marker of the assembly; And
A third recognition unit formed inside the second recognition unit and displayed as an image or text to refer to the assembly;
Memory training system based on augmented reality further comprising a.
The method according to claim 1,
The image storage unit is a tutorial unit for introducing and guiding the AR device or training program before performing memory training;
Memory training system based on augmented reality further comprising a.
The method according to claim 1,
The image providing unit recognizes a situation performed by the learning device by receiving an image through the camera and recognizing it;
A confirmation unit for determining an image recognized by the recognition unit and checking whether it is the same as the content image proceeding in the scenario unit;
A corresponding unit determining a correct answer or a wrong answer based on the data confirmed by the checking unit, and corresponding to the correct answer or incorrect answer; And
A calculation unit that accumulates data of correct or incorrect answers determined by the corresponding unit and calculates a final score;
Memory training system based on augmented reality further comprising a.
In addition, in the memory enhancement training system based on augmented reality according to an embodiment of the present invention, the image display unit augmented image with a matched content image in the scenario unit to display the image enhancement unit; And
A pop-up unit displaying a content image played on the AR device and a result according to whether or not the learning device is operated;
Memory training system based on augmented reality further comprising a.
(a) the AR device recognizing the first recognition unit;
(b) matching the content image stored in the image storage unit based on the data recognized by the first recognition unit, and playing the matched content image;
(c) recognizing an assembly suitable for the content image to be reproduced, and determining a correct answer or incorrect answer;
(d) displaying an image augmentation or pop-up that matches the correct answer or incorrect answer; And
(e) calculating a score for the final result and displaying it;
Memory enhancement training method based on augmented reality, including.
The method according to claim 8,
Step (c) is
(c-1) recognizing a second recognition unit configured in the assembly;
(c-2) determining whether the assembly is suitable for the content image;
(c-3) determining correct answers and incorrect answers based on the determined data;
Memory enhancement training method based on augmented reality, including.
The method according to claim 9,
If the step (c-3) is determined to be the correct answer, a memory enhancement training method based on augmented reality that augments an image on the learning device through an image enhancement unit.
The method according to claim 9,
Step (c-3), if it is determined that the wrong answer, a memory improvement training method based on augmented reality that displays a pop-up related to the wrong answer through a pop-up unit.
The method according to claim 11,
If the incorrect answer is repeated three times, it is determined as the final incorrect answer, and the memory improvement training method based on augmented reality that reproduces the next content image to be processed.
The method according to claim 12,
A memory improvement training method based on augmented reality in which scores are differentially assigned according to the number of incorrect answers.

Images