KR102021700B1 - System and method for rehabilitate language disorder custermized patient based on internet of things - Google Patents

Abstract

Disclosed are an IoT-based patient customized communication disorder rehabilitation system, and a method thereof. The communication disorder rehabilitation method implemented by a computer comprises the steps of: providing a screen for selecting rehabilitation tools based on a linguistic state of a patient; providing a plurality of rehabilitation content corresponding to a selected specific category for a plurality of rehabilitation content in a level corresponding to the linguistic state of the patient; and providing at least one rehabilitation content selected among the provided plurality of rehabilitation content according to at least one rehabilitation tool selected through the screen. If the selected rehabilitation tool is an augmented reality tool, a marker contained in the rehabilitation content is recognized and thus, based on the recognized marker, a 3D motion of the corresponding rehabilitation content may be provided on the basis of an augmented reality.

Description

IoT-based patient customized communication disorder rehabilitation system and method {SYSTEM AND METHOD FOR REHABILITATE LANGUAGE DISORDER CUSTERMIZED PATIENT BASED ON INTERNET OF THINGS}

The present invention relates to an IoT-based patient-specific communication disorder rehabilitation system and method, and more particularly, to a technology to help rehabilitation of language skills for patients with memory impairment due to accidents, speech disorders due to congenital developmental disabilities.

Communication disorders include language disorders, speech sound disorders, childhood fluency disorders (stuttering, child-onset fluency disorders, stuttering), and social (practical) communication disorders that slow the central development of the cerebrum. There are four types of social (pragmatic) communication disorders, which are caused by delayed or damaged brain development. For example, a patient may have a memory impairment due to brain damage due to an accident such as a traffic accident or a fall accident, or the central development of the brain may be delayed due to a congenital developmental disorder or a developmental delay due to the environment, thereby causing a communication disorder.

These communication disorders can be improved by attempting early rehabilitation treatment from infancy when born, and in case of injury caused by brain injury such as traffic accidents, the rehabilitation therapy is more important. high.

In general, patients experiencing communication disorders visit a speech therapy institution for rehabilitation, and therapists show various word cards to patients to help treat patients' speech. As such, each word card necessary for real life or learning must be prepared for actual speech therapy, and since many words are used in real life, it takes a lot of time and effort to prepare all word cards.

In addition, the vocabulary for vocabulary cards has already been made and used in the treatment institution, but the patient simply learns words by simply matching the words and memorizing the words, so that patients lose interest in learning. In the end, the learning effect is reduced.

Accordingly, to reduce the effort of the therapist (that is, the administrator) to prepare a variety of word cards (that is, word cards), to provide a patient-specific rehabilitation program to patients based on a variety of rehabilitation tools, As therapists manage patients more easily and perform rehabilitation treatment, there is a need for a technique for quickly identifying a changing patient's condition.

Korean Patent Laid-Open No. 10-2017-0008970 relates to a speech rehabilitation treatment method and a speech rehabilitation apparatus, and the master terminal device is capable of performing speech rehabilitation treatment anytime, anywhere, regardless of time and place. A technique for receiving patient information from a server, displaying patient information, and transmitting a patient identification ID and a set of individual selective training problems to a language rehabilitation server is disclosed.

Embodiments of the present invention are for the therapist to design and edit the rehabilitation program of the patient in charge, to systematically manage the patient with a rehabilitation program customized to the individual patient, and to evaluate the progress of the rehabilitation treatment.

In addition, the therapist not only conducts face-to-face rehabilitation treatment, but also provides customized rehabilitation programs through the web / mobile web and applications without the therapist face-to-face. This is to make it possible.

In the computer-implemented communication disorder rehabilitation method, providing a screen for selecting a rehabilitation tool based on the language state of the patient, targeting a plurality of rehabilitation content belonging to a level corresponding to the language state of the patient, Providing a plurality of rehabilitation contents corresponding to a selected specific category, and providing at least one rehabilitation content selected from the plurality of rehabilitation contents provided according to at least one rehabilitation tool selected through the screen. When the rehabilitation tool is an augmented reality tool, as a marker included in the provided rehabilitation content is recognized, the 3D motion of the rehabilitation content based on the recognized marker may be provided on the basis of the augmented reality. .

According to one aspect, providing a screen for selecting the rehabilitation tool, may include providing a screen for selecting an evaluation method for the treatment of speech ability of the patient based on the language state of the patient. .

According to another aspect, the evaluation method, Method 1 for selecting a word based on the provided image, Method 2 for selecting an image based on the provided word, Selecting a corresponding word based on the provided audio signal The method may include at least one of method 3, method 4 for selecting a corresponding image based on a provided audio signal, and method 5 for receiving a corresponding audio signal based on a provided image.

According to another aspect, the providing of the rehabilitation tool may provide an image corresponding to the selected at least one rehabilitation content when the selected rehabilitation tool corresponds to an image.

According to another aspect, the providing according to the rehabilitation tool, if the selected rehabilitation tool corresponds to a phoneme, providing a consonant corresponding to the selected at least one rehabilitation content, the rehabilitation tool corresponds to a syllable Providing at least a portion of letters constituting a word corresponding to the selected at least one rehabilitation content; and if the rehabilitation tool corresponds to a word, a word corresponding to the selected at least one rehabilitation content. It may comprise the step of providing the entire character (constituting).

According to another aspect, the providing according to the rehabilitation tool, if the selected rehabilitation tool corresponds to an audio sound, the step of controlling to output an audio signal of a word corresponding to the at least one selected rehabilitation content can do.

According to another aspect, the providing of the rehabilitation tool may provide the rehabilitation content based on a content providing order set for the selected at least one rehabilitation content.

According to another aspect, the method may further include providing a screen for selecting a rehabilitation mode to proceed before providing the screen for selecting the rehabilitation tool.

According to another aspect, the rehabilitation mode is a training mode for providing speech therapy in a space separate from the treatment institution, the practice mode for providing speech therapy with a therapist belonging to the treatment institution And an evaluation mode for evaluating a performance result of at least one mode of the training mode and the practice mode.

According to another aspect, if the selected rehabilitation mode corresponds to any one of the practice mode and training mode, in order to provide a rehabilitation treatment based on augmented reality, the selected at least one rehabilitation content including a marker (marker) The corresponding image can be printed.

According to another aspect, the step of providing according to the rehabilitation tool, extracting a feature point from the image taken by the camera image corresponding to the printed rehabilitation content, and generating based on the extracted feature point And recognizing a marker included in the rehabilitation content based on the pattern and the pattern associated with the rehabilitation content.

The communication disorder rehabilitation system includes a screen providing unit that provides a screen for selecting a rehabilitation tool based on the language state of the patient, and a plurality of rehabilitation contents belonging to a level corresponding to the language state of the patient. And a content providing unit configured to provide a plurality of rehabilitation contents corresponding to each other, and to provide at least one rehabilitation content selected from the plurality of rehabilitation contents provided according to at least one rehabilitation tool selected through the screen. In the case of an augmented reality tool, as a marker included in the provided rehabilitation content is recognized, the 3D motion of the rehabilitation content may be provided on the basis of the recognized marker based on the augmented reality.

According to an aspect, the screen providing unit may further provide at least one of a screen for selecting an evaluation method for treating speech ability of the patient and a screen for selecting a rehabilitation mode based on the language state of the patient.

According to another aspect, the content providing unit, when the selected rehabilitation tool corresponds to an image, provides an image corresponding to the at least one selected rehabilitation content, when the selected rehabilitation tool corresponds to a phoneme, the selected at least Providing a consonant corresponding to one rehabilitation content, and if the rehabilitation tool corresponds to a syllable, providing at least some of the letters constituting a word corresponding to the selected at least one rehabilitation content, and the rehabilitation tool is assigned to the word If applicable, provide a full character constituting a word corresponding to the selected at least one rehabilitation content, and if the selected rehabilitation tool corresponds to an audio sound, the selected at least one rehabilitation content The audio signal of the corresponding word may be output.

According to one aspect, the content providing unit, a feature point extraction unit for extracting a feature point for the image taken by the camera image corresponding to the printed rehabilitation content, and a pattern generated based on the extracted feature point ( and a marker recognizing unit that recognizes a marker included in the rehabilitation content based on a pattern) and a pattern associated with the rehabilitation content.

Web / mobile web or application allows the therapist to design and edit the patient's rehabilitation program, providing a customized rehabilitation program for the individual patient, as well as the therapist systematically managing the patient and rehabilitation The progress of treatment can be assessed and confirmed.

In addition, the therapist not only conducts face-to-face rehabilitation treatment, but also provides customized rehabilitation programs through the web / mobile web and applications without therapist face-to-face, so that patients can practice rehabilitation at home alone. You can make this possible.

In addition, by providing a customized rehabilitation treatment to patients using a variety of rehabilitation tools, such as augmented reality, image, audio output, voice recording, to maintain the patient's interest in rehabilitation treatment, and to improve the progress of rehabilitation Can be improved.

In addition, in the case of the local storage method, the patient's personal information is securely vulnerable, and personal information can be leaked due to the loss of the measuring machine, but data is stored on the server as it provides rehabilitation treatment based on the server-client method. The patient's personal information, the therapist's patient list and histories of rehabilitation, and progress can be safely stored and maintained.

1 is a diagram illustrating a network environment of a communication rehabilitation system according to an embodiment of the present invention.
2 is a flowchart illustrating a rehabilitation method of communication disorder in an embodiment of the present invention.
3 is a block diagram showing the internal configuration of a communication disorder rehabilitation system according to an embodiment of the present invention.
4 is a diagram illustrating a screen configuration for patient management according to an embodiment of the present invention.
5 is a diagram illustrating a screen configuration for constructing a content list according to one embodiment of the present invention.
6 is a diagram illustrating a screen configuration showing rotation of an object corresponding to rehabilitation content according to one embodiment of the present invention.
7 is a diagram illustrating a rehabilitation treatment screen configuration according to one embodiment of the present invention.
8 is a diagram illustrating a configuration of a rehabilitation mode selection screen according to an embodiment of the present invention.
9 is a diagram illustrating a category selection screen provided in a training mode according to one embodiment of the present invention.
FIG. 10 is a diagram illustrating a rehabilitation treatment screen provided in a training mode according to one embodiment of the present invention.
FIG. 11 is a diagram illustrating an evaluation mode screen of Method 1 for selecting a word based on a provided image according to an embodiment of the present invention.
12 is a diagram illustrating an evaluation mode screen when a method 2 for selecting an image based on a word provided as an evaluation method is selected according to an embodiment of the present invention.
FIG. 13 is a diagram illustrating an evaluation mode screen provided with method 3 for selecting a corresponding word based on an provided audio signal according to an embodiment of the present invention.
FIG. 14 is a diagram illustrating an evaluation mode screen provided with Method 4 for selecting a corresponding image based on an audio signal provided according to an embodiment of the present invention.
FIG. 15 is a diagram illustrating an evaluation mode screen provided with a method 5 for receiving a corresponding audio signal based on a provided image.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. Also, like reference numerals in the drawings denote like elements.

The present embodiment relates to a technology for allowing a patient to be provided with rehabilitation treatment anytime, anywhere, without any additional work in a server-client environment using a web / mobile web, an application, or the like. In particular, the present invention relates to various rehabilitation tools such as augmented reality, photographic images, audio outputs, and voice recordings, and a technology for providing a customized rehabilitation treatment to a patient based on words appropriate to a patient's language ability state. In addition, a variety of rehabilitation modes are provided, including a training mode for training with the therapist, a practice mode in which the patient exercises rehabilitation at home alone, and an evaluation mode for evaluating the rehabilitation treatment. It is about a technology that enables a systematic and easy identification of the progress of the system.

In the present embodiments, the rehabilitation content represents various types of words used to treat a patient's speech ability, for example, a photographic image in the form of a word card, text representing the word itself, an object corresponding to the word. Audio signal (eg, a dog barking sound in the case of a dog, a baby crying sound in the case of a baby, etc.) and augmented reality content matched with identifier information of a corresponding word. The words may be classified according to a predetermined age, such as a child's language and an adult language, and words corresponding to each age may be used for daily life (eg, home-related use), travel, school life, work life, etc. It may be divided into various predetermined categories. Identifier information of each category, identifier information of a plurality of words belonging to the category, and identifier information of rehabilitation contents associated with the word may be matched and stored and maintained in a database.

In the present embodiments, the age may not indicate the actual age of the patient but may indicate the degree of speech ability, that is, the age indicating the state of speech ability of the patient. For example, even if the patient is 20 years old, if the language cognitive ability is 7 years old (ie, the brain language age is 7 years old) due to accidents, developmental delays, etc., the patient is 7 years old. Can be set.

In the present embodiments, augmented reality (AR) is an image processing technology for displaying a three-dimensional virtual image superimposed on an image or background of the real world as a single image, wherein a user such as a patient or a therapist is provided in a user terminal. When a specific marker is illuminated using a camera (eg, a built-in camera such as a webcam or a smartphone camera), a specific marker is recognized to combine the augmented reality content with the real world in three dimensions and provide it through the screen. May be indicated. That is, the present embodiments are intended to provide animation of a rehabilitation content to be used for speech therapy in augmented reality to provide a 3D model moving in the real environment, to induce interest to the patient and to enhance the treatment effect.

In these embodiments, you can use the Unity 3D engine as a graphics engine and Vuforia as an augmented reality plug-in, and the Unity 3D engine provides an intuitive interface and quickly work on level design and programming simultaneously using Microsoft C # as a script. can do.

In the present embodiments, the user terminal may represent an electronic device such as a PC, a smartphone, a tablet, a notebook, and the like, and in a server-client environment, to provide rehabilitation treatment to the user terminal The user terminal, which is a server and a client, may include a communication module for wired / wireless network connection.

1 is a diagram illustrating a network environment of a communication rehabilitation system according to an embodiment of the present invention.

Referring to FIG. 1, the rehabilitation system 101 (ie, communication disorder rehabilitation system), the patient 102, and the therapist 103 may form a network. In FIG. 1, an arrow means that data may be transmitted and received between the patient 102, the therapist 103, and the rehabilitation system 101 using a wireless network.

The patient 102 and therapist 103 substantially represent a user terminal (eg, a patient terminal, a therapist terminal) possessed by the patient and the therapist, and the user terminal is a personal computer (PC), a smart phone, a tablet. (tablet), a laptop, etc. may refer to any terminal device capable of accessing a web / mobile website related to the rehabilitation system 101 or installing and executing a service-only application (hereinafter, referred to as a "language rehabilitation app"). . In this case, the user terminal may perform overall service operations such as service screen configuration, data input, data transmission, and data storage under the control of a web / mobile site or a dedicated application.

The rehabilitation system 101 may correspond to a server that provides a rehabilitation treatment service for a client 102 and a therapist 103 who are clients. In this case, the rehabilitation system 101 may be implemented in a form included in a service platform for providing a rehabilitation treatment service in a client-server environment, and the present invention is not limited thereto and may be provided on a user terminal (ie, a patient terminal or a therapist terminal). It is also possible to implement it in the form of an application.

If the therapist wants to perform rehabilitation treatment for the patient, a file that records the patient's information and progress information about the treatment may be stored and maintained in the database. In the event of a patient change, you can check and manage the progress of all patients in charge of the patient by accessing the web / mobile website or logging in to the server with the therapist's ID through the language rehabilitation app. The patient can be added, modified or changed.

Patients can receive speech rehabilitation at home without a therapist, even if all rehabilitation dioceses do not actually exist in their home, by accessing the web / mobile website or logging in with the patient's ID through the speech rehabilitation app. Download rehabilitation content. At this time, the individual treatment is not carried out for each patient, but individual treatment can be tailored to each patient, each patient can select and execute the treatment that is suitable for them. For example, for words belonging to various categories corresponding to the level of the patient set by the therapist, different words for each patient are based on words belonging to a category selected by the patient, rehabilitation contents related to those words, and rehabilitation tools selected by the patient. A form of rehabilitation program may be provided.

Hereinafter, the language rehabilitation treatment service to which the present invention is applied specifically describes an environment implemented as a connection environment of a web / mobile site, but is not limited thereto and may be implemented as a service environment through a language rehabilitation app.

2 is a flowchart illustrating a communication disorder rehabilitation method according to an embodiment of the present invention, and FIG. 3 is a block diagram showing an internal configuration of a communication disorder rehabilitation system according to an embodiment of the present invention.

As shown in FIG. 3, the server for communication failure rehabilitation system 300 according to the present embodiment includes a processor 310, a bus 320, a network interface 330, a memory 340, and a database 350. It may include. The memory 340 may include an operating system 341 and a service providing routine 342. The processor 310 may include a screen providing unit 311 and a content providing unit 312. In other embodiments the communication rehabilitation system 300 may include more components than some of the components of FIG. 3, or some components may be omitted.

The memory 340 is a computer-readable recording medium, and may include a permanent mass storage device such as random access memory (RAM), read only memory (ROM), and a disk drive. Also, program code for the operating system 341 and the service providing routine 342 may be stored in the memory 340. These software components may be loaded from a computer readable recording medium separate from the memory 340 using a drive mechanism (not shown). Such a separate computer-readable recording medium may include a computer-readable recording medium (not shown) such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, and the like. In other embodiments, the software components may be loaded into the memory 340 via the network interface 330 rather than the computer readable recording medium.

The bus 320 may enable communication and data transmission between components of the communication rehabilitation system 300. Bus 320 may be configured using a high-speed serial bus, a parallel bus, a storage area network (SAN) and / or other suitable communication technology.

The network interface 330 may be a computer hardware component for connecting the communication disorder rehabilitation system 300 to a computer network. The network interface 330 may connect the communication rehabilitation system 300 to a computer network through a wireless or wired connection.

The database 350 plays a role of downloading, matching, storing, and maintaining patient information, various types of rehabilitation contents, therapist information, etc., from an online DB (database) system based on a web / mobile web. The patient information may include personal information such as the patient's name, contact information, date of birth, and patient's language ability status, and the therapist information may include information such as the therapist's name, contact information, and career. Here, the rehabilitation contents may represent some of the rehabilitation contents selected by the patient or the therapist by patient among the plurality of rehabilitation contents stored in the DB system.

At this time, the communication disorder rehabilitation system 300 may support a cloud-based rehabilitation treatment service through synchronization with the database 350. In other words, the words included in the rehabilitation treatment list are logged into the user account (eg, patient ID, therapist's ID) when online, and the user of the database 350 if the user (eg, patient, therapist) has set up synchronization. By synchronizing with the information in the database 350 to match the information, words or rehabilitation contents added to the rehabilitation list by another device or PC can be maintained in the list. Thus, the communicative disorder rehabilitation system 300 may allow a patient to receive the same rehabilitation treatment on any device through a synchronized rehabilitation treatment list in the database 350.

The processor 310 may be configured to process instructions of a computer program by performing input / output operations of the basic arithmetic, logic, and communication rehabilitation system 300. The instructions may be provided to the processor 310 by the memory 340 or the network interface 330 and via the bus 320. The processor 310 may be configured to execute program codes for the screen providing unit 311 and the content providing unit 312. Such program code may be stored in a recording device such as memory 340.

The screen providing unit 311 and the content providing unit 312 may be configured to perform the steps 210 to 230 of FIG. 2.

In the content table of the database 350, information such as photographs (i.e., images), phonemes, syllables, words, sounds (i.e., audio signals) of each rehabilitation content is stored, the rehabilitation content selected by the user, and the content providing order The user (eg, identifier information of the patient, identifier information of the selected rehabilitation content, and order information) may be matched and stored and maintained.

For example, the patient or therapist may access the web / mobile website through his terminal and log in by inputting his / her ID and password. When logged in, a main menu screen for rehabilitation treatment may be provided. Patient management, content management, content output, etc. may be selected through the main menu screen. For content output for the rehabilitation treatment of a particular patient, the therapist directly identifies the patient, confirms the patient's verbal status and enters the patient's verbal status (ie the patient's verbal age information) as the patient's personal information. It can be stored and maintained with matching identifier information. The identifier information of the therapist and the identifier information of the patient in charge of the therapist may be matched and stored and maintained. Then, when the therapist is logged in, when the therapist selects his patient list through the main screen, the patient list is provided, and when the patient to be currently treated is selected among the plurality of patients included in the provided patient list, The patient's status information can be controlled to be displayed on the screen. In this case, when there is a history of rehabilitation previously performed, display information for selecting a content list including rehabilitation contents already selected in association with a specific patient may be provided together through the corresponding screen.

When the patient is logged in, when the rehabilitation list is selected through the main screen, a content list including rehabilitation contents previously selected and stored by the therapist or the patient may be provided.

In operation 210, the screen providing unit 311 may provide a screen for selecting a rehabilitation tool based on the language state of the patient for the logged-in user (patient or therapist). Here, the rehabilitation tool may include, for example, an image, a phoneme, a syllable, a word, an audio sound (ie, an audio output), and an augmented reality as a means for providing rehabilitation content.

For example, when there is no history of rehabilitation treatment, the screen providing unit 311 may provide a user who logs in to select a screen for selecting a rehabilitation tool. When the rehabilitation treatment history exists, the screen providing unit 311 may provide the logged-in user with a screen for rehabilitation contents included in the content list, editing of the content list, and the like.

In operation 220, the content providing unit 310 may provide a plurality of rehabilitation contents corresponding to a selected specific category, for a plurality of rehabilitation contents belonging to a level corresponding to a language state of the patient.

For example, if the patient's biological age is 20 years old, but the level corresponding to the patient's language status is 7 years old and corresponds to the child, rehabilitation content associated with words frequently used by the child may be provided. In this case, when training or learning is to be performed on the contents included in the content list on the screen provided in step 210 (that is, the rehabilitation contents that have already been selected due to the history of rehabilitation previously selected) (that is, newly selected rehabilitation contents are selected). Or if the user does not want to add / delete rehabilitation content), step 220 may be omitted. That is, rehabilitation treatment may be performed on rehabilitation contents included in the rehabilitation list.

In operation 230, the content provider 310 may provide at least one rehabilitation content selected by the user among the plurality of provided rehabilitation contents according to the rehabilitation tool selected through the screen. Rehabilitation tools can be used to access desired rehabilitation content (e.g., file rehabilitation content) in real time via the web / mobile web to provide a hint of the rehabilitation content that the patient will learn / learn to the patient. Can be.

For example, when the selected rehabilitation tool is an augmented reality tool, when a marker included in the provided rehabilitation content is recognized, a 3D motion of the rehabilitation content based on the recognized marker may be provided to the user based on the augmented reality. Can be. Here, the marker is a recognition tool for providing rehabilitation contents included in the content list (ie words for rehabilitation treatment and word-related images, texts, etc.) in augmented reality form, and recognizes / learns the patient. The marker may be included for each image corresponding to a specific word.

As another example, when an image is selected as the rehabilitation tool, the content provider 312 may provide an image corresponding to each of the rehabilitation contents included in the rehabilitation list to the patient based on a predetermined content providing order. The content providing order may indicate the order in which the rehabilitation content is selected, or indicate the order (eg, image display order, etc.) of learning the rehabilitation contents included in the rehabilitation list.

As another example, when the selected rehabilitation tool corresponds to a phoneme, the content provider 312 may provide a consonant corresponding to the selected at least one rehabilitation content. For example, if the word you want to learn to the user (i.e., the first word in the content list) is "lion", the initials "s" and "s" correspond to the user (eg, the patient or therapist). Control to display on the screen of the terminal).

As another example, when the selected rehabilitation tool corresponds to a syllable, the content provider 312 may provide at least some of the letters constituting a word corresponding to the selected at least one rehabilitation content. For example, if the word you want to learn to the user (that is, the first word in the content list) is "lion", some of the letters that make up the lion are "sa" or "letter" or some letter and the consonant. "S", "S" can be controlled to be displayed on the screen of the user (eg, the terminal possessed by the patient or therapist).

As another example, when the selected rehabilitation tool corresponds to a word, the content provider 312 may provide an entire character constituting a word corresponding to the selected at least one rehabilitation content. For example, if the word you want to learn to the user (that is, the first word in the content list) is "lion", you can provide the user with all of the letters that make up the lion, that is, "lion" along with the image. have.

As another example, when the selected rehabilitation tool corresponds to an audio sound, the content provider 312 may control to output an audio signal of a word corresponding to at least one selected rehabilitation content. For example, when the word to be taught to the user (that is, the first word included in the content list) is "lion", an audio signal indicating the lion's cry may be output through the user's speaker.

In this case, two or more rehabilitation tools may be selected, and the content provider 312 may control to provide specific rehabilitation content according to the selected two or more rehabilitation tools.

For example, when an augmented reality tool or image is selected, an image corresponding to a specific rehabilitation content is combined with a 3D image representing a real world photographed through a camera of a terminal possessed by a user (ie, a patient or a therapist) based on an augmented reality. Can be provided. In this case, a rehabilitation tool may be additionally selected even in the middle where the image and the 3D image are combined and provided. For example, when the phoneme is selected, the text indicating the consonant of "Zh" may be additionally combined with the augmented reality image provided by combining the desk image with the 3D image representing the real world and provided to the user (ie, the patient).

 In operation 210, when providing a screen for selecting a rehabilitation tool, the screen providing unit 311 may additionally provide a screen for selecting a rehabilitation method for treating speech ability of the patient based on the language state of the patient. . For example, the screen for selecting a rehabilitation tool may be provided to include display information (eg, an icon, a button, etc.) for selecting an evaluation method, or a screen for selecting an evaluation method may be provided separately. Here, the evaluation method, Method 1 for selecting a word based on the provided image, Method 2 for selecting an image based on the provided word, Method 3 for selecting a corresponding word based on the provided audio signal At least one of a method 4 for selecting a corresponding image based on the provided audio signal, and a method 5 for receiving a corresponding audio signal based on the provided image.

4 is a diagram illustrating a screen configuration for patient management according to an embodiment of the present invention.

Referring to 410 of FIG. 4, the therapist may log in by inputting his ID and password through a login screen. Then, when patient management display information is selected on the main screen, the screen provider 320 may provide a screen 420 for patient management. For example, the screen providing unit 320 may control the screens 420 and 430 for managing the patient to be displayed on the screen of the terminal through the web browser of the user terminal accessing the web / mobile website.

Screen 420 may include display information 421 for adding a patient and screen 430 may include display information 422 for deleting a patient, and screens 420 and 430 may further include display information (not shown) for a patient list. have.

In the screen 420, when a patient name to be added to the input window is input and the display information 421 is selected, the input patient name, patient identifier information, and identifier information of the therapist in charge may be matched with a database and stored and maintained. To this end, the processor 210 may further include a patient management unit (not shown), and the processor may add the patient name to the patient list of the therapist.

In the screen 430, a patient name to be deleted is input to the input window, and when the display information 422 is selected, the patient name included in the patient list of the therapist based on the input patient name, patient identifier information, and the identifier information of the therapist. It is possible to control such that the patient name entered among them is deleted.

Although not shown on the screen 430, when the display information 423 for the patient list is selected, the screen providing unit 311 may control the screen 420 to switch the patient list including the patient name of the specific therapist to the screen. Can be. When a specific patient is selected on the patient list, display information for deleting / modifying the name of the patient and display information for displaying personal information of the selected patient may be provided through the corresponding screen. As such, by storing and maintaining information such as patient personal information, a therapist, a rehabilitation list, and a rehabilitation history in a database, the same patient can be prevented from being duplicated, and the therapist can effectively manage his patient and the patient. It can help you manage your status.

The rehabilitation system 300 may provide a screen for content management as well as patient management as shown in FIG. 4. As the content management menu, display information for category addition and deletion, and word addition and deletion may be included in the content management screen. When there is no rehabilitation content included in the content list, if display information indicating the execution of content is selected for rehabilitation treatment, the rehabilitation content may be executed and provided based on a designated rehabilitation tool. In this case, when there is no rehabilitation content included in the content list, the content may not be executed even if display information for executing the content is selected. In this case, by selecting display information for adding a word on the corresponding screen, a word (ie, rehabilitation content) to be included in the content list may be selected by the therapist or the patient. Similarly, as the display information for deleting the word is selected, specific content selected from rehabilitation content included in the content list may be deleted. In this case, a new word may be added in addition to the word already registered in the database by using the word adding function. For example, as the times change, newly coined words (eg, following, honbab, yolo, etc.) may be added to the database.

The display information for adding / deleting a category may be used to add or delete a category in association with a specific patient. That is, the content list is classified by category and may be stored and maintained in a database by matching with identifier information of the corresponding category. For example, if a content list corresponding to a daily life category, a content list corresponding to a travel category, and a content list corresponding to a sports category are stored and maintained in a database in association with identifier information of a specific patient, a new area other than the already stored category If the patient wants to perform the rehabilitation treatment of the patient, a new category (eg, a content list corresponding to the school category) may be added by selecting display information for adding a category. When display information for category deletion is selected, a content list for each category registered in association with the patient may be provided to the user through a screen, and a content list of a specific category selected from the provided content list may be deleted. For example, when the rehabilitation treatment of the category is completed and it is no longer necessary to perform the rehabilitation treatment, the corresponding content list may be deleted.

At this time, the therapist may grasp the rehabilitation progress of a specific category by using statistical data provided based on the evaluation mode, or if the system automatically recognizes the rehabilitation progress and the improvement is over a predetermined value, the specific category May be notified to the therapist and the terminal of the patient that the rehabilitation treatment of the improved. When the degree of improvement over a predetermined period or a predetermined number of times (that is, the number of times the evaluation is performed more than a predetermined predetermined value) corresponds to a cure, the rehabilitation system 300 automatically identifies the therapist and the patient. Noti may be provided indicating that the rehabilitation treatment of the category is excellent and may be terminated. If the therapist agrees to select the rehabilitation end button of the category based on the notification, the content list of the category may be automatically excluded (ie, deleted) from the content lists associated with the patient.

5 is a diagram illustrating a screen configuration for constructing a content list according to one embodiment of the present invention.

Referring to FIG. 5, a category for giving a patient rehabilitation learning (ie, speech rehabilitation therapy) may be selected for words belonging to a level corresponding to a patient's condition.

For example, the screen 510 may include display information 511 for selecting a category, an input window 512 for directly inputting a word to be used for rehabilitation learning among a plurality of words belonging to the selected category, and search for the input word. Display information 513 for displaying information, display information 514 for printing an image of rehabilitation content associated with an input word, and display information 515 for executing rehabilitation content.

For example, if an animal category is selected on the screen 510, images corresponding to a predetermined number of words among a plurality of words belonging to the animal category may be displayed in the area 520. In this case, a specific image (eg, a cat image, etc.) selected by a therapist or a patient among the images displayed in the area 520 may be moved to the area 530 by using a drag and drop function. A content list may be configured for a plurality of images included in the area 530. In this case, when the image desired by the therapist or the patient does not exist or takes a long time to find in the area 520, a desired word such as “dog” may be directly input in the input window 512. Then, an image corresponding to the input word may be searched in the database and provided through the area 520, or an image corresponding to the input word may be added directly to the area 530. Then, when display information such as selection completion is selected, a content list including corresponding rehabilitation content may be configured for the images located in the area 530.

When the display information 514 for printing the image is selected, the image may be printed through a printer connected to the user terminal through a network. In addition, display information for printing all the rehabilitation contents included in the content list at one time may be displayed on the screen, and when the display information is selected, images corresponding to the rehabilitation contents included in the content list are printed at once. Can be. Performing printing is for recognizing markers for providing augmented reality-based rehabilitation content selected by at least one of the therapist and the patient in the practice mode training mode.

For example, the printed image may include a marker that connects the rehabilitation content with the augmented reality content. Then, when the printed image is photographed through a camera provided in the user terminal (eg, the therapist or the patient's terminal), the feature point extractor included in the content provider 312 may extract the feature points from the photographed image. For example, the feature point extractor may generate an integrated image by accumulating brightness values of an image image captured / viewed by a camera. Then, the feature point may be extracted through the detector with respect to the generated integrated image.

Then, the marker recognition unit included in the content providing unit 312 generates a pattern based on the extracted feature points, and based on whether the generated pattern matches a pattern associated with the rehabilitation content stored in the database, the printed pattern is printed. The marker included in the image can be recognized. That is, the augmented reality content corresponding to the printed image can be recognized through the marker. For example, the marker recognizing unit may determine the main direction based on the scale information on which the feature point is detected based on the feature point. In addition, the marker recognizer may generate a descriptor indicating a change in brightness of neighboring pixels in a predetermined area around the feature point. In the case of recognizing augmentation based on an existing predetermined marker of a specific shape, since additional work of inserting a marker is required, it may act as a limit to securing content in speech therapy requiring expansion of various vocabulary lists. Accordingly, as described above, the markerless picture card itself may be recognized as a marker and matched with the 3D object.

That is, the feature point extractor may extract an image captured or projected by the camera, and compare whether the pattern generated based on the extracted feature points matches the pattern stored in the database. At this time, the feature point may be extracted in real time to continuously recognize and track the image. The object may be output on the screen by recognizing a feature point included in the image itself as a marker instead of an existing marker having a specific specific shape. That is, in the past, the shape of the marker included in each image corresponding to each of the plurality of words is the same, but the rehabilitation system 300 uses a unique feature point included in the image itself as a marker, thereby forming the shape of the marker included in each image. This may vary. Then, the content provider 312 may provide augmented reality content corresponding to the recognized marker (ie, pattern) to the user terminal.

At this time, the camera of the tablet or PC is composed of a front and rear, rehabilitation content can be played on the basis of augmented reality using the rear camera. That is, a 3D model in which the rehabilitation content (for example, a motion corresponding to taking a picture) is combined with a 3D image representing a real world through a screen of a tablet may be displayed for the patient to see. In this case, since the image photographed by the rear camera is converted into two-dimensional, correction using a parameter representing a conversion relationship between three-dimensional spatial coordinates of the real world and three-dimensional image coordinates of the camera may be performed. For example, the correction may be performed based on Equation 1 below, and may be based on a transformation relationship in the pinhole camera model.

[Equation 1]

In Equation 1 above, (X, Y, Z) represents the three-dimensional coordinates of reality, and [R | t] is a transformation matrix for converting the coordinates of the three-dimensional point into the two-dimensional coordinate system of the camera. It may correspond to. In Equation 1, A may represent an internal parameter of the camera. As such, based on Equation 1, two-dimensional coordinates of the captured image may be converted into three-dimensional coordinates and corrected in three-dimensional space. In this case, the content providing unit 312 may control not only the object corresponding to the rehabilitation content in the three-dimensional space to be displayed as a two-dimensional still image but also to control the display so that the front of the object is shown while rotating. . For example, when the object corresponds to a "dog", not only the front of the dog, but also the movement of the dog rotating at a predetermined speed 360 degrees can be controlled to be displayed.

6 is a diagram illustrating a screen configuration showing rotation of an object corresponding to rehabilitation content according to one embodiment of the present invention.

When augmented reality is selected as a rehabilitation tool and a content list is selected, rehabilitation contents included in the content list may be displayed on a screen of a user terminal (a terminal owned by a therapist or a patient). At this time, if any one of the rehabilitation content is selected, as shown in the screen 610 and 620 may be provided to the user while the image of the content is gradually increased. If the object corresponding to the rehabilitation content is gradually enlarged and displayed to a predetermined predetermined size as shown in the screen 620, the object may be displayed while rotating in the rotation direction of the camera at a predetermined predetermined speed as shown in the screen 630. For example, the object may be displayed while rotating at a slow speed of 0.3f. If the rotation is too fast, the user may be confused when acquiring the word when learning through the training mode or the practice mode, and may also cause the confusion even in the evaluation mode. This allows the object to be rotated at a predefined slow speed that will not cause confusion. In this case, the space may be divided by three angle values of X, Y, and Z in Equation 1 using Euler angles based on global coordinates based on the screen rather than the object. In order to apply Euler rotation to a specific object, each rotation value may be applied as a rotation around the corresponding axis. Then, by changing only the y value in the object, the object can be controlled to rotate only left and right.

Images captured / recognized by the camera have unique feature points and recognize the markers based on patterns based on the feature points, so that even if the marker moves on the screen where augmented reality content is provided, the video stream is continuously input through the camera. I can accept it. Accordingly, the pattern corresponding to the marker is continuously recognized, and the objects included in the image may be controlled to move together as the real world where the camera is viewed moves.

7 is a diagram illustrating a rehabilitation treatment screen configuration according to one embodiment of the present invention.

Referring to FIG. 7, an image may be provided on the rehabilitation treatment screen 710 in order of providing contents to rehabilitation contents included in the content list.

For example, when the display information 515 for the content execution is selected in FIG. 5 or when the display information for the content execution is selected on the screen in which the content list related to the specific patient is provided, the content provider 312 may display the content list. The first image among the included rehabilitation contents may be provided in the content providing order.

Referring to the screen 710, the screen 710 may include display information 711, 712, 713, 714, 715, 716, 717 for changing or adding a rehabilitation content rehabilitation tool provided in an area 720. For example, when the display information 712 and 716 are selected, an image of the dog moving may be displayed in the area 720, and an audio signal corresponding to the sound of the dog being held by the speaker may be output. When the display information 714 is selected, the text 720 may overlap with the dog image or may display the text "ㅇ ㅈ" indicating the dog's initials around the dog image.

8 is a diagram illustrating a configuration of a rehabilitation mode selection screen according to an embodiment of the present invention.

Referring to FIG. 8, the rehabilitation mode is a training mode 810 for providing speech therapy in a space separate from the treatment institution, and a practice mode for providing speech therapy with a therapist belonging to the treatment institution. 820, and an evaluation mode 830 for evaluating at least one mode performing result of the training mode and the practice mode.

For example, when a therapist or a patient logs in by accessing a web / mobile website, display information for selecting the rehabilitation mode may be located in a specific area on the main screen. When the display information is selected, the rehabilitation mode selection screen 800 may be displayed on the screen of the user terminal (eg, a terminal possessed by a therapist or a patient) through a web / mobile web browser.

When face-to-face with the therapist and undergo rehabilitation treatment, the rehabilitation treatment may be provided based on a list of contents generated based on words selected by the therapist among words of the level corresponding to the therapist's input patient's condition. That is, the rehabilitation treatment may be performed based on the content list based on the rehabilitation contents configured by the therapist to be customized based on the patient's condition. In this case, the rehabilitation tool is also suitable for the patient may be selected by the therapist, the therapist may select the rehabilitation tool desired by the patient.

The training mode 810 may indicate a mode in which the patient rehabilitates at home alone, based on a list of contents used in the rehabilitation treatment with the therapist through the practice mode 820 without directly facing the therapist. In this case, the rehabilitation tool may be changed to a tool different from the tool performed with the therapist in the practice mode 820. For example, in the practice mode 820, the rehabilitation treatment was performed using the image and sound as rehabilitation tools, but in the training mode 820, the patient rehabilitation treatment based on a selective ring rehabilitation tool (eg, augmented reality, voice recording, etc.). Can be performed. That is, the identifier information of the rehabilitation tool previously specified in association with the content list may be changed to the identifier information of the selected rehabilitation tool by selecting the rehabilitation tool to be used by the patient.

When the evaluation mode 830 is selected, while playing the video corresponding to the rehabilitation content on the screen, "What are you doing?" Pre-specified questions such as may be output along with the image through the screen. For example, if you want to use the word "to take a picture" for rehabilitation therapy, the picture you are taking in the living room is played, and "What are you doing?" And the like may be displayed. Then, the correct answer rate is counted based on whether "take" is input through the input window, an audio signal corresponding to "take" is input based on the voice recording, or "take" is selected among the presented words. The progress of the rehabilitation therapy can be scored and evaluated based on the counted correct rates. Then, when the request for the statistical data occurs, the processor 210 may provide the statistical data to the therapist, the patient, and the family of the patient with the result data evaluated for a predetermined period of time. For example, statistical data may be provided in the form of a bar graph and a table.

9 is a diagram illustrating a category selection screen provided in a training mode according to one embodiment of the present invention.

9, if a user (eg, a patient) selects a training mode, a category selection screen 910 may be provided. For example, the category selection screen 910 may include display information for selecting a word of a category to be trained by a user among words (eg, a child word) corresponding to a level preset by a therapist according to the language state of the patient. Can be. Here, the display information may correspond to the images 911, 912, 913, and 914 representing the room category, the living room category, the bathroom category, and the kitchen category. When an image 911 of a room category is selected through a user touch or a mouse cursor, a plurality of words belonging to the selected room category and images matched with the words may be displayed on the screen 920. have. Similarly, if the living room category is selected, words belonging to the living room category and images matching those words are displayed on the screen 920, and if the bathroom category is selected, the words belonging to the bathroom category and the corresponding words on the screen 930 Images matching may be displayed. When the kitchen category is selected, words belonging to the kitchen category and images matching the words may be displayed.

FIG. 10 is a diagram illustrating a rehabilitation treatment screen provided in a training mode according to one embodiment of the present invention.

When the word “wear” is selected on the screen 920 in which the words belonging to the room category of FIG. 9 are displayed, an animation indicating an operation of wearing clothes may be reproduced through the screen 1010.

Referring to the screen 1020, when the display information 911 that is displayed on the screen 930 of FIG. 9 is selected, and when “take a fork” is selected, an animation indicating an operation of forking a fruit through the screen 1020 may be played. have.

Referring to the screen 1030, when “wash” is selected on the screen 940 of FIG. 9, an animation indicating an operation of washing the bathroom in the bathroom may be played through the screen 1030.

Referring to screen 1040, when “drinking” is selected on screen 950 of FIG. 9, an animation representing an operation of drinking water or a drink in the kitchen using a cup may be reproduced through screen 1040.

11 to 15 are diagrams illustrating a screen configuration provided in the evaluation mode.

FIG. 11 is a diagram illustrating an evaluation mode screen of Method 1 for selecting a word based on a provided image according to an embodiment of the present invention.

Referring to FIG. 11, the evaluation mode is a mode for evaluating the progress of a patient through rehabilitation treatment for a certain period of time through a training mode and a practice mode. The evaluation method is for selecting a word based on an image provided. Method 1 1111, Method 2 1112 for selecting images based on provided words Method 3 1113 for selecting corresponding words based on provided audio signals Based on provided audio signals And at least one of a method 4 1114 for selecting a corresponding image and a method 5 1115 for receiving a corresponding audio signal based on the provided image. That is, the training mode and the practice mode may perform the test without calculating the evaluation score, and the evaluation mode may indicate that the test is performed while calculating the evaluation score.

Method 1 (1111) for selecting words based on images is a picture and letter search test in which text representing "chair and desk", "sofa", "stand", and "bed" is provided as an option. For example, an image including a specific object may be provided as a problem.

For example, when the display information 1121 indicating the evaluation is selected on the screen 1120, the texts representing “chair”, “sofa”, “stand”, and “bed” are displayed as images on the image representing the real world as shown in the screen 1130. In combination, an AR image may be provided to the user. In addition, as a problem paper, an image 1131 representing a sofa may be combined with an augmented reality based image to which the option is combined. In this case, when the text 1132 representing the sofa is selected through a user touch or a mouse cursor or a keyboard input, the text 1133 representing the sofa may be displayed on the problem paper. As such, if the text selected by the patient corresponds to the picture in question, that is, the correct answer, the corresponding text (ie, text) disappears on the screen 1140 as shown in screen 1140 and the corresponding image (ie, sofa, 1141) is displayed. It may be arranged at a predetermined position. In this case, if the answer is incorrect, an audio signal such as "beep" or "No" may be output, and the control may be performed so that no change occurs on the screen 1130. The test may proceed until a suitable letter is selected for a predetermined time. Referring to the screen 1140, when the answer is correct, an image 1142 corresponding to the next problem may be included in the questionnaire coupled to the AR-based image and displayed. As such, in order to confirm whether the answer is correct or incorrect, the identifier information of the image and the text information represented by the image may be matched and stored in advance, and the identifier information and the corresponding text information of each text provided in the option are matched and stored in the database. Can be stored and maintained.

12 is a diagram illustrating an evaluation mode screen when a method 2 for selecting an image based on a word provided as an evaluation method is selected according to an embodiment of the present invention.

Referring to FIG. 12, Method 2 1112 for selecting an image based on a provided word is a letter-looking picture-finding test that includes images representing “chair and desk”, “sofa”, “stand”, and “bed”. As an option, each text may be combined with a 3D image representing the real world as a problem paper and provided to the user as an AR image.

For example, referring to screen 1210, if a user selects an image 1211 corresponding to text (eg, “bed”) from an option, and the selected image is a correct answer, the processor 310 displays audio indicating a predetermined correct answer. The signal may be controlled to be output through the user terminal (ie, the terminal possessed by the patient). The image 1211 may be controlled to disappear from the screen 1210. If the answer is incorrect, the audio signal indicating a pre-specified incorrect answer may be output, and the screen 1210 may be maintained without change. The text-looking picture-finding test may be performed until all the images corresponding to all the furnitures displayed on the screen 1210 disappear. If the answer is correct, the following text ("chairs") may be displayed on the questionnaire, as shown in screen 1220. At this time, when the user wants to end the test in the middle of the test, the middle of the test may be terminated by selecting the menu icon 1212. In addition, the evaluation method may be changed in the middle through the menu icon 1212. For example, the picture report text finder can be changed to the text report picture finder, the name hear the letter finder, the name finder and the picture finder.

FIG. 13 is a diagram illustrating an evaluation mode screen provided with method 3 for selecting a corresponding word based on an provided audio signal according to an embodiment of the present invention.

In FIG. 13, the method 3 is a name search test, and the texts representing “chair”, “sofa”, “stand”, and “bed” as the screen 1310 are combined with an image representing the real world as an option and augmented reality. The base image may be provided to the user. And, as a problem paper, an audio signal in which a person pronounces "sofa" can be output through a speaker. For example, when play display information 1311 is selected, the audio signal may be output. Then, as shown in screen 1320, a patient who listens to an audio signal selects a corresponding text (eg, sofa, 1321) from an option, and if the selected text is a correct answer, an audio signal indicating a correct answer is output, and an incorrect audio signal indicating a wrong answer. May be output. If the answer is correct, the text may disappear from the selection, as shown in screen 1330, and a test (ie, evaluation) may be performed until all the text disappears.

FIG. 14 is a diagram illustrating an evaluation mode screen provided with Method 4 for selecting a corresponding image based on an audio signal provided according to an embodiment of the present invention.

In FIG. 14, the method 4 is a name-finding test. As shown in screen 1410, an image representing "chair", "sofa", "stand", and "bed" is combined with an image representing the real world as an option and augmented reality. The base image may be provided to the user. And, as a problem paper, an audio signal in which a person pronounces "sofa" can be output through a speaker. For example, when play display information 1411 is selected, the audio signal may be output. Then, as shown in screen 1420, a patient who listens to an audio signal selects a corresponding image (for example, a sofa image) from an option. If the selected image is a correct answer, an audio signal indicating a correct answer is output. Can be output. If the answer is correct, the image may disappear from the selection, such as screen 1420, and a test (ie, evaluation) may be performed until all images disappear from screen 1420.

FIG. 15 is a diagram illustrating an evaluation mode screen provided with a method 5 for receiving a corresponding audio signal based on a provided image.

In FIG. 15, the method 5 is a picture-looking name test, in which an image representing the real world is randomly selected from among images representing “chair”, “sofa”, “stand”, and “bed” as shown in screen 1510. Augmented reality-based image may be provided to the user in conjunction with. When the display information 1511 indicating the recording is selected, the user's voice input through the microphone may be recorded and converted into an audio signal. Then, the correct answer may be determined by comparing whether the converted audio signal matches the audio signal matched with the image (eg, bed image) provided as the question paper. In the case of a correct answer, an audio signal representing a correct answer may be output, and in the case of an incorrect answer, an audio signal representing an incorrect answer may be output. When the display information 1512 representing the next word is selected, the next word-related image may be provided in combination with the augmented reality-based image as a problem paper. In this case, text corresponding to the recognized audio signal may be provided as shown on the screen 1520.

The evaluation results (ie, evaluation data including the noon answer rate) provided in FIGS. 11 to 15 may be stored and maintained in a database by matching patient identifier information (eg, patient ID). In addition, the processor 310 of FIG. 3 may further include a statistical processor (not shown) for statistics. When the statistical processor requests statistical data from a patient or a therapist, the entire problem of each category is prepared based on the evaluation data. Correct and incorrect percentages can be provided in tabular and graphical form.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments are, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable gate arrays (FPGAs). Can be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (5)

In the computer-implemented communication disorder rehabilitation method,
Providing a screen for selecting a rehabilitation tool based on the language status of the patient;
Providing a plurality of rehabilitation contents corresponding to a specific category selected for a plurality of rehabilitation contents belonging to a level corresponding to the language state of the patient; And
Providing at least one rehabilitation content selected from the plurality of rehabilitation contents according to a rehabilitation tool selected through the screen;
Providing according to the selected rehabilitation tool,
If the selected rehabilitation tool is an augmented reality tool, providing the selected at least one rehabilitation content on an augmented reality basis; And
Providing a three-dimensional movement of the provided rehabilitation content on the basis of augmented reality as a marker included in the provided rehabilitation content is recognized,
Providing the 3D motion of the provided rehabilitation content on the basis of augmented reality,
Gradually expanding and displaying an object of the provided rehabilitation content; And
If the object is enlarged to a predetermined predetermined size, rotating and displaying the object at a predetermined constant speed;
Providing according to the selected rehabilitation tool,
Displaying play indication information together with the provided rehabilitation content; And
If the play indication information is selected, outputting an audio signal indicating the provided rehabilitation content through a speaker;
Providing according to the selected rehabilitation tool,
If at least two rehabilitation contents of the plurality of rehabilitation contents are selected, providing the selected at least two rehabilitation contents based on a content providing order set for the selected at least two rehabilitation contents;
Rehabilitation method of communication disorders characterized in that. The method of claim 1,
Providing according to the selected rehabilitation tool,
If the selected rehabilitation tool corresponds to a phoneme, providing a consonant corresponding to the selected at least one rehabilitation content;
If the selected rehabilitation tool corresponds to a syllable, providing at least a part of letters constituting a word corresponding to the selected at least one rehabilitation content; And
If the selected rehabilitation tool corresponds to a word, providing a whole character constituting a word corresponding to the selected at least one rehabilitation content;
Communication disorder rehabilitation method further comprising. The method of claim 1,
Before the step of providing a screen for selecting the rehabilitation tool, providing a screen for selecting a rehabilitation mode to proceed
Communication disorder rehabilitation method further comprising. The method of claim 3,
The rehabilitation mode, the training mode for providing speech therapy in a space separate from the treatment institution, the practice mode for providing speech therapy with the therapist belonging to the treatment institution, and the training mode and At least one of the evaluation modes for evaluating the performance of at least one mode of the practice modes
Rehabilitation method of communication disorders characterized in that. delete

Images