KR20220108851A - Method of providing virtual reality for learning based on voice recognition - Google Patents

Abstract

Disclosed is a voice recognition-based virtual reality learning providing method. The voice recognition-based virtual reality learning providing method according to the present invention, for recognizing a user's voice and interacting with a virtual reality character, comprises the steps of: displaying situation information on a display unit of an HMD; when voice information is input, allowing a voice analysis unit of the HMD to determine whether the input voice information is matched with situation information prepared in advance in a situation information storage unit of the HMD, and when the voice information matched with the situation information is input, providing an interaction gesture and interaction sound of a character through the display unit and a speaker unit of the HMD; when sound information not matched with the situation information is input, allowing the sound information analysis unit of the HMD to divide the sound information into predetermined time intervals, compare the divided sound information with sound information of an ontology-based word database unit and extract one or more keyword information; and allowing a phrase information generation unit of the HMD to convert the extracted one or more keywords into word information, sentence information, or syntax information and then determine whether the converted information is matched with situation information previously prepared in the situation information storage unit. Learning consistency can be provided.

Description

Method of providing virtual reality learning based on voice recognition

The present invention relates to a method of providing virtual reality learning, and more particularly, to a method of providing virtual reality learning based on voice recognition.

The HMD (Head Mounted Display) device related to the provision of virtual reality is provided in a form in which a terminal device capable of reproducing a 360° 3D image is installed therein. When the user mounts the HMD device on the head and then drives the terminal device, a 3D image, that is, virtual reality content, is played on the screen of the terminal device. The HMD device is equipped with various sensors, such as a gyro sensor, and provides a 3D image corresponding to a user's movement based on these sensors.

A real industrial site is the best place to learn practical skills, but damage can occur due to dangerous or incorrect operation. In the case of providing real technology through virtual reality, it is possible to reduce risks and damage and provide an environment similar to the real one for efficient learning, so efforts are being made to utilize it. By composing a virtual environment and virtual space and providing it to users, it minimizes physical restrictions and enables a variety of experiences.

While the HMD is worn, the external environment collected through a camera mounted on the outside of the HMD may be displayed to the user. Because of this blocking purpose, the user completely depends on the UI of the virtual reality for learning, so it may be a rather passive learning rather than an active learning.

In particular, when it is necessary to manipulate the target object at each stage of the actual technology or to deliver a voice message to the object, the learning assistant understands this and instructs the student to manipulate the object or input the voice each time to proceed with the practical technology learning. Not only does it impair the sense of presence, but it also becomes a very cumbersome procedure, making it difficult to provide effective education.

Taking nursing practice technology as an example among specific practical skills practice, one of the most important learning contents in nursing practice technology is the learning of communication with an object set as a patient in a simulation environment. The conventional practical technology learning method using virtual reality does not provide a UI to communicate with the patient, so there is a problem in that it is difficult to learn or evaluate smooth care.

A voice recognition-based virtual reality learning providing method according to an embodiment of the present invention aims to provide a voice-based interaction environment with a third character in a virtual environment for special purpose learning.

An object of the voice recognition-based virtual reality learning providing method according to another embodiment of the present invention is to enable effective interaction with a third character by enabling improvement of a recognition rate according to specific language characteristics.

A voice recognition-based virtual reality learning providing method according to another embodiment of the present invention aims to provide a virtual environment in which input of erroneous interaction information is blocked by filtering surrounding noise or voice provided from a third party.

A voice recognition-based virtual reality learning providing method according to another embodiment of the present invention provides an environment similar to a place where actual technology is performed by randomly generating various noises that may occur in an environment in which learning is performed, but the provided noise is not real. It aims to provide without affecting the interaction required for skill learning.

Objects of the present invention are not limited to those mentioned above, and other objects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description.

A voice recognition-based virtual reality learning providing method for recognizing a user's voice and interacting with a virtual reality character according to an aspect of the present invention for solving the above technical problem, the method comprising: displaying situation information on a display unit of an HMD; When there is input of voice information, the voice analysis unit of the HMD determines whether or not it matches the context information prepared in advance in the context information storage unit of the HMD, and when there is input of voice information matching the context information, the display unit and providing an interaction gesture and an interaction sound of the character through a speaker unit of the HMD; When there is input of sound information that does not match the context information, the sound information analysis unit of the HMD divides the sound information at predetermined time intervals, and compares the divided sound information with the sound information of the ontology-based word database unit. extracting one or more keyword information; and determining whether the phrase information generating unit of the HMD uses the extracted one or more keywords to convert it into word information, sentence information, or syntax information, and then determines whether it matches the context information prepared in advance in the context information storage unit; .

Here, the predetermined time interval may be calculated based on each time interval required to output a word, phrase, or sentence included in the previously prepared context information, and may be used to divide the sound information.

Here, the displaying of the context information may further include displaying a third character included in the context information.

Here, the displaying of the third character may further include outputting arbitrary sound information generated from an arbitrary space in which the third character is disposed through a speaker unit of the HMD.

Here, the arbitrary sound information may include filtering sound information corresponding to a frequency of a predetermined region that can be recognized by the sound information analysis unit.

Here, the step of extracting one or more keyword information by comparing it with the sound information of the ontology-based word database unit; storing the extracted one or more keyword information in the ontology-based word database unit; may further include.

Here, the sound information analysis unit may be provided to select at least one language in advance.

Here, the displaying of the context information may further include displaying a language selection user interface for selecting the language in advance.

Here, the displaying of the context information may further include determining whether the user's gaze direction received from the gyro sensor provided in the HMD is directed toward an arbitrary virtual space area in which the character is located. .

According to another aspect of the present invention, there is provided a voice recognition-based virtual reality learning providing method for recognizing a user's voice and interacting with a virtual reality character, the method comprising: displaying situation information on a display unit of an HMD; When there is input of voice information, the voice analysis unit of the HMD determines whether or not it matches the context information prepared in advance in the context information storage unit of the HMD, and when there is input of voice information matching the context information, the display unit and providing an interaction gesture and an interaction sound of the character through a speaker unit of the HMD; When there is input of sound information that does not match the context information, the sound information analysis unit of the HMD divides the sound information at predetermined time intervals, and compares the divided sound information with the sound information of the ontology-based word database unit. extracting one or more keyword information; Performing a step of converting the phrase information generating unit of the HMD into word information, sentence information or phrase information using the extracted one or more keywords and determining whether the situation information matches the situation information prepared in advance in the context information storage unit; A storage medium in which a program is stored is provided.

The voice recognition-based virtual reality learning providing method according to an embodiment of the present invention can provide learning consistency by providing a voice-based interaction environment with a third character in a virtual environment for special-purpose learning.

The voice recognition-based virtual reality learning providing method according to another embodiment of the present invention can improve the recognition rate according to specific language characteristics so that effective interaction with the third character can be made.

The voice recognition-based virtual reality learning providing method according to another embodiment of the present invention can provide a virtual environment in which the input of erroneous interaction information is blocked by filtering surrounding noise or voice provided from a third party.

A voice recognition-based virtual reality learning providing method according to another embodiment of the present invention provides an environment similar to a place where actual technology is performed by randomly generating various noises that may occur in an environment in which learning is performed, but the provided noise is not real. It can be provided without affecting the interaction required for skill learning.

Effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description.

1 is a diagram illustrating a system block diagram of an HMD for providing a voice recognition-based virtual reality learning providing method according to an embodiment of the present invention.
2 is a flowchart illustrating a method for providing virtual reality learning based on voice recognition according to an embodiment of the present invention.
3 is a conceptual diagram illustrating voice recognition of a method for providing virtual reality learning based on voice recognition according to an embodiment of the present invention.

Objects and effects of the present invention, and technical configurations for achieving them will become clear with reference to the embodiments described below in detail in conjunction with the accompanying drawings. In describing the present invention, if it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. And the terms described below are terms defined in consideration of donation in the present invention, which may vary depending on the intention or custom of the user or operator.

However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. Only the present embodiments are provided so that the disclosure of the present invention is complete, and to fully inform those of ordinary skill in the art to which the present invention belongs, the scope of the invention, the present invention is defined by the scope of the claims it will only be Therefore, the definition should be made based on the content throughout this specification.

Throughout the specification, when a part "includes" or "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated. . In addition, terms such as "...unit", "...unit" or "...module" described in the specification mean a unit that processes at least one function or operation, which includes hardware or software or hardware and It can be implemented by a combination of software.

In addition, terms such as "HMD...unit", "HMD...unit" or "HMD...module" described in the specification refer to the HMD itself or an HMD equipped to exchange data with the HMD by wire or wireless communication. It may mean general-purpose computers, special-purpose computers, portable notebook computers, network computers, mobile devices such as smart phones, online game service providing servers, or other programmable data processing equipment for extending the functions of

On the other hand, in an embodiment of the present invention, each of the components, functional blocks or means may be composed of one or more sub-components, and the electrical, electronic, and mechanical functions performed by each component include an electronic circuit, It may be implemented with various well-known devices or mechanical elements, such as an integrated circuit, an application specific integrated circuit (ASIC), etc., and may be implemented separately or two or more may be integrated into one.

Also, combinations of each block in the accompanying block diagram and each step in the flowchart may be performed by computer program instructions. These computer program instructions may be loaded into the processor of a general-purpose computer, special-purpose computer, portable notebook computer, network computer, mobile device such as a smart phone, an online game service providing server, or other programmable data processing equipment, so that the computer device or other program The instructions, which are executed via a processor of possible data processing equipment, will create means for performing the functions described in each block of the block diagrams described below or in each step of the flow diagrams. These computer program instructions may also be stored in a computer readable memory or memory available to a computer device that may direct a computer device or other programmable data processing equipment to implement a function in a particular manner, such that each block of the block diagram Alternatively, it is also possible to produce an article of manufacture containing instruction means for performing the functions described in each step of the flowchart. The computer program instructions may also be mounted on a computer device or other programmable data processing equipment, thereby creating a process for performing a series of operational steps on the computer device or other programmable data processing equipment, such that each block in the block diagram and the flowchart It is also possible to provide steps for executing the functions described in each step of .

Further, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function(s). It should also be noted that in some alternative embodiments it is also possible for the functions recited in blocks or steps to occur out of order. For example, it is possible that two blocks or steps shown one after another may in fact be performed substantially simultaneously, or that the blocks or steps may sometimes be performed in the reverse order according to the corresponding function.

In an embodiment of the present invention, user equipment means all calculation means that can collect, read, process, process, store, and display data such as desktop computers, notebook computers, smart phones, PDA's, mobile phones, and game machines. do. In particular, the user device according to the embodiment of the present invention is a device having a function of executing software written in a decipherable code, and displaying and delivering it to the user. In addition, if necessary, the software may be stored by itself or may be read together with data from the outside.

In addition, the terminal according to the embodiment of the present invention includes functions such as input, output, and storage as well as the above data processing function, and for this purpose, a CPU, a main board, a graphic card, a hard disk, Various elements such as sound card, speaker, keyboard, mouse, monitor, USB, communication modem, as well as CPU, main board, graphic chip, memory chip, sound engine, speaker, touchpad, USB, etc. It may include an external connection terminal, a communication antenna, a communication modem capable of implementing communication such as 3G, LTE, LTE-A, WiFi, and Bluetooth. These various elements alone or two or more together, or a part of various elements may be combined to implement one or more functions, and in the embodiment of the present invention, a device indicated by one or more blocks in the drawings or detailed description, or The parts may mean that various elements included in the above-mentioned user equipment represent one or more functions alone, two or more together, or some of the various elements are combined.

Meanwhile, in an embodiment of the present invention, a user device, HMD, etc. may have a communication function, and in order to implement the communication function, wired Internet, wireless Internet, infrared communication, Bluetooth, WCDMA, Wibro, WiFi, LTE, LTE-A, Various network means such as 5G wired/wireless telephone network may be provided.

Hereinafter, a method for providing virtual reality learning based on voice recognition according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a system block diagram of an HMD for providing a voice recognition-based virtual reality learning providing method according to an embodiment of the present invention. Although not shown, voice or other sound information provided by the user may be collected by a microphone unit provided in the HMD.

Referring to FIG. 1 , the HMD includes a display unit, a voice analysis unit, a speaker unit, an acoustic information analysis unit, a word database unit, and a phrase information generation unit.

The display unit of the HMD is provided to display an image of the virtual reality space to the user. The display unit may be provided to cover the entire field of view of the user and, when the user is wearing the HMD, may be provided to block the light penetrating from the outside by the display unit.

Next, the voice analysis unit is provided to determine whether voice information matching the context information prepared in advance is input to the context information storage unit of the HMD. In an embodiment of the present invention, the voice analysis unit is provided to reduce the load on the system by determining whether the phrase is identical to the phrase of the specific context information that the user is familiar with.

Next, the speaker unit is provided to deliver a voice or sound effect according to an event occurring in the virtual space of the display unit to the user. In this case, the speaker unit may include at least two or more speaker units, and may be provided to output voice or sound information having the same orientation as a specific location in a virtual space that the user sees.

Next, the sound information analysis unit may be provided to analyze the voice or sound that has been filtered by the voice analysis unit and determined to require interpretation, and compares it with the sound information of the ontology-based word database unit to extract a keyword.

Next, the word database unit may be provided to store or update word data based on the components of the ontology in advance.

In describing an embodiment of the present invention, an ontology is an expression of real or human-recognizable things in a form that can be handled by a computer, and the ontology components are an entity (E) (or an instance), It can include a class (C), a property (P), and a value (V).

More specifically, a class among the components of an ontology may generally mean a name attached to an object or concept. Words such as "keyboard", "monitor", and "love" can be divided into classes. Next, an instance may refer to itself expressed in a substantial form such as a concrete object or event of a thing or concept. In other words, "X's XT-10 slim keyboard", "Y's master view wide OLED monitor", and "Romeo and Juliet's love" can mean instances. The distinction between class and instance like this is that depending on the application and purpose of use, the object of the same expression may become a class in some cases and an instance in other cases. Next, a property may be defined as linking a class or instance with a specific value in order to represent a specific property or tendency of the class or instance. For example, to express "Y's masterview wide OLED monitor is XX inches", a property such as hasSize can be defined. Next, the relation refers to the relationships that exist between properties between entities or between properties, classes, and instances, and can be generally divided into taxonomic and non-taxonomic relations. have. Additionally, ontology components may further include a function term, a restriction, a rule, an event, and the like. The ontology-based word database unit may store vast amounts of knowledge data based on the ontology. For example, the ontology-based word database unit may include knowledge data expressed using RDF (Resource Description Framework), and a triple as a unit of knowledge data. this can be used The ontology-based word database unit may return a triple in response to a query, for example, a SPARQL Protocol and RDF Query Language (SPARQL) query. In some embodiments, the acoustic information analysis unit may classify the user's voice into an ontology component in a natural language recognition system (not shown) external to the phrase information generation unit, and receive tokens extracted from the user's voice from the natural language recognition system can do.

The phrase information generator may receive tokens from an external natural language recognition system, and obtain semantic resources corresponding to the received tokens from an ontology-based database. For example, for the first query sentence Q1 of "description of blood glucose measurement test", entity "<Entity-Subject>" may be obtained as a semantic resource of "glucose measurement", and as a semantic resource of "test" You can get the property "<Property_Method>". Accordingly, the first pattern P1 may include "<Entity-Subject>" and "<Property_ Method>" as semantic resources. Also, for the second query sentence Q2 of "purpose and procedure description", entity "<Entity_Subject>" may be obtained as a semantic resource of "purpose", and attribute "<Peoperty_Method>" as a semantic resource of "procedure" " can be obtained. The second pattern P2 may include “<Entity-Subject>” and “<Property_Method>”, and thus may be the same as the first pattern P1 .

The keyword extracted by the acoustic information analysis unit may correspond to one of the ontology components. In describing an embodiment of the present invention, a class may be interpreted as word information, an instance as syntax information, and an attribute as corresponding to sentence information.

Hereinafter, a voice recognition-based virtual reality learning providing method according to an embodiment of the present invention will be described with reference to FIGS. 2 and 3 .

2 is a flowchart of a voice recognition-based virtual reality learning providing method according to an embodiment of the present invention, and FIG. 3 is a conceptual diagram illustrating voice recognition of a voice recognition-based virtual reality learning providing method according to an embodiment of the present invention. did it

Referring to FIG. 2 , the voice recognition-based virtual reality learning providing method according to an embodiment of the present invention is a voice recognition-based virtual reality learning providing method for recognizing a voice to interact with a virtual reality character, and context information on the display unit of the HMD. and displaying ( S110 ).

For example, in voice recognition-based virtual reality learning for providing nursing room technology learning, a phrase describing a situation facing a patient character lying on a bed in a hospital room is displayed as a sentence or symbol on the HMD display unit to display the HMD. Context information can be displayed so that the wearer recognizes the current state.

For example, if the learner corresponds to a situation for practicing measuring the blood sugar of a patient character faced by the learner, a phrase such as 'a situation in which a patient's blood sugar measurement is required' may be displayed on the display unit.

In this case, when a message requesting to transmit the current situation to the patient character by voice is displayed on the display unit, the learner may generate voice in words, phrases or sentences to input voice information into the HMD.

Meanwhile, the displaying of the context information may further include determining whether the user's gaze direction, received from the gyro sensor provided in the HMD, is directed toward an arbitrary virtual space area in which the character is located.

For example, in order to describe a situation in which a patient should face and have a conversation, it is possible to practice and familiarize with the interaction close to reality by determining whether the gaze direction of the user (learner) looks at the virtual space area where the patient character is located. .

In this case, the step of displaying the context information may further include displaying a third character included in the context information. In the case of practicing nursing skills, there is a possibility that a third party may appear among events that may occur in the actual medical field, or noise that may interfere with performing actual techniques may occur. In order to properly practice the actual technique even in such a situation, the step of displaying the context information may further include the step of displaying a third character included in the context information.

In this case, the third character may correspond to another patient, guardian, doctor, another nurse, etc., and a conversation that can be expected in the medical field may be provided to the practical skill learner.

The displaying of the third character may further include outputting arbitrary sound information generated from an arbitrary space in which the third character is disposed through a speaker unit of the HMD.

Through these steps, sound information generated at any point in the virtual space provided through the HMD can be provided, thereby providing a sense of presence close to reality and reality.

Meanwhile, the arbitrary sound information provided by the third character may include filtering sound information corresponding to a frequency of a predetermined region that can be recognized by the sound information analysis unit.

This is to prevent a learner's voice or sound other than the learner's voice from acting as an input variable of the voice recognition-based virtual reality learning providing method according to an embodiment of the present invention.

For example, a low-frequency sound or a high-frequency sound in a region other than the human audible frequency may be included in the arbitrary acoustic information to distinguish it from information input to the system.

Next, when there is input of voice information, the voice analysis unit of the HMD determines whether or not it matches the context information prepared in advance in the context information storage unit of the HMD, and when there is input of voice information matching the context information, and providing an interaction gesture and an interaction sound of the patient character through the display unit and the speaker unit of the HMD (S120).

For example, the learner transmits word, phrase, or sentence information for each situation that he or she has learned in advance to the HMD in the form of voice through a microphone. can Since a repetitive phrase is used formally, when the same word, phrase or sentence is input, an interaction gesture or an interaction sound corresponding to the response of the patient character can be checked.

However, if the learner does not fully understand the corresponding words, phrases or sentence information for the given virtual reality environment, an arbitrary sentence according to the situation can be input by voice, and the following steps are continued.

Meanwhile, the sound information analysis unit may be provided to select at least one language in advance. In each language, such as Korean, English, French, Vietnamese, Chinese, Indonesian, Hindi, and Arabic, a specific pronunciation area is strongly pronounced or omitted, so learn using a certain language in advance to increase the recognition rate in response to the characteristics of each language It is desirable to specify what to do.

In the case of Korea, due to the continuous increase of foreign patients, the language used by each patient may vary, and even if there is no deep understanding of each language, by familiarizing yourself with sufficient phrases to provide information to patients in each situation, practical skill learners can learn from patients of various cultures. You can practice your medical skills.

Here, the step of displaying the context information may further include displaying a language selection user interface for selecting the language in advance.

For example, the user interface may be provided so that the user can select which language to pronounce by scrolling or touching the screen. However, as long as the interface is intuitive and the learner can check it on the display unit, any one is not limited.

Next, when there is input of sound information that does not match the context information, the sound information analysis unit of the HMD divides the sound information at predetermined time intervals, and divides the divided sound information into the sound information of the ontology-based word database unit. and extracting one or more keyword information by comparing with (S130).

Referring to FIG. 3 , the concept of voice recognition of a user (a learner) is briefly illustrated. First, the pronunciation generated by the user (USER) is subjected to a commercial artificial intelligence-based voice preprocessing procedure, and the time boundary information for each ontology-based word is output through the search engine, and the keyword is output by word phrase (phrase) and sentence unit. can be collected Next, the voice recognition efficiency can be improved by specifying the voice information in a normal state by continuously inputting and feeding back the user's voice using the acoustic model.

In addition, the performance degradation can be minimized by preventing the performance degradation of a specific language change by applying the high-speed application learning technology to the speech recognition as described above.

Through this procedure, a procedure of analyzing the meaning of continuous speech and interworking with an ontology-based knowledge base can be performed with text. These procedures are detailed below.

Next, variables such as pronunciation and frequency domain that may be different for each language by designating a specific language may be applied to the method according to the embodiment of the present invention.

Due to the language of the learner, the pronunciation of the learner, etc., it may be difficult to determine whether the voice analysis unit matches the context information prepared in advance in the context information storage unit. This step is a method to determine if there is an input of sound information (voice or sound) that does not match the context information, and it is determined whether or not the learner's voice corresponds to the prepared context information by dividing and reconstructing the learner's voice according to a preset rule. in order to confirm

A predetermined time interval may be calculated based on each time interval required to output a word, phrase, or sentence included in the pre-prepared context information, and may be used to divide the sound information. That is, in order to efficiently separate the sound input by the learner, it may be provided to divide and analyze the sound by using the time required to output the words, etc. included in the context information prepared in advance.

For example, when the voice information matching the contextual information explaining the 'purpose of blood glucose measurement' is the phrase 'for evaluation of the blood glucose-lowering effect of basal insulin', the learner can 'check how much blood glucose has dropped' If you enter the answer 'for', set the time required to pronounce each word of 'basal', 'insulin', 'blood sugar', 'lowering', 'effect', 'evaluation', and 'for' It can be used as a basis for dividing the phrase entered by the learner as a basis. For example, after specifying the time required to pronounce the word 'insulin' as about 1 to 1.5 seconds, the word, phrase or sentence input by the learner is divided into the period of time, and the sound pattern and similarity matching the word If there is a pronunciation that matches in the range of 90%, 95% or 100%, it may be provided to recognize it as 'insulin'. A keyword included in each recognized user's voice may be defined as each ontology component stored in the ontology-based database unit.

Meanwhile, the extracting of the one or more keyword information by comparing it with the sound information of the ontology-based word database unit may further include storing the extracted one or more keyword information in the ontology-based word database unit.

By repeating the steps of extracting keywords and storing them again in the ontology-based database unit, information on arbitrary related keywords that the learner can provide can be reinforced and the word database unit can be reinforced.

Next, after the phrase information generating unit of the HMD converts the extracted one or more keywords into word information, sentence information or syntax information, determining whether or not it matches the context information prepared in advance in the context information storage unit includes (S140).

The keywords collected in the previous step are classified or combined into classes, instances, attributes, etc., respectively, using the phrase information generator, and combined in the form of words, phrases, and sentences, and converted into word information, sentence information or phrase information to store context information It is provided to determine whether it matches the situation information prepared in advance in the department.

According to another aspect of the present invention, as a voice recognition-based virtual reality learning providing method for recognizing a user's voice and interacting with a character in virtual reality, the step of displaying situation information on the display unit of the HMD, there is input of voice information In this case, the voice analysis unit of the HMD determines whether or not the context information is prepared in advance in the context information storage unit of the HMD, and when there is input of voice information matching the context information, the display unit and the speaker unit of the HMD providing the character's interaction gesture and interaction sound through the step; when there is input of sound information that does not match the context information, the sound information analysis unit of the HMD divides the sound information at predetermined time intervals, and the division extracting one or more keyword information by comparing the obtained sound information with the sound information of the ontology-based word database unit, the phrase information generating unit of the HMD converting the extracted one or more keywords into word information, sentence information or syntax information Then, a storage medium storing a program for performing the step of determining whether or not the context information is matched with the context information prepared in advance is provided in the context information storage unit.

In the present specification and drawings, preferred embodiments of the present invention have been disclosed, and although specific terms are used, these are only used in a general sense to easily explain the technical content of the present invention and help the understanding of the present invention. It is not intended to limit the scope. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

Claims (10)

In the voice recognition-based virtual reality learning providing method for recognizing a user's voice and interacting with a character in virtual reality, the method comprising:
displaying context information on a display unit of the HMD;
When there is input of voice information, the voice analysis unit of the HMD determines whether or not it matches the context information prepared in advance in the context information storage unit of the HMD, and when there is input of voice information matching the context information, the display unit and providing an interaction gesture and an interaction sound of the character through a speaker unit of the HMD;
When there is input of sound information that does not match the context information, the sound information analysis unit of the HMD divides the sound information at predetermined time intervals, and compares the divided sound information with the sound information of the ontology-based word database unit. extracting one or more keyword information;
Containing; a step of converting the phrase information generating unit of the HMD into word information, sentence information, or phrase information using the extracted one or more keywords, and then determining whether it matches the context information prepared in advance in the context information storage unit; A method for providing virtual reality learning based on voice recognition, characterized in that.
According to claim 1,
The predetermined time interval is calculated based on each time interval required to output the words, phrases or sentences included in the prepared context information, and is used to divide the sound information. How to provide learning.
According to claim 1,
Displaying the context information;
Displaying a third character included in the context information; voice recognition-based virtual reality learning providing method, characterized in that it further comprises.
4. The method of claim 3,
Displaying the third character; outputting arbitrary sound information generated from an arbitrary space in which the third character is disposed through the speaker unit of the HMD; voice recognition-based virtual, characterized in that it further comprises How to provide real learning.
5. The method of claim 4,
The voice recognition-based virtual reality learning providing method, characterized in that the arbitrary sound information includes filtering sound information corresponding to a frequency of a predetermined region that can be recognized by the sound information analysis unit.
According to claim 1,
Comparing with the sound information of the ontology-based word database unit, extracting one or more keyword information;
Storing the extracted one or more keyword information in the ontology-based word database unit; voice recognition-based virtual reality learning providing method further comprising a.
According to claim 1,
The voice recognition-based virtual reality learning providing method, characterized in that the sound information analysis unit is provided to select at least one language in advance.
8. The method of claim 7,
Displaying the context information; Displaying a language selection user interface for selecting the language in advance; Voice recognition-based virtual reality learning providing method further comprising a.
According to claim 1,
The displaying of the context information further comprises: determining whether the user's gaze direction received from the gyro sensor provided in the HMD is directed toward an arbitrary virtual space region in which the character is located; A method of providing virtual reality learning based on voice recognition.
A storage medium storing a program for performing the method according to any one of claims 1 to 9.

Images