KR20210076431A - Apparatus and method for providing immersive learning content through voice recognition - Google Patents

Abstract

The present invention relates to a device and a method for providing an immersive learning content through voice recognition, in providing immersive learning content using virtual reality, complex reality, and augmented reality, etc. According to an embodiment of the present invention, the device for providing immersive learning content includes: a display module which outputs immersive learning content; a voice recognition module that receives a voice signal from a user; and a control module for identifying control data corresponding to a voice signal and controlling the immersive learning content according to the identified control data.

Description

Apparatus and method for providing realistic learning content through speech recognition {APPARATUS AND METHOD FOR PROVIDING IMMERSIVE LEARNING CONTENT THROUGH VOICE RECOGNITION}

The present invention relates to an apparatus and method for providing immersive learning content for controlling immersive learning content through voice recognition in providing immersive learning content using virtual reality, complex reality, augmented reality, and the like.

Due to the recent development of various digital devices, there is a trend to adopt an educational method incorporating digital devices in an educational environment.

Reflecting this trend, the government has been distributing digital textbooks for the upper grades of elementary school and the first grade of middle school since 2018 as part of a smart education policy, and is presenting an educational curriculum using digital textbooks.

In line with this, recently, educational methods using various digital devices are being developed.

However, the education method up to now is a method in which a digital device is simply grafted onto the existing one-way education method in which the teacher injects knowledge into the learner. there was.

An object of the present invention is to provide an apparatus and method for providing immersive learning content capable of controlling immersive learning content through voice recognition in providing immersive learning content using virtual reality, complex reality, augmented reality, etc. do it with

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the examples of the present invention. Moreover, it will be readily apparent that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

To achieve the above object, an apparatus for providing immersive learning content according to an embodiment of the present invention provides a display module for outputting immersive learning content, a voice recognition module for receiving a voice signal from a user, and a control corresponding to the voice signal and a control module for identifying data and controlling the immersive learning content according to the identified control data.

In an embodiment, the display module outputs the immersive learning content as a virtual image.

In an embodiment, the display module is characterized in that the immersive learning content and the live-action image are output together.

In one embodiment, the display module is characterized in that the output of the immersive learning content to the live-action image as an augmented image.

In one embodiment, it characterized in that it further comprises a camera module for shooting the live-action image.

In an embodiment, the control module extracts text data corresponding to the voice signal with reference to a memory, and identifies control data corresponding to the extracted text data.

In an embodiment, the display module is a wearable device.

In an embodiment, the display module includes at least one of a Head Mounted Device (HMD), a video see-through device, an optical see-through device, and a projector. characterized in that

In addition, a method for providing immersive learning content according to an embodiment of the present invention for achieving the above object includes receiving a voice signal from a user through a voice recognition module, and referring to a memory, corresponding to the received voice signal. Identifying control data and controlling the immersive learning content output by a display module according to the control data.

In one embodiment, the step of identifying the control data corresponding to the received voice signal with reference to the memory comprises: extracting text data corresponding to the received voice signal with reference to the memory; and identifying control data corresponding to the text data.

In the present invention, in providing immersive learning content using virtual reality, complex reality, augmented reality, etc., the learner controls the immersive learning content on his/her own through voice recognition, thereby inducing the learner's participation in education to improve learning efficiency. Not only can various contents be used for learning and education in an interactive way, but also the leadership of educators can be improved.

In addition to the above-described effects, the specific effects of the present invention will be described together while describing specific details for carrying out the invention below.

1 is a diagram schematically illustrating components of an apparatus for providing immersive learning content according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an embodiment of the apparatus for providing immersive learning content shown in FIG. 1;
3 is a flowchart illustrating a method for providing immersive learning content according to an embodiment of the present invention.

The above-described objects, features and advantages will be described below in detail with reference to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains will be able to easily implement the technical idea of the present invention. In describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in specific cases, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

When a part "includes" a certain element throughout the specification, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention illustrated below may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to more completely explain the present invention to those of ordinary skill in the art.

The present invention relates to an apparatus and method for providing immersive learning content for controlling immersive learning content through voice recognition in providing immersive learning content using virtual reality, complex reality, augmented reality, and the like.

Hereinafter, an apparatus for providing immersive learning content according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 and 2 .

1 is a diagram schematically illustrating components of an apparatus for providing immersive learning content according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating an embodiment of the apparatus for providing immersive learning content shown in FIG. 1 to be.

Referring to FIG. 1 , an apparatus 100 for providing immersive learning content according to an embodiment of the present invention includes a display module 110 , a voice recognition module 120 , a control module 130 , and a camera module 140 . can do. The immersive learning content providing apparatus 100 illustrated in FIG. 1 is according to an embodiment, and its components are not limited to the embodiment illustrated in FIG. 1 , and some components may be added, changed, or may be deleted.

Each component constituting the immersive learning content providing apparatus 100 is ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gates) arrays), processors, controllers, micro controllers, and microprocessors.

The display module 110 may be any module capable of displaying an image to a user. For example, the display module 110 may be implemented as a mobile-based device or a wearable device according to a display method (display method).

When the display module 110 is implemented as a mobile-based device, the display module 110 is, for example, an Ultra Mobile PC (UMPC), a workstation, a net-book, a Personal Digital Assistants (PDA), or a portable device. It may be a computer, a web tablet, a wireless phone, a mobile phone, a smart phone, an e-book, a portable multimedia player (PMP), or the like.

When the display module 110 is implemented as a wearable device, the display module 110 includes a head mounted device (HMD), a video see-through device, an optical see-through device, It may be either a screen-based device or a projector-based device.

Referring to FIG. 2 , the immersive learning content providing apparatus 100 may be implemented as a wearable device designed to be worn on the head. In this case, the display module 110 may be located in front of the user's eyes when the user wears the immersive learning content providing apparatus 100 , and may output an image in the direction of the user's eyes.

The immersive learning content providing apparatus 100 and the display module 110 constituting the same may be implemented in various shapes other than the embodiment shown in FIG. 2 , but below, for convenience of description, the immersive learning content providing apparatus 100 ) will be described on the assumption that it is implemented as shown in FIG. 2 .

The display module 110 may output immersive learning content under the control of the control module 130 to be described later.

The immersive learning content is content for the user's learning according to the learning curriculum, and provides a sense of immersion to the user through virtual reality (VR), mixed reality (MR), augmented reality (AR), etc. The give can be any content.

Here, augmented reality (AR) refers to a computer graphics technique that synthesizes a virtual object or information in a real environment to make it look like an object existing in the original environment. Unlike conventional virtual reality, which targets only virtual space and objects, augmented reality has a feature of providing additional information that is difficult to obtain only in the real world by synthesizing virtual objects on the basis of the real world.

Virtual reality (VR) refers to an interface between a human and a computer that makes a specific environment or situation into a computer and makes it as if the person using it is interacting with the actual surrounding situation/environment.

On the other hand, mixed reality (MR) refers to a technology that creates a space that fuses the two worlds by combining information from the real world and the virtual world. Complex reality (MR) refers to a technology implemented by bringing only the advantages of augmented reality (AR) and virtual reality (VR).

In an example, the display module 110 may output realistic learning content as a virtual image based on virtual reality (VR) technology.

More specifically, the control module 130 may generate immersive learning content according to the learning curriculum as a virtual image output through the display module 110 as a whole, and the display module 110 is controlled by the control module 130 . The generated virtual image can be output.

Accordingly, a user wearing the immersive learning content providing apparatus 100 as shown in FIG. 2 cannot secure a view of the real environment in front, and can only check a virtual image corresponding to the immersive learning content. .

In another example, the display module 110 may output immersive learning content and a live-action image together based on augmented reality (AR) technology.

More specifically, the control module 130 may generate immersive learning content according to the learning curriculum as a virtual image output through a portion of the display module 110 , and the display module 110 sends the You can output the virtual image created by

In this case, the display module 110 may be made of a transparent material to project an image in front of the user. In other words, the user may secure a view of the real environment in front through the transparent display module 110 .

Accordingly, a user who wears the immersive learning content providing apparatus 100 as shown in FIG. 2 secures a view of the real environment in front, and at the same time, receives immersive learning content through a portion of the display module 110 . You can check the virtual image corresponding to .

Also, the display module 110 may output immersive learning content as an augmented image on the live-action image.

More specifically, the control module 130 may generate a live-action image of a real environment around the user and an augmented image corresponding to the immersive learning content according to the learning curriculum. The display module 110 may output the augmented image by overlapping the actual image generated by the control module 130 .

In order to generate a live-action image, the immersive learning content providing apparatus 100 may further include a camera module 140 for capturing a live-action image. The camera module 140 may be provided at an arbitrary location of the immersive learning content providing apparatus 100 to photograph a real environment around the user, and the control module 130 may generate a live-action image corresponding to the captured real environment. can

Referring back to FIG. 2 , the camera module 140 may be provided in front of the immersive learning content providing apparatus 100 to provide a sense of presence according to the user's viewing direction, and may photograph an actual environment in front of the user.

Accordingly, the control module 130 may generate an actual image for the user's viewing direction, and the display module 110 may overlap the augmented image on the actual image and output it.

The voice recognition module 120 may receive a voice signal from a user.

More specifically, the voice recognition module 120 may receive an analog voice signal, which is the user's voice, and then frequency-analyze it and convert it into a digital signal. To this end, the voice recognition module 120 may include a plurality of analog-to-digital converters.

The control module 130 may identify control data corresponding to the voice signal.

More specifically, when the voice signal is converted into a digital signal by the voice recognition module 120 , the control module 130 may identify control data corresponding to the digital signal with reference to the memory. To this end, control data corresponding to each voice signal may be stored in the form of a look-up table (LUT).

In addition, when the voice signal is converted into a digital signal by the voice recognition module 120, the control module 130 extracts text data corresponding to the digital signal with reference to the memory, and controls data corresponding to the extracted text data. can be identified. To this end, text data corresponding to each voice signal and control data corresponding to the text data may be stored in the memory in the form of a lookup table.

For example, a lookup table as shown in Table 1 below may be previously stored in the memory.

[voice signal] [character data] [control data] One a "next" A 2 b "enlargement" B 3 c "reduction" C 4 d "rotation" D

When an analog voice signal a is received by the voice recognition module 120 , the control module 130 may identify text data corresponding to the voice signal as “next”. Subsequently, the control module 130 may identify the control data corresponding to the text data of “next” as A.

Similarly, when the analog voice signals b, c, and d are received in the voice recognition module 120, the control module 130 converts the problem data corresponding to each voice signal to “enlarge”, “reduce”, and “rotate”, respectively. can be identified. Then, the control module 130 may identify the control data corresponding to the text data of “enlargement”, “reduction” and “rotation” as B, C, and D, respectively.

The control module 130 may control the immersive learning content according to the identified control data. More specifically, the control module 130 may control the immersive learning content currently being output from the display module 110 according to the control data.

Referring to [Table 1] described above, when the control data is identified as A, the control module 130 may change the immersive learning content according to the learning curriculum. In other words, when the first sensory learning content and the second sensory learning content are sequentially output according to the learning curriculum, the control module 130 controls the first sensory learning content currently being output when the control data is identified as A may be changed to the second immersive learning content. Accordingly, the user who spit out the word “daum” can enjoy the changed immersive learning content.

Also, when the control data is identified as B or C, the control module 130 may enlarge or reduce the immersive learning content. In other words, the control module 130 may enlarge or reduce some or all of the immersive learning content currently being output through the display module 110 . Accordingly, a user who spit out the words “enlarged” or “reduced” can enjoy the enlarged or reduced immersive learning content.

Also, when the control data is identified as D, the control module 130 may rotate the immersive learning content. For example, the realistic learning content may be an augmented image having a three-dimensional structure. At this time, the control module 130 may rotate the immersive learning content currently being output through the display module 110, and the user who spit out the word “rotation” uses the other side of the augmented image having a three-dimensional structure. can enjoy

Hereinafter, a method for providing immersive learning content according to an embodiment of the present invention will be described with reference to FIG. 3 . The method for providing immersive learning content shown in FIG. 3 may be performed by the control module 130 shown in FIG. 1 .

3 is a flowchart illustrating a method for providing immersive learning content according to an embodiment of the present invention.

Referring to FIG. 3 , the method for providing immersive learning content includes receiving a voice signal from a user (S10), identifying control data corresponding to the voice signal (S20), and controlling the immersive learning content according to the control data. It may include a step (S30) of doing. The method for providing immersive learning content shown in FIG. 3 is according to an embodiment, and each step constituting the method is not limited to the embodiment shown in FIG. 3, and some steps may be added, changed, or deleted as necessary. can

Each step will be described in detail. First, the control module 130 may receive a voice signal from the user through the voice recognition module 120 ( S10 ). Since the content of the voice recognition module 120 processing the voice signal has been described above, a detailed description thereof will be omitted here.

Then, the control module 130 may identify the control data corresponding to the voice signal with reference to the memory (S20). To this end, the voice signal and the control data may be stored in advance to correspond to each other in the memory, as described above.

Additionally, the control module 130 may extract text data corresponding to the voice signal with reference to the memory and may identify control data corresponding to the extracted text data. To this end, voice signals, text data, and control data may be stored in advance to correspond to each other in the memory, as described in [Table 1] above.

Subsequently, the control module 130 may control the immersive learning content output by the display module 110 according to the control data ( S30 ). Since the contents of the display module 110 outputting the immersive learning content under the control of the control module 130 and the contents of controlling the immersive learning content according to various examples have been described above, a further detailed description will be omitted here. do.

As described above, in the present invention, in providing realistic learning content using virtual reality, complex reality, augmented reality, etc., the learner controls the immersive learning content by voice recognition to induce the learner's participation in education. Not only can the learning efficiency be improved, but various contents can be used for learning and education in an interactive way, thereby improving the leadership of the educator.

As described above, the present invention has been described with reference to the illustrated drawings, but the present invention is not limited by the embodiments and drawings disclosed in the present specification. It is obvious that variations can be made. In addition, although the effects according to the configuration of the present invention are not explicitly described and described while describing the embodiments of the present invention, it is natural that the effects predictable by the configuration should also be recognized.

Claims (10)

a display module for outputting immersive learning content;
a voice recognition module for receiving a voice signal from a user; and
and a control module that identifies control data corresponding to the voice signal and controls the immersive learning content according to the identified control data.
A device for providing realistic learning content.
According to claim 1,
The display module is
Outputting the immersive learning content as a virtual image
A device for providing realistic learning content.
According to claim 1,
The display module is
Outputting the immersive learning content and the live-action video together
A device for providing realistic learning content.
4. The method of claim 3,
The display module is
Outputting the immersive learning content as an augmented image to the live-action image
A device for providing realistic learning content.
4. The method of claim 3,
Further comprising a camera module for shooting the live-action image
A device for providing realistic learning content.
According to claim 1,
The control module is
extracting text data corresponding to the voice signal with reference to a memory, and identifying control data corresponding to the extracted text data
A device for providing realistic learning content.
According to claim 1,
The display module is
A wearable device
A device for providing realistic learning content.
8. The method of claim 7,
The display module is
HMD (Head Mounted Device), video see-through device (video see-through device), optical see-through device (optical see-through device), including at least one of the projector (projector)
A device for providing realistic learning content.
receiving a voice signal from a user through a voice recognition module;
identifying control data corresponding to the received voice signal with reference to a memory; and
Controlling the immersive learning content output by the display module according to the control data
How to provide realistic learning content.
10. The method of claim 9,
The step of identifying control data corresponding to the received voice signal with reference to the memory comprises:
extracting text data corresponding to the received voice signal with reference to the memory;
identifying control data corresponding to the extracted text data
How to provide realistic learning content.

Images