KR102320229B1 - Smart tables for MR learning content and how to provide them - Google Patents

Abstract

The present disclosure implements a simulated real world comprising virtual objects (simulated objects and simulated environments) using a mixed reality technology for language, numeracy, foreign language, geography, and cultural education of children, collects, analyzes, and digitizes the voice data of a user using an AI voice recognition technology, and can provide educational services which allow students to study according to current levels of the students. In addition, the present disclosure provides information on a plurality of markers sequentially or simultaneously without a plurality of paper card type markers by electronically presenting markers containing information on a virtual object through a display device provided on a table in order for a user to use the MR content so that virtual object is implemented, thereby providing an educational service through immersive content.

Description

A smart table device for educating children capable of displaying a marker containing virtual object information recognized as an HMD device on a display device on a table, and an education method using the same. {Smart tables for MR learning content and how to provide them}

Disclosed are a method for providing MR learning content, a device for MR learning content, and an operating system for operating the MR learning content device.

Mixed reality (MR) is a technology that creates a new environment in which real and virtual objects coexist by combining reality and virtual objects, and enables users to experience various digital information more realistically by interacting with the environment in real time. It is an attempt to graft the virtual world based on the real world because it is realistically difficult to construct a complete virtual world.

Republic of Korea Patent No. 10-1964192 (registered on March 26, 2019. Smart table device for simulation) Republic of Korea Patent Publication No. 10-2019-00002815 (published on January 9, 2019. Smart touch table)

delete

Among the existing methods of executing MR contents using an HMD, the marker method is performed by recognizing a marker in the form of a paper card containing information about a virtual object by a camera provided in the HMD device and executing the virtual object corresponding to the marker through the HMD. Because of this method, a corresponding paper-type marker was needed for a virtual object. Therefore, in order to execute a large number of virtual objects, a large number of markers are required, which limits the storage, storage, and use of these markers. In addition, in order to give motion to a virtual object, a complex production process through a program for 3D image production such as Unity was required.

However, according to an embodiment of the present disclosure, an infinite number of markers can be stored and used within the server storage space by presenting the marker in the form of an electronic image stored in the server instead of the marker in the form of a paper card through a display device provided on the table.

In addition, an embodiment according to the present disclosure uses a relatively simple program such as PowerPoint without a complicated production process through a program for 3D image production such as Unity to move a marker image on the display to create the movement of the virtual object through the virtual object. It is possible to reduce the cost of content creation, so that it is possible to provide an efficient educational service method and device.

According to an embodiment, a method of providing an MR education service includes: presenting, by a teacher, a marker of an educational target object or environment to be transmitted to a learner; Recognizing the presented educational target object or the marker of the environment in the device; displaying an object corresponding to the recognized marker; Conducting education based on the displayed object, analyzing the pronunciation of the learner during the education process, measuring accuracy, and determining whether to repeat learning and progress to the next education stage.

In the method of providing an MR education service according to an embodiment, the marker includes a real object recognized by the device, and the step of presenting the marker may be presented in the form of executing a marker displayed on an electronic display.

In the method of providing an MR education service according to an embodiment, the recognizing step is recognized through a camera attached to a smartphone or tablet PC, and the displaying may be performed through a display screen of a smart phone or tablet PC. .

In the method of providing an MR language education service according to an embodiment, the step of conducting the education is to quantify the accuracy of pronunciation by analyzing the pronunciation of the learner through the AI speech recognition engine, and based on this, the next step In the case of a low-score education, the existing learning can be repeated, and the progress of the education stage can be controlled through the controller.

A device for providing an MR education service according to an embodiment includes a display device; a table supporting the display device; a main server including an AI voice recognition device and an MR content storage device provided in the table; and a process for operating the device.

The present disclosure is for the effective education of children, and presents objects and environments with high realism and immersion through MR contents to children who have difficulty experiencing various objects or environments due to safety or time and cost issues, and AI language recognition technology It has the effect of increasing the effectiveness of education by enabling step-by-step and repeated education.

1 is a conceptual diagram illustrating a method of providing an MR education service in a smart table device according to an embodiment.
2 is a diagram for explaining a method of providing an MR education service through a mock object in a smart table device according to an embodiment.
3 is a diagram for explaining a method of providing an MR education service through a simulated environment in a smart table device according to an embodiment.
4 is a content execution flowchart of experiential learning content according to an embodiment.

Terms used in this specification will be briefly described, and the present invention will be described in detail. The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term. Terms including an ordinal number such as 1st, 2nd, etc. may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related items or any one of a plurality of related items. When a part "includes" a certain element throughout the specification, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. Also, as used herein, the term “unit” refers to a hardware component such as software, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), and the “unit” performs certain roles. However, "part" is not meant to be limited to software or hardware. A “unit” may be configured to reside on an addressable storage medium and may be configured to refresh one or more processors. Thus, by way of example, “part” includes components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables.

The functionality provided within components and “parts” may be combined into a smaller number of components and “parts” or further divided into additional components and “parts”.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a diagram for schematically explaining an embodiment of an operating method of a smart table device 100 according to an embodiment.

The smart table device 100 according to an embodiment includes a display 110 capable of showing a marker 160, a table frame 120 to support the display, an AI voice recognition device 131 provided in the table, and The main server 130 including the MR content storage device 132, the process 133 for operating it, and the HMD device 140 for the execution of the MR content, the controller 150 for the training progress, the marker 160, the virtual An object 170 is provided.

Referring to FIG. 1 , the user can view the MR content image executed while wearing the HMD device 140 , and at the same time see the actual space around the user's viewing angle.

According to an embodiment, when a user wearing the HMD device 140 gazes at the marker 160 displayed on the display 110 capable of showing the marker 160 , the virtual object 170 corresponding to the marker 160 is displayed. It overlaps with the actual surrounding space and can be seen by the user's eyes through the HMD device 140 .

2 and 3 are diagrams for explaining an example of a state in which learning is performed through the smart table device 100 .

FIG. 2 shows that the user wears the HMD device 140 and gazes at the marker so that the virtual object is overlapped with the real environment, and learning can be performed while being visible to the user's eyes. For example, in FIG. 2, the user gazes at a marker 160 containing information about the virtual deer object 170, and the virtual deer object appears in a three-dimensional form on the desk. Education such as learning Hangul or alphabet spelling may be conducted.

In FIG. 3 , according to an embodiment, a user wears the HMD device 140 and gazes at a marker so that a virtual environment is visible to the user's eyes and learning about geography, culture, environment, etc. may be performed. For example, in FIG. 3 , the user gazes at the marker 160 containing information about the virtual environment called 'school', and the virtual school environment appears in a three-dimensional form. ', etc., can conduct language education about the environment.

4 is a flowchart of a method of experiencing learning content and content through the smart table device 100 .

According to an embodiment of FIG. 4 , the educational content of the present disclosure allows a user wearing the HMD device 140 to sit in front of the smart table device 100 and select educational content such as Hangul, English, mathematics, geography, culture, and history. content can be experienced.

For example, if history learning content is selected, education can be conducted to show people at that time, cultural times, etc. as MR content, and to review key information and content to be learned through simple quizzes. .

It can be understood that the above-described embodiments are illustrative and not restrictive. In addition, it is not limited to the order of the operations shown in the flowchart, and it may be understood that, according to various embodiments, some steps may be omitted or added, and the order of some steps may be changed.

Meanwhile, the above-described embodiment can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. In addition, the structure of the data used in the above-described embodiment may be recorded in a computer-readable medium through various means. In addition, the above-described embodiment may be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. For example, methods implemented as software modules or algorithms may be stored in a computer-readable recording medium as computer-readable codes or program instructions.

Computer-readable media may be any recording media that can be accessed by a computer, and may include volatile and nonvolatile media, removable and non-removable media. The computer readable medium may include a magnetic storage medium, for example, a ROM, a floppy disk, a hard disk, etc. does not In addition, computer-readable media may include computer storage media and communication media.

In addition, a plurality of computer-readable recording media may be distributed in network-connected computer systems, and data stored in the distributed recording media, for example, program instructions and codes, may be executed by at least one computer. have.

The specific implementations described in the present disclosure are merely exemplary, and do not limit the scope of the present disclosure in any way. For brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted.

The description of the present disclosure described above is for illustration, and those of ordinary skill in the art to which the present disclosure pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present disclosure. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.

The use of all examples or exemplary terms, “for example,” “etc.,” in the present disclosure is merely for the purpose of describing the present disclosure in detail, and unless limited by the appended claims, the use of the examples or exemplary terms causes the present disclosure. is not limited in scope.

In addition, unless specifically stated as "essential", "importantly", etc., the components described in the present disclosure may not necessarily be components for the implementation of the present disclosure.

Those of ordinary skill in the art related to the embodiments of the present disclosure will understand that it may be implemented in a modified form without departing from the essential characteristics of the description.

The present disclosure can apply various transformations and can have various embodiments, and the present disclosure is not limited by the specific embodiments described in the specification, and all transformations and equivalents included in the spirit and scope of the present disclosure It should be understood that alternatives are included in the present disclosure. Therefore, the disclosed embodiments are to be understood in an illustrative rather than a restrictive sense.

The scope of the present disclosure is indicated by the claims rather than the detailed description of the invention, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present disclosure.

As used herein, terms such as “…unit” and “module” mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

“Part” and “module” are stored in an addressable storage medium and may be implemented by a program that can be executed by a processor.

For example, “part” and “module” refer to components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and programs. It may be implemented by procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays and variables.

In this specification, the description "A may include one of a1, a2, and a3" has a broad meaning that exemplary elements that may be included in the element A are a1, a2, or a3.

The above description is not necessarily limited to a1, a2, or a3 elements constituting the element A.

Therefore, it should be noted that the elements constituting A are not exclusively interpreted as meaning excluding other elements not exemplified other than a1, a2, and a3.

In addition, the above description means that A may include a1, a2, or a3. The above description does not necessarily mean that the elements constituting A are selectively determined within a predetermined set. For example, it should be noted that the above description is not necessarily to be construed as restrictive as that a1, a2, or a3 selected from the set including a1, a2 and a3 constitutes component A.

Also in the present specification, the description "at least one of a1, a2 and a3" means "a1", "a2", "a3", "a1 and a2", "a1 and a3", "a2 and a3", and represents one of "a1, a2 and a3".

Smart Table Device 100 Display 110
Table Frame (120) Main Server (130)
AI voice recognition device (131) MR content storage device (132)
operating processor (133) HMD device (140)
Controller(150) Marker(160)
virtual object 170

Claims (5)

delete table frame; display; AI speech recognition device; MR content storage; main server; HMD; marker image; and a processor for implementing MR content through the HMD device and executing voice recognition training through the AI voice recognition device,
The main server including the MR content storage device is coupled to the inside of the table frame, and the display connected to the main server and displaying the marker image is mounted on the table frame, and the processor operates through the display. Recognizing the executed marker image through the HMD device, creating a virtual object through the HMD device and making it visible to the user, asking the user about the learning content through the AI voice recognition device, and asking the user about the question A smart table device that provides educational MR content that analyzes and quantifies the voice data of the user to proceed to the next learning stage if the user's answer to a question is above a certain score or repeats the existing learning if it is below a certain score.
3. The method of claim 2,
A smart table device including a controller that can adjust MR contents for learning progress.
3. The method of claim 2,
The marker includes a real object included in the scene recognized by the HMD,
The property of the marker includes at least one of a type of the real object, a time and a location at which a scene including the real object is recognized by the HMD,
The processor, as the marker is identified, extracts an MR object corresponding to the identified marker from a previously stored mapping database, a smart table device for providing MR education content.
3. The method of claim 2,
A smart table device capable of controlling the movement of the MR content for learning through the movement of a marker image executed on the display.

Images