WO2019117495A1

WO2019117495A1 - Method for providing augmented reality-applied education service

Info

Publication number: WO2019117495A1
Application number: PCT/KR2018/014355
Authority: WO
Inventors: 안형준
Original assignee: 코디소프트 주식회사
Priority date: 2017-12-15
Filing date: 2018-11-21
Publication date: 2019-06-20
Also published as: KR101909461B1

Abstract

The present invention relates to a method for providing an augmented reality-applied education service. According to one embodiment of the present invention, provided is a method for providing an augmented reality-applied education service, the method, in order to provide an augmented reality-applied education service, comprising the steps of: (a) an education content providing device checking the type of an education content reproduction device; and (b) providing, to the education content reproduction device, video data comprising an augmented reality tag, augmented reality script data executed according to the augmented reality tag, and a tag recognition program for recognizing the augmented reality tag.

Description

How to Provide Educational Services with Augmented Reality

The present invention relates to a method of providing an educational service to which an augmented reality is applied.

Unlike the offline education system where educators and students usually teach directly at schools or academies by education or lecture system, the e-learning system is a system in which the lecture video of the lecture progressed by the lecturer is transmitted from the streaming server to the network Through a learner's terminal through a streaming service.

However, since learning in the e-learning system is a training method in which a lecture image of a lecturer is reproduced unilaterally on a learner's terminal through a streaming service, interaction between a learner and an instructor or a learner is not easily performed A problem arises.

In addition, since the e-learning system is taught by unidirectional lectures, it is easy to get bored and the concentration of the e-learning system is lowered.

Therefore, it is urgently needed to improve learner 's concentration ability in e - learning system.

The present invention provides a method of providing an educational service using an augmented reality capable of preventing a decrease in learning concentration and providing various information to a learner.

According to an embodiment of the present invention, there is provided a method for providing an educational service to which an augmented reality is applied, the method comprising the steps of: (a) And a tag recognition program for recognizing an augmented reality tag, the augmented reality script data being executed according to the augmented reality tag, and the tag for identifying the augmented reality tag to the educational content reproduction apparatus.

Also, in the step (a), the educational contents providing apparatus may check the type of the educational contents reproducing apparatus to provide moving picture data having a resolution suitable for the educational contents reproducing apparatus.

In the step (b), the educational content providing apparatus may provide data set according to a content providing mode to the educational content reproducing apparatus, wherein the content providing mode may include an editing video providing mode and a live video providing mode have.

In the edited moving picture providing mode, the educational contents providing apparatus provides the moving picture data, the augmented reality script data, and the tag recognizing program stored in the storage unit to the educational contents reproducing apparatus, Wherein the augmented reality tag includes at least one augmented reality tag arranged in a placement area, and the augmented reality script data may be set to output preset data.

In the live moving picture providing mode, the educational contents providing apparatus can provide the moving picture photographed in real time and the augmented reality script data and the tag recognizing program preset by the contents provider to the educational contents reproducing apparatus.

In addition, the augmented reality script data may be set to output additional information related to the education contents as a list.

The education contents providing apparatus may further include a step of accepting and processing the questions of the learner from the educational contents reproducing apparatus.

In addition, the training content providing apparatus may transmit the inquiry of the learner to the education service provider, receive the answer to the inquiry from the education service provider, and reflect the response to the augmented reality script data.

Also, the augmented reality tag may be represented by a symbol including at least one of a letter and a number.

In addition, the tag recognition program may include template data for recognizing the augmented reality tag in a moving image transmitted in real time.

In addition, the tag recognition program can recognize the augmented reality tag by performing template matching using the template data.

Also, the tag recognition program may compare the ratio of the remaining area with the area occupied by the augmented reality tag based on a preset tag recognition area, and compare the comparison result with the template data to detect the code of the augmented reality tag have.

The template data may include an area occupied by the augmented reality tag and a code of the augmented reality tag according to a ratio of each of the remaining areas.

The template data may include a code of the augmented reality tag set for each resolution of the moving image.

The present invention can provide a variety of information and interest to a learner by providing educational contents to which augmented reality is applied by using moving picture data including an AR tag and script data executed according to an AR tag. This can improve the learner's attention or concentration.

Also, the present invention can provide the learner with the educational contents to which the augmented reality is applied, so that the education contents and the supplementary information can all be displayed in one screen.

Further, according to the present invention, when the educational contents providing apparatus provides the moving picture streaming service, the AR contents data is provided to the educational contents reproducing apparatus separately from the moving picture data, thereby providing the AR that is operated without deterioration in picture quality to the learner.

In addition, the present invention can selectively output AR data during playback of moving images. Thus, the present invention can improve the convenience in which the learner can select the drive of the AR.

1 is a block diagram illustrating a configuration of a content providing system to which an augmented reality is applied according to an embodiment of the present invention.

2 is a diagram illustrating a configuration of an educational contents providing apparatus according to an embodiment of the present invention.

3 is a diagram illustrating moving image data including an augmented reality tag.

4 is a diagram showing a moving image in which augmented reality script data is executed.

5 is a diagram for explaining template matching of a tag recognition program.

6 is a diagram illustrating a configuration of an educational contents reproducing apparatus according to an embodiment of the present invention.

FIG. 7 is a flowchart illustrating a method of providing an education service using an augmented reality according to an embodiment of the present invention. Referring to FIG.

FIG. 8 is a diagram illustrating training contents to which an augmented reality is applied according to an embodiment of the present invention.

Hereinafter, the present invention will be described more specifically based on preferred embodiments of the present invention. However, the following embodiments are merely examples for helping understanding of the present invention, and thus the scope of the present invention is not limited or limited.

1 is a diagram illustrating a configuration of a content providing system to which an augmented reality is applied according to an embodiment of the present invention.

Referring to FIG. 1, a content providing system using an augmented reality according to an embodiment of the present invention may include an educational content providing apparatus 100, a communication network 50, and an educational content reproducing apparatus 200.

The educational contents providing apparatus 100 can perform various functions for providing contents by communicating with the educational contents reproducing apparatus 200 through the communication network 50 or providing contents such as member management and approval management. In particular, in order to provide the learner with the video streaming service applied with the augmented reality, the educational contents providing apparatus 100 may be provided with video data including the AR tag, AR script data executed according to the AR tag, To the educational contents reproducing apparatus 200. [

In one embodiment, the educational contents providing apparatus 100 may be configured as a server for providing a moving image streaming service. 2, the training content providing apparatus 100 may include a server communication unit 110, a server control unit 120, and a server storage unit 130. [

The server communication unit 110 may communicate with the educational contents reproducing apparatus 200 through the communication network 50 in at least one of wired and wireless manner to transmit and receive data for providing a moving image streaming service. For example, the server communication unit 110 can transmit the moving picture data, the AR script data, and the tag recognition program described above to the education contents reproduction apparatus 200. [

Here, the server communication unit 110 is described as a module included in the educational contents providing apparatus 100, but is not limited thereto, and may be configured as a separate device for communication.

The server control unit 120 controls the overall function of the educational contents providing apparatus 100 and controls the server communication unit 110 to receive script setting information from the educational contents reproducing apparatus 200, It is possible to control the server communication unit 110 to transmit moving image data including the AR tag to the server 200, AR script data to be executed according to the AR tag, and a tag recognition program for recognizing the AR tag.

Specifically, the server control unit 120 may control the server communication unit 110 to transmit moving image data including the AR tag 300 in real time as shown in FIG. 3 to provide a moving image streaming service. Also, the server control unit 120 may control the server communication unit 110 to transmit the AR script data and the tag recognition program together with the moving image data.

Here, the AR tag 300 may include a code represented by a symbol including at least one of a letter and a number. For example, the AR tag 300 may include a code represented by at least one of A00 through Z99.

Also, the moving picture data can be transmitted to the educational contents reproducing apparatus 200 without the AR script data being encoded.

Further, the AR script data can be set to provide additional information in the moving picture data as shown in FIG. For example, the AR script data may be linked to the moving picture data to show the AR data provided by the content provider or may be set to execute the AR program.

Here, the AR script data may include video data, audio data, or a program that is matched to the AR tag 300 and the output is executed. That is, the AR script data may include data or a program that is matched and executed to the AR tag 300 extracted from the educational content reproducing apparatus 200.

In addition, the AR script data can be set to specify the location and effects of video data, audio data or programs. For example, the AR script data may be set to show a picture in the upper left portion of the moving picture to be reproduced, and to show an effect of descending from the top to the bottom.

In the case where the "A" tag is extracted from the educational content reproduction apparatus 200, the AR script data is displayed by overlaying a "B" photograph matched with the "A" tag on the moving image and extracting the "C" Quot; D "program matched with the" C "tag.

In addition, the tag recognition program may include template data for recognizing the AR tag 300 in the moving image transmitted in real time. At this time, the tag recognition program can recognize the AR tag 300 by performing template matching using the template data.

Specifically, referring to FIG. 5, the tag recognition program determines the ratio of the first region 320 occupied by the symbol of the AR tag to the second region 330, which is the remaining region, based on the tag- The comparison result is matched with the template data to recognize the AR tag.

Here, the template data may include a code set according to the ratio of the first area 320 occupied by the symbol of the AR tag and the second area 330, which is the remaining area. For example, the template data may store the symbol of the AR tag with the first region 320 at about 40% and the second region 330 at about 60% as the code for A01, 41% and the second area 330 is about 59%, as codes for A02.

That is, the template data may include a plurality of codes that match the AR tags according to the ratio of the first area 320 and the second area 330.

In addition, the template data may include a code of the AR tag set for each resolution of the moving picture. For example, since the size of the tag recognition area 310 or the symbol is different according to the moving image resolution of each of the 1080P, 720P, and 360P template data, the template data is divided into the first area 320 and the second area 320 And a code of the AR tag set according to the ratio of the area 330.

On the other hand, the server control unit 120 can determine which kind of the educational content reproducing apparatus 200 connected through the server communication unit 110 is, and control the content to be provided to the corresponding type of apparatus. For example, the server control unit 120 may determine whether the educational contents reproducing apparatus 200 is a mobile apparatus or a PC, and provide the contents to the educational contents reproducing apparatus 200 in a form suitable for the mobile apparatus or the PC.

The server storage unit 130 may store moving picture data, AR script data, and a tag recognition program. Here, the server storage unit 130 stores moving picture data, AR script data, and a tag recognition program set by the server control unit 120, and stores the stored moving picture data, AR script data, and tag recognition And can provide the program to the server control unit 120.

The educational contents reproducing apparatus 200 may be constituted by a mobile device or a PC capable of reproducing contents received from the educational contents providing apparatus 100. [

The educational contents reproducing apparatus 200 may include a terminal communication unit 210, a terminal output unit 220, a terminal input unit 230, a terminal control unit 240, and a terminal storage unit 250.

It should be noted that the educational contents reproducing apparatus 200 is configured to include only the terminal communication unit 210, the terminal output unit 220, the terminal input unit 230, the terminal control unit 240, and the terminal storage unit 250 , Which may be variously modified and modified for components included in the educational contents reproducing apparatus 200 without departing from the essential characteristics according to the embodiment of the present invention.

The terminal communication unit 210 may communicate with the educational contents providing apparatus 100 through the communication network 50 in at least one of a wired and wireless manner to transmit and receive data for receiving a video streaming service. For example, the terminal communication unit 210 can receive moving picture data, AR script data, and a tag recognition program from the educational contents providing apparatus 100.

The terminal output unit 220 may output the data received from the educational contents providing apparatus 100 in order to receive the moving picture streaming service. To this end, the terminal output unit 220 may include a display for outputting video data and a speaker for outputting audio data. For example, the terminal output unit 220 may output a moving picture through a display and output a sound through a speaker.

The terminal input unit 230 may receive a learner's input related to moving picture playback and a learner's input for executing an AR script. To this end, the terminal input unit 230 may include a keypad having a plurality of keys, a touch pad coupled to a display, and a microphone for converting a learner's voice input into an electrical signal.

The terminal control unit 240 controls the overall function of the educational contents reproducing apparatus 200 and controls the terminal communication unit 210 to transmit the moving picture data including the AR tag from the educational contents providing apparatus 100, The AR communication module 210 can control the AR communication module 210 to receive the AR script data and the tag recognition program for recognizing the AR tag.

In detail, the terminal control unit 240 may control the terminal communication unit 210 to receive the moving image data including the AR tag in real time in order to realize the moving image streaming service. Also, the terminal control unit 240 may control the terminal control unit 240 to receive the AR script data and the tag recognition program together with the moving image data.

Here, the terminal control unit 240 may extract the AR tag from the moving image data and execute the AR script data matched with the extracted tag.

For example, when the "A" tag is extracted from the video data received from the training content providing apparatus 100, the terminal control unit 240 overlays the "B" Quot; D "program matched with the" C "tag may be displayed on the screen when the" C "

The terminal control unit 240 can recognize the AR tag in the moving picture transmitted in real time by executing the tag recognition program received from the educational contents providing apparatus 100. [ At this time, the terminal control unit 240 can perform template matching using the template data.

5, the terminal control unit 240 executes a tag recognition program to generate a first area 320 occupied by the symbol of the AR tag based on the preset tag recognition area 310, And the comparison result is matched with the template data to recognize the AR tag.

Here, the terminal control unit 240 may perform template matching using the template data set for the resolution of the moving picture even if the video resolution is changed by the learner.

The terminal storage unit 250 may store moving picture data, AR script data, and a tag recognition program. The terminal storage unit 250 stores the moving picture data, the AR script data, and the tag recognition program set by the terminal control unit 240, and stores the stored moving picture data, AR script data, and tag recognition And provide the program to the terminal control unit 240.

The contents providing system to which the augmented reality is applied according to an embodiment of the present invention can provide the AR driven contents to the learner by recognizing the AR tag in the moving picture provided in real time and executing the AR script.

Hereinafter, a method of providing a training service to which an augmented reality is applied according to an embodiment of the present invention will be described with reference to FIG.

Referring to FIG. 7, in a method of providing an education service using an augmented reality according to an embodiment of the present invention, in order to provide an education content to which an augmented reality is applied, (Step S200) of providing moving picture data including the AR tag, AR script data executed according to the AR tag, and a tag recognizing program for recognizing the AR tag to the educational contents reproducing apparatus.

In step S100, the educational contents providing apparatus can confirm what type of apparatus the educational contents reproducing apparatus connected through the server communication unit provides to provide the contents in a format suitable for the type of the educational contents reproducing apparatus. For example, the server control unit 120 may determine whether the educational contents reproducing apparatus 200 is a mobile apparatus or a PC, and provide the moving picture data having a resolution suitable for the mobile apparatus or the PC to the educational contents reproducing apparatus 200.

In step S200, the educational contents providing apparatus can provide the data set according to the contents providing mode to the educational contents reproducing apparatus.

First, the educational contents providing apparatus can provide moving picture data, AR script data, and a tag recognizing program stored in the server storage unit to the educational contents reproducing apparatus in the editing moving picture providing mode. To this end, the educational contents providing apparatus may store the edited moving image data including the AR tag in the server storage unit in advance so that at least one AR tag is arranged in the tag arrangement area set on the left and right edges of the screen. Further, the educational contents providing apparatus may store the AR script data to be executed to output the preset data and the tag recognition program for recognizing the AR tag in advance in the server storage unit. In particular, the AR script data can be set to output a list of supplementary information related to the contents of education.

Here, the specific details of the AR script data and the tag recognition program are the same as those described above, and thus redundant description will be omitted.

On the other hand, the educational contents providing apparatus can provide the educational contents reproducing apparatus with the AR script data and the tag recognizing program preset by the contents provider together with the moving picture photographed in real time in the live moving picture providing mode.

Specifically, the educational contents providing apparatus can receive the moving picture photographed in real time from the education service provider and preset AR script data. At this time, the education service provider stores the supplementary information added to the training content in the training content storage device in advance in the training content provider, and when the AR script is executed together with the video captured in real time in the training content reproduction device, the supplementary information is converted into 3D modeling or multimedia And so on.

For example, referring to FIG. 8, when the educational service provider stores the operation schedule and commentary data in the lower eyelid surgery scene as the AR script data, the operation schedule and commentary data are overlaid on the video image of the lower eyelid surgery in real- To the learner.

Meanwhile, a method of providing an educational service using an augmented reality according to an exemplary embodiment of the present invention may further include receiving an inquiry of a learner from the training content reproduction apparatus and processing the training content.

Here, the educational contents providing apparatus delivers the learner's inquiry to the education service provider, receives the answer to the inquiry from the education service provider, and reflects it in the AR script data.

In the conventional video streaming service, the video data and the AR data are encoded together, and when the resolution of the moving picture is 360P, the quality of the AR is reduced to 360P. However, according to the present invention, since the AR script data is provided to the educational contents reproducing apparatus separately from the encoded moving picture data, the AR can maintain the original level resolution irrespective of the resolution of the encoded moving picture. As a result, the learner can enjoy the AR that is not encoded even if a low-resolution encoded video is watched for data saving, and the original-level resolution is maintained.

Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. In addition, it is a matter of course that various modifications and variations are possible without departing from the scope of the technical idea of the present invention by anyone having ordinary skill in the art.

Claims

In order to provide educational contents with augmented reality,

(a) the educational contents providing apparatus verifying the type of educational contents reproducing apparatus; And

(b) providing video data including an augmented reality tag, augmented reality script data executed according to an augmented reality tag, and a tag recognition program for recognizing an augmented reality tag to the educational content playback apparatus;

Wherein the augmented reality is applied to the training service.
The method according to claim 1,

Wherein the training content providing apparatus confirms the type of the training content reproducing apparatus in order to provide moving image data having a resolution suitable for the training content reproducing apparatus in the step (a).
The method according to claim 1,

In the step (b), the educational contents providing apparatus supplies data set according to the contents providing mode to the educational contents reproducing apparatus,

Wherein the content providing mode includes an editing video providing mode and a live video providing mode.

A method of providing educational service with augmented reality.
The method of claim 3,

In the editing moving picture providing mode, the educational contents providing apparatus provides moving picture data, augmented reality script data, and a tag recognizing program stored in a storage unit to the educational contents reproducing apparatus,

Wherein the moving image data includes an augmented reality tag such that at least one augmented reality tag is arranged in a tag arrangement area set on the left and right edges of the screen and the augmented reality script data is set to output preset data,

A method of providing educational service with augmented reality.
The method of claim 3,

In the live video providing mode, the training content providing apparatus provides the training content playback apparatus with the augmented reality applied thereto, which provides the moving image photographed in real time, the augmented reality script data and the tag recognition program preset by the content provider, Way.
The method according to claim 1,

Wherein the augmented reality script data is configured to output supplementary information related to education contents as a list.
The method according to claim 1,

Further comprising the step of accepting and processing an inquiry of a learner from the training content reproduction apparatus in the training content providing apparatus, and processing the augmented reality.
8. The method of claim 7,

Wherein the training content providing apparatus delivers an inquiry of a learner to a training service provider and receives an answer to a question from a training service provider and reflects it in the augmented reality script data.
The method according to claim 1,

Wherein the augmented reality tag is represented by a symbol including at least one of letters and numbers.
10. The method of claim 9,

Wherein the tag recognition program includes template data for recognizing the augmented reality tag in a moving picture transmitted in real time.
11. The method of claim 10,

Wherein the tag recognition program recognizes the augmented reality tag by performing template matching using the template data.
12. The method of claim 11,

Wherein the tag recognition program compares the ratio of each of the remaining areas with the area occupied by the augmented reality tag based on a preset tag recognition area and detects the code of the augmented reality tag by matching the comparison result with the template data, And the method of providing education services to which the present invention is applied.
13. The method of claim 12,

Wherein the template data includes a region occupied by the augmented reality tag and a code of the augmented reality tag according to a ratio of each of the remaining regions.
14. The method of claim 13,

Wherein the template data includes the code of the augmented reality tag set for each resolution of the moving image.