KR20220124351A - System and method for providing language learning service based on artificial intelligence - Google Patents

Abstract

The present invention relates to a system and method for providing a language learning service. According to one embodiment of the present invention, the system for providing a language learning service includes: a service server providing image data and subtitle data corresponding to the image data; and a user terminal recognizing and evaluating at least one between voice data and text data input by a user in a section pre-set based on the corresponding subtitle data when playing an image based on the image data and performing continuous playing of the image or repetition learning according to a result of the evaluation.

Description

SYSTEM AND METHOD FOR PROVIDING LANGUAGE LEARNING SERVICE BASED ON ARTIFICIAL INTELLIGENCE

The present invention relates to a system and method for providing an artificial intelligence-based language learning service, and more particularly, to a system and method for providing an artificial intelligence-based language learning service that enables conversation with a person in media based on images and subtitles.

Recently, a number of methods of learning a language have been proposed through the Internet or video media. It mainly provides words, sentences, and grammar for the language to be learned. In particular, as part of improving language listening ability, It also provides the pronunciation of native speakers.

However, if a foreign language is simply interpreted in the mother tongue in units of words or sentences, or if a long sentence is repeatedly heard, the learning effect may be reduced and the learning effect may be reduced and boredom may occur. In addition, if you understand only the dictionary meaning of a word and learn the pronunciation, you cannot use it properly for English writing or conversation with foreigners, so you cannot achieve the purpose of learning a foreign language. It is not something that can be used as a tool, so it only makes them lose interest in learning and make them tired.

In addition, since most lectures were conducted through a monitor screen between a single learning provider and a single or multiple learners, accurate and efficient language learning does not proceed according to the learner's level of understanding, as well as a unilateral injection method. Since it is conducted in the form of learning, there were problems such as not being able to bring out the learner's personal achievement to the maximum.

Therefore, a technology that enables language learning based on images such as animation, movies, and dramas, but uses artificial intelligence to recognize the user's voice or text, so that the language can be learned in a conversational manner will be developed. There is a need.

Korean Patent Application Laid-Open No. 10-2003-0079497 (published on October 10, 2003)

The present invention has been proposed in order to solve the above problems, so that language can be learned based on an image, and the language can be learned in a conversational manner by recognizing a voice or text input from a user, An object of the present invention is to provide a system and method for providing a language learning service that arouses user's interest while improving learning efficiency.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present invention, there is provided a system for providing a language learning service, comprising: a service server providing image data and caption data corresponding to the image data; and when reproducing an image based on the image data, recognizes and evaluates at least one of voice data and text data input by the user in a preset section based on the corresponding subtitle data, and continues the image according to the evaluation result It includes a user terminal for playing or performing repetitive learning.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, it is possible to learn a language based on an image, but by recognizing a voice or text input from a user to learn a language in a conversational manner, thereby stimulating the user's interest and improving the learning efficiency. make it possible

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a network configuration diagram of a language learning service providing system according to an embodiment of the present invention. 2 is a flowchart illustrating a method of providing a language learning service according to an embodiment of the present invention. 3A to 3F are diagrams illustrating a display of a user terminal that changes as a user performs learning based on a method of providing a language learning service according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully inform those skilled in the art of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly specifically defined.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. A spatially relative term should be understood as a term that includes different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, "part" or "module" is not meant to be limited to software or hardware. A “part” or “module” may be configured to reside on an addressable storage medium and may be configured to reproduce one or more processors. Thus, by way of example, “part” or “module” refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or additional components and “parts” or “modules”. can be further separated.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a network configuration diagram of a language learning service providing system according to an embodiment of the present invention.

Referring to FIG. 1 , the language learning service providing system 100 may include a service server 110 and a user terminal 130 . Here, the user terminal 130 may be a tablet PC, a smart phone, a wearable device, a laptop, a daystop, and the like, but is not limited thereto.

The service server 110 is a server for providing a language learning service, and provides image data for language learning and caption data corresponding to the image data to the user terminal 130 .

Here, the subtitle data includes a subtitle set composed of sentences (texts) having a predetermined length, and each sentence included in the subtitle set has a timeline according to the reproduction time of the image. Also, the subtitle data may include a plurality of subtitle sets. In this case, the plurality of subtitle sets may be configured in different languages.

For example, the subtitle information may be configured to include a first subtitle set in which the first language is Korean, and second subtitle sets in which the first subtitle set is translated into English, Chinese, Japanese, etc. as a second language, respectively. have. In this case, there is no limitation on the type and number of languages, and this is not limited thereto. In addition, according to a user's setting, at least one subtitle set among a plurality of subtitle sets may be displayed on the display of the user terminal 130 when an image is reproduced, or only an image may be reproduced without a subtitle set.

Meanwhile, the extension of the image data may be any one of avi, mov, wmv, asf, mpeg, and flv, and the extension of the subtitle data may be any one of smi and srt.

The user terminal 130 is a terminal used by a user to perform language learning, and performs language learning by receiving image data and subtitle data from the service server 110 . In this case, the user terminal 130 may perform language learning based on a web page or a separate application.

Specifically, when the image data and the subtitle data corresponding thereto are downloaded from the service server 110 , the user terminal 130 reproduces the image based on the image data, and displays the subtitles according to the timeline based on the reproduction data. to output

Before playing the video, the user can designate a character appearing in the video or select a specific sentence to set the learning sections in which he or she wants to learn.

The user terminal 130 stops outputting the speaker or lowers the volume to a preset size at the point in time when the subtitle corresponding to the learning section is to be output, but outputs the output, but an API (Application Application for Recognition of the user's voice and/or text) programming interface).

The user terminal 130 recognizes voice and/or text based on voice data input as the user utters and/or text data input as the user inputs text, and uses a preset algorithm to determine the similarity ( accuracy). In other words, the similarity (accuracy) of the input voice data or text data is determined based on the original data, that is, the image data and the caption data.

However, this is only one embodiment, and it may be set to recognize only one of voice and text, or may be set to recognize at the same time, but is not limited thereto. Here, when it is set to recognize text, the user may input text data by touching a keyboard displayed on the display of the user terminal 130 or pressing a button provided in the user terminal 130 , and may be displayed on the display. Character data can also be input by directly writing characters using a touch pen. Additionally, when the voice recognition is set, the input voice data may be converted into text and displayed on the display of the user terminal 130 .

Meanwhile, the user terminal 130 may determine whether to continue playing the image according to the similarity determination result. At this time, a numerical value indicating the degree of similarity is calculated by comparing the voice data or text data with the original data, and if the calculated numerical value is equal to or greater than a preset threshold value, the image is continuously played, and the calculated numerical value is preset If it is less than the threshold value, iterative learning may be performed.

For example, a ratio between a word included in the input voice data or text data and a word included in the original data is calculated, and if the ratio is greater than or equal to a preset threshold, the image is continuously played, and the ratio is set in advance. If it is less than the set threshold, iterative learning is performed.

2 is a flowchart illustrating a method of providing a language learning service according to an embodiment of the present invention.

Referring to FIG. 2 , the user terminal 130 downloads image data and subtitle data corresponding thereto from the service server 110 and inputs (stores) it (S201), and detects that the user touches (clicks) the play button. At the bottom (S203), it is checked whether there is subtitle data (S205).

As a result of checking in step S205, if there is no caption data, simple video playback is performed based on the video data (S207), and if caption data exists, it is checked whether the caption-based voice recognition API function is allowed to be set (S209) ).

As a result of the check in step S209, if the subtitle-based voice recognition API function is not allowed to be set, it is checked whether the subtitle-based character recognition API function is set allowed (S211).

As a result of checking in step S211, if the subtitle-based character recognition API function is also not allowed to be set, simple video playback is performed (S207), and if the subtitle-based character recognition API function is set, character recognition is performed when playing the video. to operate the API (S219).

On the other hand, as a result of checking in step S209, if the subtitle-based voice recognition API function is allowed to be set, it is checked whether the subtitle-based character recognition API function is set allowed (S213).

As a result of checking in step S213, if the subtitle-based character recognition API function is allowed to be set, both the API for voice recognition and the API for character recognition operate during video playback (S215), and the subtitle-based character recognition API function is enabled. If the setting is not allowed, the API for voice recognition is operated during video playback (S217).

That is, the API for voice recognition and the API for character recognition operate together or which Make only one optional operation.

On the other hand, Figure 2 does not show the step of designating a character appearing in the video or setting the learning sections to be learned by selecting a specific sentence before playing the video, but learning after step S201 or step S203 A step of setting a section may be included.

3A to 3F are diagrams illustrating a display of a user terminal that changes as a user performs learning based on a method of providing a language learning service according to an embodiment of the present invention.

When a user executes language learning based on an application or a web page through the user terminal 130 , a video list that the user can select for language learning is displayed on the display as shown in FIG. 3A . Although FIG. 3A illustrates a case in which one image is included in the image list, this is only an example and a plurality of images may be displayed.

When the user touches and selects an image included in the image list, a subtitle list (display sublist) corresponding to the selected image is displayed on the display of the user terminal 130 as shown in FIG. 3B . In this case, the subtitle list may include at least one subtitle set for the corresponding image, and each subtitle set may be translated into different languages. Here, the subtitle set is to be displayed together with the image on the display when the image is reproduced.

Thereafter, as shown in FIG. 3C , a subtitle list including subtitle sets to be used for voice recognition or character recognition is displayed on the display of the user terminal 130, and the user can select one of the subtitle sets by touching one of the subtitle sets. . Through the selected subtitle set, the user can evaluate whether the similarity (accuracy) of voice data input by the user or text data input through a touch method or the like.

In this way, when the user completes the selection of the subtitle set, the learning section can be set by directly selecting the sentence the user wants to learn from among the subtitle data included in the subtitle set as shown in FIG. 3D . Specifically, the user selects a sentence he wants to learn from among the sentences corresponding to each timeline, and when he wants to learn the sentence through voice recognition, it can be configured to select by touching the microphone icon displayed along with the sentence. have. Meanwhile, although not shown in FIG. 3D , when a pen icon is further displayed along with a sentence to learn a corresponding sentence through character recognition, the pen icon may be selected to be selected. Of course, if you want to learn through both voice recognition and character recognition, you can select both icons.

When the user plays an image after setting all the learning sections, learning through voice recognition or character recognition may be performed according to the set learning section. That is, when it is set to learn through voice recognition for the corresponding learning section, the voice uttered by the user can be recognized by calling the API for voice recognition on the timeline corresponding to the selected sentence as shown in FIG. 3E. When it is set to learn through character recognition for the corresponding learning section, the API for character recognition is called on the timeline corresponding to the selected sentence as shown in FIG. 3f to recognize characters input through touch from the user. .

The steps of the method or algorithm described in relation to the embodiment of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: language learning service provision system 110: service server 130: user terminal

Claims (2)

a service server providing image data and caption data corresponding to the image data; And When an image is reproduced based on the image data, at least one of voice data and text data input by the user is recognized and evaluated in a preset section based on the corresponding subtitle data, and the image is continued according to the evaluation result A system for providing a language learning service based on artificial intelligence, characterized in that it comprises a user terminal for reproducing or performing repetitive learning. The artificial intelligence of claim 1, wherein the user terminal calls at least one of an API for voice recognition and an API for character recognition in a learning section set based on the subtitle data when playing an image. based language learning service provision system.

Images