KR20220123170A - Language Learning System and Method with AI Avatar Tutor - Google Patents

Abstract

The present invention provides a method of providing a conversation learning system using an artificial intelligence avatar tutor, which allows a user to effectively perform conversation learning while correcting an expression of the user to a better expression or suggesting a better expression. The method comprises: a step in which an artificial intelligence avatar explains a conversation subject and a situation to a user; a step in which the artificial intelligence avatar tells a question suitable for the subject and the situation first while starting a conversation; a step in which the artificial intelligence avatar shows proper reactions while the user speaks; a step of converting a response of the user into text to understand the response in accordance with the context of the current conversation and generating and expressing a response and a gesture of the artificial intelligence avatar; a step of generating a sentence as a better expression than the response of the user or an expression in which a grammatical error is corrected; and a step of inserting an advertisement banner into the background of the artificial intelligence avatar in accordance with the conversation subject and the situation.

Description

Conversation learning system and method using AI Avatar Tutor {Language Learning System and Method with AI Avatar Tutor}

The present invention relates to a method by which a user can improve his or her conversational skills while having a conversation with an artificial intelligence avatar, and more particularly, the artificial intelligence avatar becomes a tutor and is displayed on the screen, while presenting a specific topic and situation to the user. , receives the user's voice input, understands the meaning, and plays the audio response appropriate to the context of the conversation. We have a continuous conversation with the user and suggest corrections to the user's expression or a more appropriate expression. Also, it relates to a learning system and a conversation learning method that enable conversational learning by showing gestures suitable for each context when an AI avatar is listening or speaking in order to give a feeling of natural conversation like a human.

With the development of technology for learning foreign languages, we have reached an era where we can access a conversational learning system using live-action human tutors. Instead of a real human tutor, a live-action avatar with the same voice and mouth shape as that of a tutor is shown to the user, and after a specific topic is selected, the user is instructed to follow the set conversation. That is, through a set topic and a dialogue script, the live-action avatar reads the script and changes the mouth shape to match the sound. The user sees the response text of the set dialogue script and learns by reading along. You cannot have any other conversation other than the script given to you. Also, at the level of reading according to the set script, the user cannot use the expression immediately thought and created during the conversation, so the expression cannot be corrected.

In the conventional live-action avatar technology, it is focused on correcting a specific pronunciation rather than the experience of having a conversation with a user. The user looks at the given script and checks whether the pronunciation is good, and when the actual live-action avatar responds, it concentrates on the expression of the mouth shape to make the correct pronunciation.

As a disadvantage of this technique, conversational learning using a live-action avatar has a limit in improving the user's conversational ability by focusing on learning the correct pronunciation and fixed expressions rather than pre-talking.

The present invention has been proposed to solve the problems of the prior art as described above. The present invention provides an artificial intelligence avatar tutor to explain a given topic and situation to a user, lead a conversation naturally, and put the user's utterance into a conversation context. It shows the reaction with appropriate gestures and sounds while understanding it properly, and after the user's utterance is finished, it generates a response that matches it, and then plays the response as audio along with the gesture. Another object of the present invention is to provide a conversation learning system and a conversation learning method using an artificial intelligence avatar tutor that allows the user to effectively learn conversation while correcting or suggesting a better expression of the user.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention provides an artificial intelligence avatar explaining a conversation topic and situation to a user, the artificial intelligence avatar explaining a conversation topic and situation while starting a conversation, , saying the appropriate question or request, the AI avatar showing appropriate reactions and movements while the user is speaking, converting the user's response into text to understand it according to the context of the current conversation, and It includes the steps of generating and expressing gestures, generating sentences with expressions that are better than the user's response or correcting grammatical errors, and inserting advertisement banners into the background of the AI avatar according to the topic and situation of the conversation. .

According to any one of the above-described problem solving means of the present invention, the artificial intelligence avatar becomes a conversational learning tutor, leads a conversation suitable for a given conversation topic and situation, and has a real conversation through gestures and chuimsae during conversation with the user experience, understand user utterances based on the context of the current conversation, generate responses and gestures corresponding to them, suggest better expressions for user utterances, and generate correct sentences for grammatical errors. Conversation that helps users to improve their conversational skills by overcoming the limitations of existing AI tutors who learn specific response expressions and pronunciations, and can have natural conversations with users on arbitrary topics A learning system can be provided.

1 is a diagram for explaining a conversation learning system using an artificial intelligence avatar tutor according to an embodiment of the present invention.
2, 3, 4, 5, and 6 are diagrams for explaining an embodiment in which a conversation learning system using an artificial intelligence avatar tutor according to an embodiment of the present invention is implemented.
7 is a diagram illustrating a process in which data is transmitted/received between components included in the conversation learning system using the artificial intelligence avatar tutor of FIG. 1 according to an embodiment of the present invention.

Embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts not related to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is 'connected' with another part, this includes not only the case where it is 'directly connected', but also the case where another module is connected in the middle. In addition, when a part 'includes' a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

The terms 'about', 'substantially', etc. to the extent used throughout the specification are used in or close to the numerical value when manufacturing and material tolerances inherent in the stated meaning are presented, and are intended to enhance the understanding of the present invention. To help, precise or absolute figures are used to prevent unfair use by unscrupulous infringers of the stated disclosure. As used throughout the specification of the present invention, the term 'step for (to)' or 'step for' does not mean 'step for '.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware.

In this specification, some of the operations or functions described as being performed by the terminal, apparatus, or device may be performed instead of in a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal means mapping or matching the terminal's unique number or personal identification information, which is the identification data of the terminal. can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a conversation learning system using an artificial intelligence avatar tutor according to an embodiment of the present invention. Referring to FIG. 1 , a conversation learning system 1 using an artificial intelligence avatar tutor includes a client 100 that a user can access to the system, a network 200, and a server providing an AI avatar tutor conversation learning service. (300) may be included. 1 is only an embodiment of the present invention, and thus the present invention is not limitedly interpreted through FIG. 1 .

Each component of FIG. 1 is generally connected through a network 200 . Here, the network refers to a connection structure in which information exchange is possible between each node, such as a plurality of terminals and servers. Examples of such networks include RF, 3rd Generation Partnership Project (3GPP) network, Long Term Evolution (LTE). ) network, 5GPP (5th Generation Partnership Project) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, NFC network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, and the like are included, but are not limited thereto. The term at least one is defined as a term including the singular and the plural, and even if the at least one term does not exist, each element may exist in the singular or the plural, and it is obvious that it may mean the singular or the plural. will be. In addition, whether each component is provided in singular or plural may be changed according to an embodiment.

The client 100 includes an avatar control unit 110 that allows the AI avatar to have a direct conversation with the user, a content control unit 120 that provides a conversation topic and situation guide, and displays the conversation between the AI avatar and the user in text. , the dialogue management unit 130 that displays and corrects more improved or different expressions or grammatical errors than the user's expression, the connection between the three modules 110, 120, 130, and the internal system function of the terminal and the server 300 ) may be composed of a system control unit 140 that controls the system, and a service management unit 150 that can set a user's account, conversation history, and settings.

The avatar controller 110 shows the artificial intelligence avatar's whole body or body part including the face with a specific background. The types of AI avatars exist in various forms and shapes depending on the character, and the appearance of the same AI avatar may vary depending on the current conversation topic and situation. The background also changes. For example, if it is a topic related to a summer vacation, it may appear as a scene drinking a mojito in a swimsuit on the beach.

The AI avatar changes its gestures depending on when it hears the user speaking, when it speaks to the user, and when it is waiting for the next input. While the user is uttering, it understands the context of the user's situation and the meaning of the utterance, shows the appropriate reaction according to the context, shows that the user is listening well, and allows the conversation to continue naturally. For example, when the user speaks with joy, he communicates with the user sympathetically by making bright gestures and gestures that respond to such feelings. When the AI avatar speaks to the user, it shows a gesture that fits the context, effectively conveying the meaning to the user and allowing the user to be immersed in the conversation. For example, when an AI avatar tries to persuade a user in the context of utterance, the facial expression, gesture, and tone of the AI avatar are expressed differently from their usual appearance. If you wait for the user to speak or take no action, the AI avatar will wait and continue the conversation with appropriate gestures. For example, if there is no response from the user for several seconds after the AI avatar asks a question to the user, the user waits with a waiting gesture that can sufficiently think of a response.

Also, the avatar controller 110 may automatically adjust the artificial intelligence avatar and the user's speech input turn. For example, after the AI avatar finishes speaking, the microphone turns on automatically for the user to speak after a few seconds, and the sound and microphone icon are changed to a recording icon so that the user can recognize it. When the user has finished speaking, the system recognizes it and displays the microphone icon as off. The user's utterance input can also be performed manually. When the AI avatar is speaking, the status display is displayed as AI avatar is speaking, and when the AI avatar finishes speaking, the status display changes to a microphone icon. In this case, when the user presses the microphone button, the user can record his or her utterance. When the user finishes speaking and presses the icon during recording again, the recording ends, and the microphone icon is turned off.

The avatar control unit 110 naturally displays an advertisement banner in the background together with the artificial intelligence avatar. A background suitable for the topic and situation of the current conversation appears, and the advertisement banner in the background is curated by the advertisement providing unit 360 of the server 300 according to the topic and situation of the current conversation. For example, if the topic of the current conversation is about printer devices, advertisements are naturally exposed to users by inserting logos or products of printer device manufacturers such as Xerox and Canon in the background.

The content control unit 120 provides conversation topics and context information between the artificial intelligence avatar and the user. Here, the conversation topic includes most areas including daily life and professional areas, and the conversation topic may be displayed in text and pictures. The context information may be set in various ways, such as a role between the AI avatar and the user, problem solving, help, and explanation. It also reads a specific fingerprint and initiates a conversation between the AI avatar and the user with a conversation topic and situation related to the fingerprint.

The content controller 120 provides a control button so that the user can move the AI avatar and a new topic. Also, the avatar controller 110 may tell the user to change the subject through utterance. For example, the user may suggest to the AI avatar to have a conversation on a different topic or specify a specific topic to talk to. When the conversation topic changes in this way, the AI avatar explains the changed topic and situation and then naturally continues the conversation.

When the AI avatar or the user's utterance is finished in the avatar control unit 110 , the text of the utterance is displayed on the conversation management unit 130 . Here, the inputted text is transmitted to the interaction processing unit 330 and the language proofing processing unit 340 of the server 300, and the dialogue management unit 130 corrects errors for the user input text and corrects the results of paraphrasing. It is received from the processing unit 340 and displayed on the screen of the conversation management unit 130 . Error display and correction for user input text displays an error section in the text and also displays a corrected expression. Paraphrasing refers to a sentence containing a better expression or a different expression for the same meaning based on the user input text, and may be displayed with a separate mark at the bottom of the user input text. For example, when there are only some words rather than a complete sentence due to the lack of language expression skills of the user, the language correction processing unit 340 may also consider creating a complete sentence suitable for the conversation context as paraphrasing.

The conversation management unit 130 also provides audio files for the artificial intelligence avatar and the user's conversation text. It includes audio of the voice of the AI avatar, audio uttered by the user, audio generated by the user's utterance text with the voice of the AI avatar, and audio of paraphrased text. By providing these texts and audio, users can hear their own pronunciation and the pronunciation of the AI avatar again. Also, when the AI avatar incorrectly recognizes the user's utterance or makes a response that does not fit the context, the conversation manager 130 may also provide a function for the user to directly provide feedback to the system.

The system control unit 140 controls the connection between the three modules 110 , 120 , 130 and the internal system function of the terminal and the server 300 . More specifically, a conversation topic with the first user is received from the content management unit 320 of the Server 300 through the network 200 and delivered to the content control unit 120 . In addition, the avatar management unit 310 receives the persona of the artificial intelligence avatar and a background suitable for the current conversation topic, and the advertisement providing unit 350 receives an advertisement banner suitable for the current conversation topic and transmits it to the avatar control unit 110 . Thereafter, the avatar controller 110 triggers a conversation on a new topic, and the AI avatar recognizes what the avatar has said and delivers it to the conversation manager 130 . In addition to the flow described above, it may include all connections and control between other client-server modules.

The service manager 150 manages the user's account, selects an artificial intelligence avatar with which the user wants to talk, and provides a history of previous conversations in audio and text format. In relation to the user's account part, the user can select a rate plan, a certain amount of conversation topics can be used free of charge to the user who signed up for the service for the first time, and the content control unit 120 changes the topic or the number of allocated conversation turns is exhausted. If enabled, a pop-up message will allow the user to select a plan. The conversation history is automatically tagged with a conversation topic, and may be displayed as a thumbnail along with a picture of an AI avatar or a conversation topic background.

The server 300 defines the persona and gesture of the artificial intelligence avatar, the avatar manager 310 manages the artificial intelligence avatar that varies depending on the topic and situation of the conversation, the user and the artificial intelligence avatar have a conversation topic and A content management unit 320 that provides a situation, a persona of an artificial intelligence avatar, a conversation topic and situation, an interaction processing unit 330 that generates a response and an appropriate gesture according to the context of the current conversation, and the text uttered by the user Artificial intelligence in the language proofing processing unit 340 that analyzes to generate a better expression suitable for the context and detects grammatical errors, the account management unit 350 that manages users registered through the client 100, and the avatar control unit 110 It may be composed of an advertisement providing unit 360 that inserts a natural advertisement banner in the background of the avatar.

The avatar manager 310 defines and manages avatars that are AI tutors. The AI tutor may have a human-like personality, history, or experience, and based on this, the interaction processing unit 330 generates a response and a gesture. AI avatars can have human-like shapes and can appear as various characters, such as animals or characters from games. Such an AI avatar can take various gestures depending on the topic or context of the conversation, and each avatar may have different gestures even in the same conversation context. The main gestures defined in this way are used in the interaction processing unit 330 . The AI avatar can use its own unique expressions, and these main expression methods are also defined in the avatar management unit 310 . AI avatars also have their own voice. The voice synthesis model for generating such a voice is learned in advance and registered in the avatar management unit 310 , and is downloaded to the client 100 when the service management unit 150 selects an artificial intelligence avatar with which the user wants to talk.

The content management unit 320 provides a topic and a situation for a conversation between the user and the AI avatar. These topics and situations depend on the conversational ability of the conversational user. Conversation topics may vary from topics related to daily life to knowledge in specialized fields, and conditions such as roles, questions and answers, problem solving, explanations, etc., may be given between the user and the AI avatar.

The interaction processing unit 330 generates a response and an appropriate gesture according to the persona of the AI avatar, the topic and situation of the conversation, and the context of the current conversation. In generating a response, a base model is created by learning actual conversation data between people through deep learning, and a conversation model is created by additionally learning conversation data suitable for the persona of each AI avatar on a specific topic. In addition, since the purpose of conversational learning is rather than the conversation between the AI avatar and the user, frequently asked questions or requests by the user are generated using a rule-based learning model. In addition, in the case of greetings or conversation topic guides with users to start a conversation, a predefined template-based conversation phrase is created and used.

The gestures of the AI avatar are the gestures of the situation the AI avatar hears when the user speaks, the gestures that the AI avatar takes while speaking and then interprets the responses after hearing, and the gestures that the AI avatar takes while the user speaks and does not perform any action on the user's input. Create a gesture. In a listening situation, it shows that the user is listening to what is being said, and if there is a pause in the conversation, it gives a reaction based on understanding up to that point. For example, if the AI avatar is persuaded or persuaded by the user's words, it can generate a reaction saying yes while nodding its head.

The AI avatar generates a response after understanding the user's words, and when the AI avatar needs emphasis or instructions while the AI avatar speaks, a gesture using body language may be included.

The language proofing processing unit 340 generates a better expression according to the context of the text spoken by the user, and generates a corrected expression when there is a grammatical error. Paraphrasing of the language proofing processing unit 340 uses a fine-tuning model to create a language model with a large amount of text data in advance, and to generate a sentence with different expressions when an input sentence comes in. do. For grammatical errors, it is determined whether the input sentence contains errors through the pre-trained language model, finds the section with errors, and replaces the sentence with the corrected expression.

The account management unit 350 analyzes the user's account information and learning. By analyzing the conversation history the user had with the AI avatar, it identifies which topics and expressions need improvement. Thereafter, the content management unit 320 may periodically generate similar topics and situations as a conversation guide until the user is familiar with the corresponding topics and expressions.

The advertisement providing unit 360 manages an advertisement banner that will naturally fit into the background of the avatar control unit 110 , and when the content management unit 320 sets a conversation topic and situation, it is included in the corresponding topic and situation. The avatar control unit 110 inserts a banner having the highest degree of relevance into the background by calculating the degree of association between the background and the advertisement banner.

2 to 6 are diagrams for explaining an embodiment in which a conversation service using a conversation learning system using an artificial intelligence avatar tutor according to an embodiment of the present invention is implemented.

Referring to FIG. 2 , when a user starts a conversation session with an AI avatar, a greeting or ice breaking topic is given to the content controller 120 and a conversation begins. When the AI avatar speaks, the AI avatar control unit 110 indicates that the AI avatar is speaking, and takes appropriate gestures according to the speaking context. When the AI avatar finishes speaking, it is displayed as text on the conversation management unit 130 .

Referring to FIG. 3 , after the AI avatar finishes speaking, a microphone for receiving user utterance is turned on automatically after a few seconds, a recording icon is displayed so that the user can recognize it, and a recording start sound is played. Accordingly, when the user responds appropriately to the AI avatar, says to have a full-fledged conversation, or presses a button to move to the next topic in the content control unit 120, the AI avatar continues the conversation with the next conversation topic and situation. . The user's speech input can also be performed manually. That is, when the AI avatar finishes speaking, the user presses the microphone button of the AI avatar control unit 110 to input a utterance, and at this time, a recording is displayed. .

Referring to FIG. 4 , the content controller 120 is given a topic and situation with which the AI avatar and the user want to communicate. These conversations and situations are curated by the content management unit 320 of the Server 300 . When the AI avatar is given a new conversation topic and situation, the AI avatar control unit 110 delivers an appropriate conversation guide to the user through words and gestures, and displays the text on the conversation management unit 130 . In order for the AI avatar to speak in the corresponding conversation session, the current conversation topic and context information are sent to the interaction processing unit 330 of the server 300 through the system control unit 140, where the text and gestures to be spoken by the AI avatar are sent. is generated and transmitted back to the system control unit 140 . In the AI avatar control unit 110 , the AI avatar takes the gesture received from the interaction processing unit 330 , and converts the text into the voice audio of the AI avatar and speaks. When the AI avatar finishes speaking, the conversation management unit 130 displays what the AI avatar said as text.

In the AI avatar background of the AI avatar control unit 110, visual data on a current conversation topic and situation are displayed, and an advertisement banner may be included in the visual data. For example, the topic of conversation in Fig. 3 is occupation and hobbies, and the context information provides the background for the conversation setting that the user likes to be a software engineer and mobile game, and the background for meeting with Google Analytics at the company and playing games on the Play Station at home. can do.

Referring to FIG. 5 , when the AI avatar finishes speaking, the user inputs a response by voice, and the conversation management unit 130 displays text indicating that the AI avatar recognizes the user's utterance. The user sends the spoken text and the conversation content of the current conversation topic session to the proofing processing unit 340 of the server 300 through the system control unit 140, and a better expression or grammatical error than the response sentence entered by the user A sentence including the corrected expression is generated and sent back to the system controller 140 . The corrected sentence is displayed at the bottom of the user's utterance text of the conversation management unit 130 . In addition, a section with a grammatical error is displayed in the user utterance text, and a corrected phrase is also provided.

Referring to FIG. 6 , the user utterance text and the conversation content of the current conversation topic session are also sent to the interaction processing unit 330 of the server 300 , where the user utterance is understood based on the context of the conversation, and a response sentence corresponding thereto and create a gesture. It generates responses or follow-up questions that continue the flow of conversation by identifying entity names, referential pronouns, and related relationships in the current conversation session. As described above, the interaction processing unit 300 also generates a sequence of gestures to be displayed while the AI avatar speaks a response or follow-up question.

As described above, the conversation service using the conversation learning system using the artificial intelligence avatar tutor in the drawings from FIGS. 2 to 6 is not described above. Since it is the same as the content described with respect to the method of providing or can be easily inferred from the described content, the following description will be omitted.

The method of providing a conversation learning system using an artificial intelligence avatar tutor according to an embodiment described with reference to FIGS. 2 to 6 is a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. It can also be implemented in the form of Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Referring to FIG. 7 , it is a diagram illustrating a process in which data is transmitted/received between components included in the conversation learning system using the artificial intelligence avatar tutor of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process in which data is transmitted and received between each component will be described with reference to FIG. 7 , but the present application is not to be interpreted as such an embodiment, and the It is obvious to those in the technical field that the process in which data is transmitted and received can be changed.

Referring to FIG. 7 , the artificial intelligence avatar tutor conversation learning system server 300 is provided with an artificial intelligence avatar interaction model, a language correction model, a conversation topic and an advertisement curation model from at least one model learning server 400 , (S1100), as the system starts, the corresponding model is loaded into the engine (S1100).

The user selects an artificial intelligence avatar through the client 100 and enters a conversation session (S2000). The server 300 selects a topic and situation in which the user and the artificial intelligence avatar will talk, and sets the appearance of the related avatar and a background including advertisements (S2100). The client 100 receives the conversation topic, situation, avatar appearance, and background information from the server 300 ( S2200 ), and displays it on the screen.

The artificial intelligence avatar explains the conversation topic and guide to the user, and starts the conversation (S3000). The user provides his or her own response to the AI avatar's question or request through speech, and the client transmits the user's speech text and current conversation content to the server 300 through speech recognition (S3100). The server 300 understands the conversation context, generates response text and gestures, generates a paraphrasing sentence for the user input text, and also generates a corrected expression after detecting an error section ( S3200 ).

Accordingly, the client 100 receives the response, gesture, and paraphrasing sentence and error section detection and correction expression generated by the server 300 ( S3300 ), and responds to the user through the artificial intelligence avatar, and talks Paraphrasing sentences and corrected errors in the user input text are displayed in the text view (S3400). Until the session of the current conversation topic ends ( S3500 ), the above-described steps of the conversation between the artificial intelligence avatar and the user are repeated in a loop.

When the specific conversation topic session ends (S3500), the server 300 selects the next conversation topic based on the user's conversation tendency or level, and selects the conversation topic and situation. The above-described steps for each conversation topic are repeated in a loop, and when each specific conversation topic session is ended, the conversation content and user feedback between the user and the AI are updated to the model learning server 400 (S4100).

The order between the above-described steps ( S1100 to S4100 ) is merely an example and is not limited thereto. That is, the order between the above-described steps S1100 to S4100 may be mutually changed, and some of the steps may be simultaneously executed or deleted.

The matters not described for the method of providing a conversation learning system using the artificial intelligence avatar tutor of FIG. 7 are the same as those described for the method of providing the conversation learning system using the artificial intelligence avatar tutor through the other drawings described above. or, since it can be easily inferred from the described content, the following description will be omitted.

The above-described method for providing a conversation learning system using an artificial intelligence avatar tutor according to an embodiment of the present invention may include an application basically installed in a terminal (which may include a program included in a platform or an operating system basically installed in the terminal) ), and may be executed by an application (ie, a program) installed directly on the master terminal by a user through an application providing server such as an application store server, an application, or a web server related to the corresponding service. In this sense, the method for providing a video chat service using virtual reality-based interactive artificial intelligence according to an embodiment of the present invention described above is implemented as an application (that is, a program) installed basically in a terminal or directly installed by a user and may be recorded on a computer-readable recording medium such as a terminal.

The foregoing description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (9)

A method of providing a conversation learning system using an artificial intelligence avatar tutor running on a user terminal and a server for providing a conversation learning system, the method comprising:
Collecting artificial intelligence avatar dialogue modeling data, gesture change data related to the dialogue, data obtained by paraphrasing dialogue text, dialogue text error section and correction expression data from the dialogue data providing server;
An interaction model is learned using an artificial intelligence neural network algorithm with the collected dialog modeling data and gesture change data, and a paraphrasing model is learned using an artificial intelligence neural network algorithm with dialog modeling data and paraphrasing data. and learning a Grammar Error Correction model using the dialogue text, the error section, and the correction expression data;
accessing a conversation learning system using an artificial intelligence avatar tutor from a user terminal;
Driving the conversation topic and situation from the conversation content providing server, the avatar persona and the gesture model from the artificial intelligence avatar management server to the client;
loading an avatar persona and a gesture model from an AI avatar management server to a client when another AI avatar is selected by the user terminal;
After the AI avatar understands the conversation topic and situation, generating sentences and gestures to explain to the user, and explaining to the user;
The AI avatar generates and speaks questions and gestures based on understanding the current conversation topic and situation, and conversation history with the user;
When the user understands the question and gives a response, the user performs voice recognition and then takes appropriate reactions and actions;
when the user's response is finished, converting the response into text, and understanding it according to the current conversation topic and situation, and the conversation context;
generating and saying appropriate responses and gestures based on understanding;
generating and displaying a sentence in which the text responded to by the user has better expression and grammatical errors that fit the conversation context;
Evaluating the user's conversation level using an artificial intelligence neural network algorithm based on the user's conversation history with the AI avatar, and learning a curation model that presents conversation topics and situations similar to weak topics or expressions;
learning a relevance model using an artificial intelligence neural network algorithm with the collected dialogue modeling data and advertisement contents collected from an advertisement banner providing server;
and inserting an advertisement banner in the background of the artificial intelligence avatar according to the conversation topic and situation.
A method of providing a conversational learning system using an artificial intelligence avatar tutor.
The method of claim 1,
Before the step of accessing the conversation learning system using the artificial intelligence avatar tutor from the user terminal,
defining at least one AI avatar persona and performing conversation modeling;
Conversation modeling data and gesture data used by the at least one artificial intelligence avatar are mapped, dialogue modeling data and paraphrasing data are mapped, and dialogue text data and error sections and corrected expression data are mapped and stored. to do;
storing a user's response in the conversation topic and situation, data to evaluate it, and data mapped between similar conversation topics and situations;
mapping and storing advertisement content related to the given conversation topic and situation;
A method of providing a conversational learning system using an artificial intelligence avatar tutor that further comprises.
The method of claim 1,
After analyzing the input user voice through a natural language processing algorithm, the step of determining a response text and a gesture of the input voice comprises:
extracting a feature (spectogram) from the input voice signal and recognizing an emotion using the extracted feature;
Including, wherein the emotion recognition is to collectively refer to general emotions such as joy, sadness, and pain that a person can feel,
A method of providing a conversational learning system using an artificial intelligence avatar tutor.
The method of claim 1,
After analyzing the input user voice through a natural language processing algorithm, the step of determining a response text and a gesture of the input voice comprises:
partially understanding the inputted user response and providing a chuimsae and a gesture reaction while the user is speaking;
Including, wherein the partial understanding is to show the chuimsae and gestures suitable for the context of the paragraph by breaking it into a paragraph in which the meaning is distinguished in the middle of what the user is saying,
A method of providing a conversational learning system using an artificial intelligence avatar tutor.
The method of claim 1,
The step of speaking the generated response text together with the generated gesture of the AI avatar comprises:
extracting a pre-stored gesture animation clip corresponding to the determined gesture;
generating and synthesizing intermediate-stage images for smoothing the connection between the extracted animation clips;
A method of providing a conversational learning system using an artificial intelligence avatar tutor, which is executed by performing
The method of claim 1,
The step of generating and displaying the text that the user responded to with better expression and grammatical errors corrected for the context of the conversation,
generating another representation of the user response in a paraphrasing model, using the current conversation context, a conversation history including user response text, a conversation topic and situation as inputs;
Putting the inputted user response text into a Grammar Error Correction model, detecting error sections, and generating a corrected representation of each section;
A method of providing a conversational learning system using an artificial intelligence avatar tutor, which is executed by performing
The method of claim 1,
Evaluating the user's conversational level using an artificial intelligence neural network algorithm based on the history of the user's conversation with the artificial intelligence avatar comprises:
storing data obtained by mapping element item scores such as fluency, contextual consistency, and accuracy to the conversation data for the specific topic and situation;
creating a conversation level evaluation model with the stored data;
calculating a conversation level score by putting the conversation history of the user with the artificial intelligence avatar into the conversation level evaluation model;
A method of providing a conversational learning system using an artificial intelligence avatar tutor, which is executed by performing
The method of claim 1,
The step of presenting a similar conversation topic and situation in which the user can re-learn a vulnerable topic or expression,
classifying the entire conversation topic and situation based on similarity;
analyzing conversation topics and situations in which the user frequently makes errors or has a low conversation level score;
selecting a conversation topic and situation belonging to a class similar to the conversation topic and situation having the low score;
A method of providing a conversational learning system using an artificial intelligence avatar tutor, which is executed by performing
The method of claim 1,
The step of inserting an advertisement banner in the background of the artificial intelligence avatar according to the conversation topic and situation comprises:
The place where the artificial intelligence avatar of the advertisement banner is displayed is displayed on the screen and is embedded in the place, and
The place where the artificial intelligence avatar is displayed is displayed on the screen, and it is shown floating on the place;
separately showing the place where the AI avatar is located and a space separated from the screen;
What the artificial intelligence avatar is wearing and displayed in the shoes, clothing, etc.
A method of providing a conversational learning system using an artificial intelligence avatar tutor comprising a.

Images