KR20230076733A - English speaking teaching method using interactive artificial intelligence avatar based on emotion and memory, device and system therefor - Google Patents

Abstract

According to one embodiment of the present invention, an emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprises the steps of: receiving a user input including a first English sentence; performing a natural language understanding operation for the first English sentence; determining whether there is a preset intention classification matching a conversation intention included in the first English sentence as a result of performing the natural language understanding operation; generating at least one first natural language based on the first English sentence when there is no preset intent classification matching the conversation intent; calling a template associated with the preset intent classification when there is the preset intent classification matching the conversation intention and generating at least one second natural language based on the called template; extracting an embedding value from the at least one first or second natural language and extracting the first or second natural language based on the extracted embedding value; and outputting the extracted first or second natural language as a second English sentence in response to the first English sentence. The present invention has the effect of improving an English-speaking learning effect by promoting natural and smooth interaction/communication.

Description

English speaking teaching method using interactive artificial intelligence avatar based on emotion and memory, device and system therefor}

The present specification proposes an emotion- and memory-based interactive artificial intelligence avatar English speaking education method, apparatus, and system therefor.

As overseas exchanges become more active due to the development of communication and transportation, the importance of learning English is increasing day by day. In Korea, various opportunities are provided for learners to encounter and learn English by utilizing textbooks and multi-media learning materials, starting from elementary school. In particular, the 2015 revised curriculum emphasizes the areas of speaking and listening, with the goal of developing the ability to understand and express basic English so that English can be used naturally in everyday life from elementary school.

However, in the case of many English learners, the level of the speaking area is low compared to other areas, which is mentioned as a chronic problem in English education in Korea. In the TOEIC test, an English test that mainly deals with listening, reading, and writing, Korea's average score as of 2020 ranks 17th out of 49 countries in the world and 3rd in Asia, ranking among the top in the world. On the other hand, according to ETS's 'Worldwide TOEFL Score Data in 2018', Koreans' TOEFL speaking score was 20 points, tied for 122nd, and ranked at the bottom (Money Today, May 10, 2020).

As a way to overcome these problems, artificial intelligence chatbots are being actively developed to increase opportunities for learners to be exposed to English and to provide interaction-based English education. An artificial intelligence chatbot is a computer program that communicates in a form similar to human conversation through text or voice interaction, and can respond to user's instructions and questions or perform requests. These artificial intelligence chatbots interact with learners through conversation, increase opportunities for exposure to English, and support learning English based on various contents.

Representative domestic and foreign artificial intelligence chatbots include text-based chatbots such as Cleverbot, Replika, and Mitsuku, and voice-based chatbots such as Siri, Echo, Ellie, and PengTalk.

However, in the case of artificial intelligence chatbots developed so far, it is common to provide only short-answer responses during conversation, or to provide factors that reduce learning effects and learners' interest by providing too much information. In addition, a conversation starts only when the learner utters it first, and natural interaction with the learner is often difficult because the conversation context other than the specific situation given like a scenario is not smoothly recognized. In addition, in the case of some educational artificial intelligence chatbots, there is a limitation that it is difficult to actively learn English outside of the time to learn about a specific subject because the contents are limited.

Therefore, in the present specification, it is intended to solve all the problems of existing artificial intelligence chatbots and to provide an artificial intelligence avatar/chatbot engine for efficiently/actively learning to speak English.

A method for teaching English speaking of an interactive artificial intelligence avatar based on emotion and memory according to an embodiment of the present invention, comprising: receiving a user input including a first English sentence; performing a natural language understanding operation on the first English sentence; As a result of performing the natural language understanding operation, determining whether there is a predetermined intention classification matching the conversation intention included in the first English sentence; generating at least one first natural language based on the first English sentence when there is no predetermined intent classification matching the conversation intent; calling a template associated with the preset intent classification and generating at least one second natural language based on the called template when there is the preset intent classification that matches the conversation intent; extracting an embedding value from the at least one first or second natural language, and extracting the first or second natural language based on the extracted embedding value; and outputting the extracted first or second natural language as a second English sentence that is a response to the first English sentence. can include

According to an embodiment of the present invention, since the user/learner's intention is identified as the highest priority and the optimal answer that most closely matches the user/learner's intention is output, natural and smooth interaction/communication proceeds in English It has the effect of improving the speaking learning effect.

In addition, according to one embodiment of the present invention, since a conversation based on emotion and memory is conducted with an artificial intelligence avatar, the learner's tension/stress that may be caused by a compulsory English learning environment is prevented and alleviated in advance, and the same There is an effect that repeated conversation on the subject is prevented so that the user/learner can continue to progress without easily getting tired of or losing interest in the conversation.

In addition, according to an embodiment of the present invention, since the artificial intelligence avatar provides feedback on English speaking sentences in real time, there is an effect that faster and more efficient English speaking learning can proceed.

Other effects of the present invention other than that will be described in detail below with reference to each drawing.

1 is a diagram illustrating an English learning progress screen with an artificial intelligence avatar according to an embodiment of the present invention.
2 is a flowchart illustrating an English speaking education method using an interactive artificial intelligence avatar of a user device according to an embodiment of the present invention.
3 is a diagram illustrating a user device that initiates a conversation by sensing a motion input according to an embodiment of the present invention.
4 is a flowchart illustrating a Rasa open source framework creation method according to an embodiment of the present invention.
5 is a diagram illustrating programming for a policy step according to one embodiment of the present invention.
6 is a diagram illustrating a template according to an embodiment of the present invention.
7 and 8 are diagrams illustrating a first natural language generation method of an educational content model according to an embodiment of the present invention.
9 is a diagram illustrating an English speaking education method through education linkage with a school/academy system according to an embodiment of the present invention.
10 is a diagram illustrating a conversation flow with an artificial intelligence avatar according to an embodiment of the present invention.
11 illustrates an artificial intelligence avatar according to an embodiment of the present invention.
12 is a block diagram of a user device according to an embodiment of the present invention.

Since the technology to be described below can have various changes and various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technology described below to specific embodiments, and it should be understood to include all modifications, equivalents, or substitutes included in the spirit and scope of the technology described below.

Terms such as first, second, A, B, etc. may be used to describe various elements, but the elements are not limited by the above terms, and are merely used to distinguish one element from another. used only as For example, without departing from the scope of the technology described below, a first element may be referred to as a second element, and similarly, the second element may be referred to as a first element. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items. For example, 'A and/or B' may be interpreted as meaning 'at least one of A or B'. Also, '/' can be interpreted as 'and' or 'or'.

In the terms used in this specification, singular expressions should be understood to include plural expressions unless clearly interpreted differently in context, and terms such as “comprising” refer to the described features, numbers, steps, operations, and components. , parts or combinations thereof, but it should be understood that it does not exclude the possibility of the presence or addition of one or more other features or numbers, step-action components, parts or combinations thereof.

Prior to a detailed description of the drawings, it is to be clarified that the classification of components in the present specification is merely a classification for each main function in charge of each component. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each more subdivided function. In addition, each component to be described below may additionally perform some or all of the functions of other components in addition to its main function, and some of the main functions of each component may be performed by other components. Of course, it may be dedicated and performed by .

In addition, in performing a method or method of operation, each process constituting the method may occur in a different order from the specified order unless a specific order is clearly described in context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

A user device to be described below includes an electronic server/s/ It may correspond to a device/device. Therefore, even if not separately described below, the method/embodiment proposed in this specification can be interpreted as being performed/executed through a server/device/device where the application/program/software is running, and each server/device/device The operation of the device may be interpreted as the operation/function of the application/program/software. Hereinafter, for convenience of description, the description will be centered on the 'user device', which is the terminal of the user in which the application/program/software is installed, but the 'user device' will operate/manage/manage the 'application' or the application/program/software. It can be described by being replaced with the 'application server' that controls.

1 is a diagram illustrating an English learning progress screen with an artificial intelligence avatar according to an embodiment of the present invention.

As illustrated in this figure, the user device may output an English speaking conversation interface screen 110 with the artificial intelligence avatar 120. The interface screen 110 may include an artificial intelligence avatar 120 and a chat window 130 . The learner can have a conversation with the artificial intelligence avatar 120 by directly inputting a chat message 150 into the chatting window 130 or by selecting the voice input button 140 to perform voice input. At this time, the learner can directly set the dialogue language to a language the learner wants to learn (eg, English, Korean, etc.).

The conversation between the two is recorded and output through the chat window 130, and the artificial intelligence avatar 120 can proceed with natural conversation / communication with the learner in the target language using a pre-learned AI model, conversation template, etc. there is. In particular, the artificial intelligence avatar 120 proposed in this specification focuses on identifying the learner's conversational intention, and outputs an answer according to the identified intention, thereby enabling natural communication with the learner. In addition, for more natural and smooth communication, the artificial intelligence avatar 120 comprehensively considers the user's emotions, past conversation records and/or conversation situations, etc. to extract candidate answers and selects the most appropriate answer from among the candidate answers. can be printed out.

Through this, natural and smooth communication between the learner and the artificial intelligence avatar 120 proceeds, the learner's tension/stress for learning is alleviated, and the learner's learning ability can be further improved through active conversation.

Hereinafter, a detailed system/device structure and method for implementing the artificial intelligence avatar 120 proposed in this specification will be described.

2 is a flowchart illustrating an English speaking education method using an interactive artificial intelligence avatar of a user device according to an embodiment of the present invention.

In this flowchart, at least one step may be excluded or a new step may be added, and the order of each step may be changed according to embodiments. Furthermore, each step may correspond to a functional configuration of a user device performing the step. For example, the user input step may correspond to a user input unit sensing/sensing/receiving a user input. Each functional configuration may be implemented using at least one hardware/software configuration.

Referring to FIG. 2 , first, the user device may receive a user input including a first English sentence (S201). User input refers to various inputs to the user device, and may correspond to, for example, text input and voice input. The user device normally maintains a standby/power-save mode, and may start a natural English conversation for learning when a user input is detected. '

According to an embodiment, the user device may initiate a direct conversation when a preset input other than a user input including the first English sentence is detected, which will be described later with reference to FIG. 3 .

3 is a diagram illustrating a user device that initiates a conversation by sensing a motion input according to an embodiment of the present invention.

Referring to FIG. 3, the user device includes at least one camera sensor, and recognizes/identifies a visual object (eg, face, face, movement, facial expression, etc.) 330 of the learner 320 using the camera sensor. can do. In particular, when a motion is detected using a camera sensor, the user device may determine whether the user device is a learner 320 registered in the learning system in advance by recognizing/identifying the visual object 330 for the motion-detected object.

If the detected target is the learner 320 registered in the system, the user device comprehensively considers the current time, learner 320's facial expression, learning subject, and/or past conversation records, etc. to initiate a conversation in English. A conversation with the learner 320 may be actively initiated by determining a sentence and outputting the determined English sentence 310 . The learner may perform an answer to the opening sentence 310 of the user device, and the corresponding answer may be recognized/detected and processed by the user device as in step S201 as a user input including the first English sentence. .

Referring back to FIG. 2 , next, the user device may perform a natural language understanding operation on the first English sentence (S202). Here, the natural language understanding operation (S202) may mean a series of operations for the user device to understand the meaning, intention, emotion, etc. contained in the input first English sentence.

These natural language understanding operations (S202) include a preprocessing step (not shown) of the first English sentence, a step of classifying the emotion category of the preprocessed first English sentence (S202c), and a conversation intent and object name from the preprocessed first English sentence. It may be configured to include the step of extracting (S202a, S202b).

The step of preprocessing the first English sentence may refer to a series of processes of preprocessing the first English sentence in order to improve a natural language understanding recognition rate for the first English sentence. The user device may perform a tokenizer ('WhitespaceTokenizer') based on whitespace characters included in the first English sentence, and words unnecessary to natural language understanding (eg, articles such as a, an, the, etc.), spaces, etc. Preprocessing operation can be performed by filtering ('StopwordFilter'). Here, the tokenizer can perform a function of splitting meaningful elements into tokens in a specific sentence/syntax.

In the step of classifying the emotion category (S202c), the preprocessed first English sentence is input to an emotion classification model built in advance through a preset learning model, and the emotion contained in the first English sentence is classified into a preset emotion category. step may be applicable. Various AI learning models can be used in constructing an emotion classification model, but the ELECTRA model is proposed in this specification. The ELECTRA model is a learning model that proposes a language pre-training method that can efficiently learn while maintaining the advantages of BERT, and is a model that is mainly used to recognize emotions included in sentences. The preset emotion category may be defined as a category preset by an operator or manager, and may be defined as various emotions, such as, for example, "happy", "sadness", "anger", and the like. there is.

In the step of extracting the conversation intention and entity name (S202a, S202b), the preprocessed first English sentence is input to a conversation intention extraction model built in advance through a preset learning model, and the conversation intention and It may correspond to the step of extracting the entity name. Various AI learning models can be used to build conversational intent and object name extraction models, but in this specification, a Dual Intent and Entity Transformer (DIET) classifier model is proposed. The DIET classifier model corresponds to a transformer architecture that can handle both intent classification and object recognition.

Conversation intent can be extracted into various categories, such as “chithat (daily life conversation)”, “greeting (greeting)”, etc. Entity names, for example, “user_name (learner name)”, “user_persona (artificial greeting)” Intelligence avatar name)" may correspond to a name, title, personal pronoun, and/or referential pronoun.

Next, the user device may determine whether there is a predetermined intention classification that matches the conversation intention extracted from the first English sentence (S203). If there is a predetermined intent classification that matches the extracted conversation intent, the user device may perform a template-based natural language generation operation (S204 to S206). Conversely, when there is no preset intent classification that matches the extracted conversation intent, the user device may perform a DL-based natural language generation operation ( S207 ).

First, the case where there is a preset intent classification that matches the extracted conversation intention will be described, and then the case where there is no preset intent classification will be described.

If there is a preset intent classification that matches the extracted conversation intent, the user device may perform an operation of searching for and calling a template associated with the preprocessed first English sentence in order to generate a template-based second natural language (S204).

As an operation for this, in this specification, a method using the Rasa open source framework is proposed. The Rasa open source framework is an open source machine learning framework that automates text- and voice-based conversations.

4 is a flowchart illustrating a Rasa open source framework creation method according to an embodiment of the present invention, and FIG. 5 is a diagram illustrating programming for a policy step according to an embodiment of the present invention.

Referring to Figures 3 and 4, the Rasa open source framework largely includes a tracking step (S204a, S420) and a policy step (S204b, S430).

The tracking step (S420) corresponds to a step of storing information necessary for conducting a conversation. In this step (S204a, S420), the user device extracts the input text, time, intent classification result, entity name extraction result, and/or emotion classification result from the preprocessed first English sentence, and extracts it in a format defined in the Rasa open source framework. You can create and save conversation scenarios according to An example of such a dialogue scenario is shown in Table 1 below.

The policy step (S204b, S430) corresponds to a step of determining what action to take for the intention classification result extracted in the previous tracking step. As illustrated in FIG. 5 , the user device may have actions defined in advance for each intention classification result, and may perform the defined action according to the recognized intention classification result. For example, referring to FIG. 5 , when the user device classifies the conversational intention of the first English sentence as 'greet', the user device takes an action defined in 'action_chitchat_generator' (eg, 'Hi!! How voice and chatting output such as 'are you today?') can be performed.

Policy steps (S204b, S430) may be performed only when there is no template matching the dialogue scenario of the first English sentence as a result of performing step S205, which will be described below.

Referring back to FIG. 2 , the user device may next determine whether a template matching/corresponding to a conversation scenario created based on the Rasa open source framework exists (S205).

If there is no preset conversation template that matches the conversation scenario, as described above with reference to FIG. 5 , the user device may take actions defined in the policy step (S430). Conversely, if there is a preset conversation template that matches the conversation scenario, the user device may call the matching template and take an action according to what is defined in the called template (S206).

6 is a diagram illustrating a template according to an embodiment of the present invention.

In particular, this drawing corresponds to a template for the 'morning' scenario. Referring to FIG. 6 , in the template, actions (or responses) to be taken for each predetermined intent category may be defined in a one-to-n correspondence relationship. The user device may select (S206b) at least one action (or response) corresponding to a preset intent classification that matches the first English sentence and generate it as a second natural language (S206a). For example, when the user device extracts the 'morning' intention classification from the first English sentence, it can select 'utter_resp_morning', which is an action defined in response to this (S206b), and as a result, 'Good morning to you, too! Did you sleep well, {name}?" can be generated as a second natural language (S206a).

In this way, when a conversation is conducted using a preset template, there is an effect that natural communication is possible in a state in which the current situation and the learner's intention are more accurately grasped. Furthermore, it is possible to freely configure templates according to learning purposes, educational engineering purposes, etc., thereby improving the overall degree of freedom and flexibility of the system. has the effect of overcoming

In the above, the case where there is a preset intention classification that matches the conversation intention in step S203 has been reviewed. Hereinafter, a case where there is no preset intent classification matching the conversation intent in step S203 will be described.

Referring back to FIG. 2 , when there is no preset intent classification that matches the conversation intent extracted in step S202, the user device may generate at least one first natural language spontaneously based on the first English sentence (S207). . This can be done based on various AI learning models.

As an embodiment, the user device may generate a first natural language by making an API call to the davinci model of Open AI GPT3 (S207b) and inputting a first English sentence (in particular, a preprocessed first English sentence) to the davinci model. (S207a).

As another embodiment, the user device may generate a first natural language that is a response to a user input by utilizing an empathic conversation model built by pre-learning an empathic conversation method meaningful in terms of educational engineering (S207c). In constructing this empathic conversation model, the Electra model, which is a learning model for grasping the learner's emotion, can be actively utilized, and through this, it can be learned to grasp the learner's emotion and respond empathetically to it.

As another embodiment, the user device may generate a first natural language by inputting a first English sentence to the Blender Bot model (S207d). Here, the Blender Bot model is a chatbot model developed by the Facebook team that can generate Internet search queries and has the ability to have conversations by referring to previous knowledge as knowledge is added over time. It is characterized by generating a response in which the persona is reflected in response to the user input.

As another embodiment, the user device may generate a first natural language based on a knowledge base (or based on a web search) (S207e). More specifically, the user device may extract a keyword from the first English sentence, search the extracted keyword on the web, and generate a first natural language based on the search result.

As another embodiment, the user device may generate a first natural language based on a past conversation record (S207f). More specifically, the user device may group conversations between learners and artificial intelligence avatars into one set, learn them, and store them in a database, and generate a first natural language based on a record of learned conversations based on long-term memory based conversations. model can be built. The user device may generate the first natural language based on the output response by inputting the first English sentence to the long-term memory-based conversation model built in this manner. The long-term memory conversation model extracts keywords from the first English sentence, searches the learned contents or database, and outputs the search results.

As another embodiment, the user device may receive a pre-learned and built educational content model and generate a first natural language based on the model (S207g). This will be described in detail below with reference to FIGS. 7 and 8 .

7 and 8 are diagrams illustrating a first natural language generation method of an educational content model according to an embodiment of the present invention.

An English speaking educational content producer may build an educational content model in which educational contents are learned according to an English educational topic, and may input the educational content model thus built into a user device. For example, as illustrated in FIG. 7 , when the education topic is 'tense', the English speaking education content creator learns a list of questions leading to answers to English sentences related to 'tense' and an appropriate response to the AI learning model. to build an educational content model.

The educational content model built in this way may be input/received by a user device, and the user device may generate at least one question (eg, ask_past in FIG. 7 ) predefined in the educational content model as a first natural language. 8 can be output (810-1, 810-4). Furthermore, the educational content model may receive a user input 810 - 2 that is a response to the output question, and extract a third English sentence from it. The educational content model may evaluate the degree of completeness of the third English sentence based on the degree of matching with pre-learned responses in response to the questions 810-1 and 4. In particular, the educational content model may evaluate the sentence completeness of the third English sentence 810-2 by assigning a weight to whether a condition corresponding to the educational content subject is satisfied. For example, the completeness of a sentence with an error except for the tense may be evaluated higher than the completeness of the sentence than a sentence with an error only in the tense.

When an error is found in a part related to a subject of educational content, the educational content model generates natural language (natural language for correcting tense in this figure) to correct it, as illustrated in FIG. Yes (810-3).

The training content model may continuously/sequentially generate questions defined therein in the third natural language until the evaluated sentence completeness becomes equal to or higher than the preset sentence completeness, and the fourth English sentence including the third natural language (810- 4) can be continuously output as a response to the third English sentence 810-2.

Referring back to FIG. 2 , next, the user device may rank at least one first and/or second natural language generated in advance (S208). More specifically, the user device may extract an embedding value from at least one generated first and/or second natural language, and extract the first or second natural language having the highest embedding value. Here, embedding means a result or a series of processes in the field of natural language processing that converts natural language into a vector in the form of numbers that can be understood by a machine. The user device may input an embedding value extracted from at least one first and/or second natural language into a BERT-based ranker, and the ranker may return a list in which predicted values for each embedding value are sorted in descending order (S208a). The user device may select the first or second natural language ranked at the top in the returned list.

Next, the user device may detect danger of the selected first or second natural language (S209). More specifically, the user device may classify toxicity by inputting the selected first or second natural language into the GPT3 content filter (S209a). When the input first or second natural language is classified as toxic, the first or second natural language ranked next may be selected from the returned list in step S208. If toxicity classification is performed in units of morphemes/words, the user device paraphrases morphemes/words classified as toxic in the first or second natural language to other morphemes/words not classified as toxic, and 2 Natural language conversion can be performed (S209b).

Finally, the user device may output/response the finally converted/output first or second natural language as a second English sentence that is a response to the first English sentence (S210). The second English sentence may be output as voice and/or chatting of the artificial intelligence avatar through the interface screen as illustrated in FIG. 1 .

Step S209 may be performed or excluded according to embodiments. In this case, the first or second natural language selected in step S208 may be output as a second English sentence.

9 is a diagram illustrating an English speaking education method through education linkage with a school/academy system according to an embodiment of the present invention.

The English speaking education method/system according to an embodiment of the present invention can provide a higher level of education by linking/linking with a learner's school and/or academy system 910 (or a learning leader's user device).

More specifically, through the school/academy system 910, a learning leader may create a template or learning model related to learning content to be taught and transmit it to the user device. A template may be used to generate a template-based first natural language in step S206 of FIG. 2 , and a learning model may be used to generate a second natural language in step S207 .

A learning leader may create a template by creating learning content to be taught according to a preset format. For example, a learning leader can input learning content in accordance with the Rasa open source framework format, and the learning leader's user device can convert it into a template format and transmit it to the learner's user device.

In addition, the learning leader can create a learning model by learning the learning content to be taught to the AI model. At this time, the learning leader can create a learning model by learning the AI model in a language (eg, English, Korean, etc.) in which the learner wants to learn the learning content. The model created in this way can be transmitted to the learner's user device.

In this way, based on the template and/or learning model received from the school/academy system 910, the artificial intelligence avatar 920 can naturally communicate with the learner in the learning language, and through this, not only learning to speak, but also learning about other subjects. There is an effect that learning can be performed simultaneously.

10 is a diagram illustrating a conversation flow with an artificial intelligence avatar according to an embodiment of the present invention.

In particular, this figure shows a conversation flow between an artificial intelligence avatar actually implemented according to the present invention and a learner. As shown in this figure, the artificial intelligence avatar can have a natural conversation by accurately recognizing the learner's speech/conversation intention and situation.

11 illustrates an artificial intelligence avatar according to an embodiment of the present invention.

As illustrated in this figure, it is possible to freely set artificial intelligence avatars for each learning subject, and it is possible to directly set specific characteristics (eg, name, age, place of birth, personality, pronunciation, etc.) of each artificial intelligence avatar. The user device can proceed with a conversation by activating the artificial intelligence avatar set for each recognized learner, and can directly reflect and output characteristics set for the artificial intelligence avatar to the conversation.

For example, an artificial intelligence avatar feature-based conversation conversion step may be added to the last step of FIG. 2, and the user device converts the finally generated second English sentence based on the artificial intelligence avatar feature (eg, words, Tone, tempo, etc.) can be finalized.

12 is a block diagram of a user device according to an embodiment of the present invention.

Referring to FIG. 12 , a user device 1200 may include a control unit 1210, a memory unit 1220, a communication unit 1230, and/or a user input/output unit 1240.

The controller 1210 may control at least one component/unit to perform at least one embodiment proposed in this specification. The control unit 1210 may be a Central Processing Unit (CPU), Micro Processor Unit (MPU), Micro Controller Unit (MCU), Application Processor (AP), Application Processor (AP), or any form well known in the art. It may be configured to include at least one processor. The controller 1210 may perform calculations for at least one application or program for executing a method according to embodiments of the present invention.

The memory unit 1220 may store various digital data such as video, audio, photos, moving pictures, and applications. The memory unit 1210 represents various digital data storage spaces such as a flash memory, a hard disk drive (HDD), and a solid state drive (SSD).

The communication unit 1230 may transmit/receive data by performing communication using at least one wired/wireless communication protocol. The communication unit 1230 may transmit/receive digital data by accessing an external network by wire or wirelessly.

The user input/output unit 1240 senses a user's input using at least one sensor, and uses at least one output means (eg, a display, a speaker, etc.) to visual/audible screen/effect/ Feedback can be output. The at least one sensor may include a gravity sensor, a geomagnetic sensor, a motion sensor, a gyroscope sensor, an acceleration sensor, an infrared sensor, an inclination sensor, a brightness sensor, an altitude sensor, an olfactory sensor, a temperature sensor, a depth sensor, and a pressure sensor. It may include at least one of various sensing means such as a sensor, a bending sensor, an audio sensor, a video sensor, a Global Positioning System (GPS) sensor, a touch sensor, and a grip sensor.

An embodiment according to the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one embodiment of the present invention provides one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs ( field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is stored on a recording medium readable through various computer means. can be recorded. Here, the recording medium may include program commands, data files, data structures, etc. alone or in combination. Program instructions recorded on the recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs (Compact Disk Read Only Memory) and DVDs (Digital Video Disks), floptical It includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, such as a floptical disk, and ROM, RAM, flash memory, and the like. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. These hardware devices may be configured to act as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, an apparatus or terminal according to the present invention may be driven by a command that causes one or more processors to perform the functions and processes described above. For example, such instructions may include interpreted instructions, such as script instructions such as JavaScript or ECMAScript instructions, or executable code or other instructions stored on a computer readable medium. Furthermore, the device according to the present invention may be implemented in a distributed manner over a network, such as a server farm, or may be implemented in a single computer device.

In addition, a computer program (also known as a program, software, software application, script or code) loaded into a device according to the present invention and executing the method according to the present invention includes a compiled or interpreted language or a priori or procedural language. It can be written in any form of programming language, and can be deployed in any form, including stand-alone programs, modules, components, subroutines, or other units suitable for use in a computer environment. Computer programs do not necessarily correspond to files in a file system. A program may be in a single file provided to the requested program, or in multiple interacting files (e.g., one or more modules, subprograms, or files that store portions of code), or parts of files that hold other programs or data. (eg, one or more scripts stored within a markup language document). A computer program may be deployed to be executed on a single computer or multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

For convenience of description, each drawing has been divided and described, but it is also possible to design to implement a new embodiment by merging the embodiments described in each drawing. In addition, the present invention is not limited to the configuration and method of the described embodiments as described above, but the above-described embodiments are configured by selectively combining all or part of each embodiment so that various modifications can be made. It could be.

In addition, although preferred embodiments have been shown and described above, this specification is not limited to the specific embodiments described above, and those skilled in the art in the art to which the specification pertains without departing from the subject matter claimed in the claims Of course, various modifications are possible by the person, and these modifications should not be individually understood from the technical spirit or perspective of the present specification.

Claims (13)

Emotion and memory-based interactive artificial intelligence avatar English speaking education method,
Receiving a user input including a first English sentence;
performing a natural language understanding operation on the first English sentence;
As a result of performing the natural language understanding operation, determining whether there is a predetermined intention classification matching the conversation intention included in the first English sentence;
generating at least one first natural language based on the first English sentence when there is no predetermined intent classification matching the conversation intent;
calling a template associated with the preset intent classification and generating at least one second natural language based on the called template when there is the preset intent classification that matches the conversation intent;
extracting an embedding value from the at least one first or second natural language, and extracting the first or second natural language based on the extracted embedding value; and
outputting the extracted first or second natural language as a second English sentence that is a response to the first English sentence; Including,
Generating at least one first natural language based on the first English sentence when there is no preset intent classification matching the conversation intent,
receiving an educational content model; and
Generating a plurality of questions related to learning objects predefined in the educational content model as the second English sentence and the fifth English sentence, respectively, based on the first natural language,
The step of outputting as the second English sentence,
receiving a user input including a third English sentence as a response to the second English sentence;
inputting the third English sentence into the educational content model;
Evaluating sentence completeness of the third English sentence output from the educational content model;
If the evaluated sentence includes an error according to the learning target criterion and the completeness level is less than the preset sentence completeness, immediately generating another question in which the error is corrected, which is predefined in the educational content model, as the third natural language ;
outputting the third natural language as a fourth English sentence sequentially succeeding the second English sentence and the third English sentence in response to the third English sentence; and
After the fourth English sentence is output, outputting the generated fifth English sentence,
Storing the first and second English sentences as a set in a conversation database; further comprising,
Generating at least one first natural language based on the first English sentence when there is no preset intent classification matching the conversation intent,
extracting keywords from the first English sentence;
searching for the keyword on the database;
generating the first natural language based on a search result; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 1,
Performing a natural language understanding operation for the first English sentence,
performing pre-processing on the first English sentence;
Classifying emotion categories of the preprocessed first English sentences; and
extracting a conversation intention and object name from the preprocessed first English sentence; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 2,
The step of performing preprocessing on the first English sentence,
performing a tokenizer based on blank characters included in the first English sentence; and
removing articles included in the first English sentence; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 2,
Classifying the emotion category,
inputting the preprocessed first English sentence to an emotion classification model pre-constructed through ELECTRA, and classifying the emotion contained in the first English sentence into a preset emotion category; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 2,
The step of extracting the conversation intention and object name,
inputting the pre-processed first English sentence to a dialogue intention and entity name extraction model built in advance through a DIET classifier, and extracting a conversation intention and entity name contained in the first English sentence; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 5,
The object name,
Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method including at least one of a name, title, personal pronoun and referential pronoun included in the preprocessed first English sentence. According to claim 2,
The step of calling a template associated with the preset intent classification,
generating a conversation scenario according to a preset framework format by tracking at least one of text, time, the emotional category, the conversation intention, and the object name from the preprocessed first English sentence;
performing an action according to a preset behavioral policy corresponding to the preset intent classification when there is no template matching the created dialogue scenario; and
calling the template when there is the template that matches the created dialogue scenario; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 7,
The preset framework is a Rasa Open source frame work, an interactive artificial intelligence avatar English speaking education method based on emotion and memory. According to claim 8,
The template is a template in which responses for each predetermined intent classification are defined in a correspondence relationship of 1 to N (N is a natural number), an emotion- and memory-based interactive artificial intelligence avatar English speaking education method. According to claim 9,
Generating at least one second natural language based on the called template,
generating at least one response corresponding to a preset intent classification of the first English sentence as the second natural language within the called template; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 1,
Generating at least one first natural language based on the first English sentence when there is no preset intent classification matching the conversation intent,
generating the first natural language by calling a davinci model of Open AI GPT3 through an application programming interface (API) and inputting the first English sentence into the davinci model; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 1,
Generating at least one first natural language based on the first English sentence when there is no preset intent classification matching the conversation intent,
generating the first natural language by inputting the first English sentence to a Blender Bot model; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a. According to claim 1,
Generating at least one first natural language based on the first English sentence when there is no preset intent classification matching the conversation intent,
extracting keywords from the first English sentence;
Searching for the keyword on the web; and
generating the first natural language based on a search result; Emotion and memory-based interactive artificial intelligence avatar English speaking teaching method comprising a.

Images