KR20230043558A - Dialogue system and its method for learning to speak foreign language - Google Patents

Abstract

In a foreign language learning domain where the amount of dialogue corpus is small, the present invention efficiently improves the dialogue performance of a system by accurately identifying whether a learner is uttering in accordance with a topic even for examples that do not exist in the dialogue corpus through a mission item candidate selection module. In addition, the present invention can process even unpredictable free dialogues by processing free dialogues that may occur during target dialogues by using a free dialogue tracking technology in a dialogue state prediction step.

Description

Dialogue system and its method for learning to speak foreign language}

The present invention relates to a conversation apparatus and method for learning to speak a foreign language, and more particularly, to a conversation system for learning a foreign language for learners who want to learn sentence patterns in a foreign language through dialogue with the system.

In a general goal-directed conversation system, in order to identify user requirements, a step of extracting information corresponding to the user's purpose from the user's utterance and conversation history is required. In order to do this accurately, a learning corpus including various examples is essential. .

For example, for the subject of 'favorite hobby', if the system learns or prepares "I enjoy [$hobby]" as the learner's expected utterance, the learner utters "I enjoy [drawing]", and drawing If it does not exist as an example of $hobby in the rule, it may not be recognized as information about 'favorite hobby' even though the learner utters the correct sentence.

As another example, for the subjects 'favorite sport' and 'favorite food', the learner expresses $sport and $food in one sentence without uttering the two subjects as separate sentences, saying "I enjoy [skiing] and eating" [hamburger]", the system may not judge the expression of the sentence as the correct expression because the learner uttered correctly on two topics, but there is no example of uttering both topics at the same time in the conversation corpus. there is.

In a conversation system for foreign language learning, it would be more effective to express sentences suitable for oneself in various forms than to express given example sentences as they are. Therefore, even for examples of sentence patterns not prepared by learning corpus or rules, that is, words, phrases, or similar sentence patterns, the system must be able to determine whether the sentence is a correct expression according to the learner's situation.

In addition, in the foreign language learning conversation system, a case in which a learner utters a non-purpose conversation, that is, a free conversation, that has nothing to do with the learning topic may occur during a conversation.

For example, a learner may lose concentration or lose interest while exchanging learning conversations with the system, and may try free conversations such as "It's not fun" or "How is the weather today?" This is a natural phenomenon that can occur frequently in an actual learning environment. In order to maximize the learning effect, the conversation system should be able to handle even unpredictable free conversation.

However, in the conventional goal-oriented conversation system, there is a limitation in not processing free conversation that may occur during the learner's task performance.

In order to solve the above problems, the present invention accurately recognizes the speech expression of the learner even for various examples not in the learning corpus, and determines whether the learner is correctly performing the expression to be learned or uttering free conversation, It is an object of the present invention to provide a conversation apparatus and method for learning to speak a foreign language capable of performing a response and a conversation accordingly.

The foregoing and other objects, advantages and characteristics of the present invention, and methods of achieving them will become clear with reference to the embodiments described below in detail in conjunction with the accompanying drawings.

In order to achieve the above object, a method for learning to speak a foreign language performed by a conversation device includes a voice recognition module using a learner's voice uttered for any one mission item among a plurality of mission items given by the conversation device. recognizing and generating text corresponding to the learner's voice; selecting, by a mission item candidate selection module, a mission item candidate having the highest similarity with the text from among the plurality of mission items; The dialog state prediction module uses the text, the mission item candidate, mission items other than the mission item candidate among the plurality of mission items, and other items unrelated to the plurality of mission items to determine the slot and predicting a conversation state including a slot value connected to the slot; and outputting, by a system speech generation module, a system speech corresponding to 'mission dialogue' or 'free dialogue' based on the predicted dialogue state. includes

According to the present invention, in a foreign language learning domain with a small amount of dialogue corpus, the dialogue performance of the system is improved by accurately determining whether the learner is uttering in accordance with the topic even for examples that do not exist in the dialogue corpus through the mission item candidate selection module. can be improved efficiently.

In addition, by using the free conversation tracing technology in the conversation state prediction step, free conversation that may occur during the target conversation is also processed, so that even unpredictable free conversation can be processed.

1 is an overall configuration diagram schematically showing the internal configuration of a conversation apparatus for learning to speak a foreign language according to an embodiment of the present invention.
2 is a diagram for explaining an example of a mission item and a dialogue corpus shown in FIG. 1;
FIG. 3 is a diagram for explaining a process of the mission item candidate selection module shown in FIG. 1 .
4 is a detailed configuration diagram of the dialog state prediction module shown in FIG. 1;
5 is a detailed configuration diagram of the system utterance generation module shown in FIG. 1;
FIG. 6 is a table for explaining output examples of the slot type extension unit and slot value prediction unit shown in FIG. 4 and output examples of the dialogue type recognition unit and system speech generation unit shown in FIG. 5 .
7 is a block diagram of a conversation apparatus for learning to speak a foreign language using a language model according to another embodiment of the present invention.
8 is a flowchart illustrating a method for learning to speak a foreign language performed by a conversation apparatus for learning to speak a foreign language according to an embodiment of the present invention.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are only illustrated for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention It can be implemented in various forms and is not limited to the embodiments described herein.

Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

The conversation system for learning a foreign language according to the present invention provides learners with three or four topics, asks questions to utter sentences suitable for the topics, and performs a conversation in which the learners respond.

Although not particularly limited, the present invention may have a structure similar to a destination-oriented conversation system in which reservations are made based on system questions and user responses in domains such as hotels and taxis.

Hereinafter, with reference to the drawings, an embodiment of the present invention will be described in detail.

1 is an overall configuration diagram schematically showing the internal configuration of a conversation apparatus for learning to speak a foreign language according to an embodiment of the present invention.

Referring to FIG. 1 , in the conversation apparatus for learning to speak a foreign language according to an embodiment of the present invention, it is assumed that the foreign language learning domain is English.

The dialogue device can be divided into a learning phase and an execution phase. In the learning step, parameters of the system module are updated and learned through the dialogue corpus 107. In the execution stage, a dialogue in the form of question-response is performed between the actual learner using the learned module and the system.

The example below shows the mission item 100 and the dialogue corpus 107, which are the components of FIG. 1, as an example.

In the execution step, the learner 101 receives a mission item 100 representing a subject or learning expression to be uttered in a conversation by the learner in a corresponding language expression from the conversation device.

The learner receiving the mission item utters a sentence suitable for the given mission item, and at this time, the learner's utterance can be input in the form of voice or text.

If the learner's utterance is voice, the voice recognition module 102 converts the learner's 101's voice into text, and if the learner's 101's utterance is in text form, the voice recognition module 102 converts the learner's 101's voice into text. The text corresponding to the utterance is transferred to the mission item candidate selection module 103 as it is.

The mission item candidate selection module 103 is a module that determines whether the utterance of the learner 101 corresponds to a mission item, and if so, which mission item it corresponds to, the text corresponding to the learner's utterance. is used as an input to select and output the mission item candidate most similar to the spoken sentence.

The dialogue state prediction module 104 receives the mission item candidates and the learner's speech transmitted from the mission item candidate selection module 103 and predicts a dialogue state having information on mission items up to the current point in time.

The conversation state is to manage whether the learner correctly utters the mission item according to the learning intention, and each mission item is displayed in the form of a slot at each turn of the conversation to predict the value of each slot.

The system speech generating module 105 receives the dialogue state output from the dialogue state prediction module 104 and the learner's speech as inputs and outputs the next system speech that matches the learner's speech. The output system utterance is delivered to the learner 101 as it is in text form, or is input to the voice output module 106 and converted into a voice form and delivered to the learner.

In the learning phase, as shown in the example below, the dialog corpus 107 composed of the mission item, the learner-system dialog, and the dialog state of each turn is input and the dialog state prediction module 104 and the system utterance generation module 105 learn. do.

The loss calculation unit 108 calculates a loss value (Loss) representing the difference between the result generated from the two modules 104 and 105 and the correct answer of the actual dialogue corpus 107, and modifies and stores the value forwarded to (109). Here, the correct answers of the dialog corpus 107 may be, for example, system utterances and dialog states indicated by reference numerals '107A', '107B', '107C', and '107D' in FIG. 2 , respectively.

The parameter correction and storage unit 109 updates the parameters of each module in a direction that minimizes the loss value (Loss) transmitted from the loss calculation unit 108 and stores them.

The stored parameters are utilized in the dialogue state prediction module 104 and the system speech generation module 105 in the execution phase. However, the mission item selection module 103 does not need to proceed with learning when pre-learned embedding is used, and when the similarity determination of words or sentences of the mission item selection module 103 is performed by learning, a large-capacity language such as BERT Based on the model, binary classification learning is performed to determine whether it is true or false about the similarity between the expression of the mission item and the word, phrase, or sentence.

When mission item candidates are selected with pre-learned embedding information in the mission item candidate selection module 103, learner utterances (101, 102) and mission items (100) in the form of text are converted into Word2Vec, Sent2Vec, FastText, Bert (Bidirectional Encoder) By using embedding models such as Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT), embedding is performed in the form of vectors that reflect semantic information to determine cosine similarity between embedding vectors, Euclidian distance, etc. If the similarity of is greater than a specific threshold (eg, 0.5), the most suitable mission item candidate with the learner's uttered sentence (text corresponding to the learner's voice) is selected and transmitted to the dialogue state prediction module 104. If there is no more than a specific threshold value, another item (eg, other) for processing free conversation is selected and transmitted to the conversation state prediction module 104 .

2 is a diagram for explaining an example of a mission item and a dialogue corpus shown in FIG. 1 , and FIG. 3 is a diagram for explaining a process of a mission item candidate selection module shown in FIG. 1 .

Referring to FIGS. 2 and 3, the mission items presented in the embodiment of the present invention are favorite hobby, favorite sport, and favorite food, and a mission item most closely related to the learner's utterance of each turn is selected, and the selected candidate is selected. forward to the next step

Among the table items shown in FIG. 3, the item “(1) Sentence and mission item embedding” is an embedding of the learner's utterance and each mission item in a vector form.

The item "(2) similarity" indicates the similarity result between each mission item for the learner's utterance.

Items "(3) Mission item candidate determination" and "(4) Mission item candidate selection" indicate the final mission item candidate selected based on the result after determining whether the similarity is greater than or equal to a threshold value.

In the case of Turn 1, for the utterance "I enjoy drawing", a favorite hobby having a similarity higher than a threshold value (0.5) is selected as a mission item candidate. In this way, in the mission item candidate selection module 103, even if there is no example of drawing in the conversation corpus, it is possible to predict whether a corresponding utterance corresponds to a favorite hobby mission item through an embedding similarity measurement method.

In the case of Turn 2, other items are selected because there is no mission item whose similarity is higher than the threshold value (0.5) for the utterance This is boring.

In the case of Turn 3, favorite sports and favorite hobbies with a similarity greater than a threshold value (0.5) are selected as mission item candidates. Even if there is no example of uttering two mission items at the same time, it can be confirmed that the two mission items are uttered by the learner through the selected mission item candidates.

When the mission item candidate selection module 103 proceeds with learning to determine the similarity of words or sentences, the text-type learner utterances 101 and 102 and each mission item 100 are used as inputs to determine each similarity (true or false), a true mission item is selected as a mission item candidate and transmitted to the dialog state prediction module 104. If the mission item candidate is empty, another item (eg, other) is selected and transmitted to the dialog state prediction module 104 .

4 is a detailed configuration diagram of the dialog state prediction module shown in FIG. 1;

Referring to FIG. 4 , the conversation state prediction module 104 extracts information of a predefined slot type from a corresponding utterance every turn in order to continue the next conversation based on the conversation history between the system and the learner.

In this dialog system, slot types correspond to mission items such as favorite hobby, favorite sport, and favorite food, and slot values represent values corresponding to food slots such as hotdog, pizza, and hamburger.

To track free conversations in conversation state prediction, the slot type extension 104A includes other slots to existing slot types. For example, the other slot type other is added to the existing slot type (favorite hobby, favorite sport, favorite food) to complete the extended slot type (favorite hobby, favorite sport, favorite food, other).

The slot value prediction unit 104B synthesizes the expanded slot type, mission item candidates received from the mission item candidate selection module 103, learner utterances 101, 102, etc., and converts them into a slot-value form. The constructed dialogue state at the present time is predicted and transmitted to the system speech generation module 105. In this case, as a method of utilizing the mission item candidate in the slot value predictor 104B, the learner utterance may be encoded by simply concatenating the learner utterance or by applying attention to assign a weight.

5 is a detailed configuration diagram of the system utterance generation module shown in FIG. 1;

Referring to FIG. 5 , in the system speech generation module 105, the learner utters a sentence including a word corresponding to a mission item from the conversation type recognition unit 105A according to the conversation state received from the conversation state prediction module 104. Then, it determines whether it is a 'mission dialogue', which is a dialogue form in which the system responds, or a 'free dialogue', which is a dialogue form in which a learner and the system exchange utterances unrelated to mission items, and reflects this in the system speech generation unit 105B. An appropriate response is generated for the received learner utterances (101, 102). The generated response is output as a system utterance in the form of text and delivered to the next steps 101 and 106.

FIG. 6 is a table for explaining output examples of the slot type extension unit and slot value prediction unit shown in FIG. 4 and output examples of the dialogue type recognition unit and system speech generation unit shown in FIG. 5 .

Referring to FIG. 6, the item '(5) Extended slot type' is an output of the slot type extension unit 104A, and indicates a slot type extended to include '[other slots]'.

In the item '(6) Predicted dialogue state (slot-slot value)', it can be seen that slot values for other slot types (other) are predicted as the dialogue state of the current turn predicted from the slot value prediction unit 104B. .

The item '(7) Conversation type' is an output of the conversation type recognizing unit 105A, and the conversation type of the current turn is predicted from the updated '(6) predicted conversation state (slot-slot value)'. Based on this, the system speech generator 105B outputs '(8) system speech', which is a response to the learner's speech. In the case of the system utterance generated in Turn 1, a response to the learner's appropriate utterance for his/her hobby and a response to move on to the next mission item are presented.

In the case of the system utterance of Turn 2, a response that allows the learner to focus on learning again and a response asking about the mission item are generated for the learner utterance not related to the mission item (favorite sport).

In the case of Turn 3, the system ends the conversation with a closing salutation as the learner uttered appropriately for the remaining two mission items (favorite sport and favorite food).

7 is a block diagram of a conversation apparatus for learning to speak a foreign language using a language model according to another embodiment of the present invention.

Referring to FIG. 7 , a conversation apparatus according to another embodiment of the present invention utilizes a pre-learned language model 200 in which a dialogue state prediction module 104 and a system utterance generation module 105 are integrated, and the dialogue state and system firing at once.

The pre-learned language model 200 sequentially inputs the mission item, the learner's speech, and the mission item candidates outputted through the mission item candidate selection module 103 to the system, and then outputs a conversation state and system speech.

8 is a flowchart illustrating a method for learning to speak a foreign language performed by a conversation apparatus for learning to speak a foreign language according to an embodiment of the present invention.

The performers of each step performed below can be clarified from the description of FIGS. 1 to 6 . However, the configurations shown in FIG. 1 may be implemented as a computing device including a processor, a memory, an input/output device, a storage medium, a memory, and a system bus connecting them. In this case, the performer of each step performed below may be a processor.

Referring to FIG. 8, first, the voice recognition module 102 recognizes a learner's voice uttered for any one mission item among a plurality of mission items given by the conversation device, and then text corresponding to the learner's voice. A step of generating is performed (810).

Subsequently, a step of selecting a mission item candidate having the highest similarity with the text among the plurality of mission items is performed by the mission item candidate selection module 103 (820).

Then, by the conversation state prediction module 104, the text, the mission item candidate, mission items other than the mission item candidate among the plurality of mission items, and other items not related to the plurality of mission items. A step of predicting a conversation state including a slot and a slot value connected to the slot is performed (830).

Subsequently, a system speech generation module 104 outputs a system speech corresponding to 'mission dialogue' or 'free dialogue' based on the predicted dialogue state (840).

In an embodiment, the mission item may be a foreign language representing a topic or learning expression that a learner should utter in a conversation with the conversation device.

In an embodiment, in step 820 of selecting a mission item candidate having the highest similarity with the text, by using an embedding model, a word included in each of the plurality of mission items and the text is included in semantic information. Expressing the text as an embedding vector, and selecting a mission item candidate having the highest similarity with the text from among the plurality of mission items using the embedding vector.

In an embodiment, the step 830 of predicting a conversation state including the slot and a slot value connected to the slot comprises configuring the mission item candidate as the slot, and corresponding to the mission item candidate in the text. configuring the word as the slot value, configuring the remaining mission items as slots, configuring 'none' as a slot value connected to slots respectively composed of the remaining mission items, and configuring the other items as 'none'. It may include configuring a slot representing 'other' and configuring 'false' as a slot value connected to the 'other'.

In an embodiment, in the step 840 of outputting the system speech corresponding to the 'mission dialogue', if the slot value connected to the 'other' is 'false', the learner learns about any one mission item. Determining that the learner has uttered appropriately, recognizing the conversation type between the learner and the conversation device as the 'mission conversation' and a first response indicating an appropriate response to the learner's utterance on any one mission item Expression and a second response expression for the next mission item selected from among the remaining mission items are extracted from the dialog corpus (107 in FIGS. 1 and 2), and the system utterance representing the extracted first and second response expressions is It may include an output step.

In an embodiment, the first response expression may be, for example, “Oh, that's good drawing” in turn 1 of FIG. 6 or “Oh, that's good.” in turn 3 of FIG. 6 .

In an embodiment, the second response expression is an expression in the form of a question for the next mission item, for example, “What's your favorite sport?” in turn 1 of FIG. 6 .

In an embodiment, in the step of generating text corresponding to the learner's voice (810), when the text corresponding to the learner's voice that does not include words related to a plurality of mission items given by the conversation device is generated, The configuring of the slot value connected to 'other' may be a step of changing the slot value indicating 'false' to a slot value indicating 'true'.

In an embodiment, when the slot value connected to the 'other' is 'true', it is determined that the learner has not properly uttered any mission item among the plurality of mission items, and the relationship between the learner and the conversation device is determined. Recognizing the conversation type as 'free conversation' and any first response expression for allowing the learner to focus on the conversation for not properly uttering any mission item among the plurality of mission items; and outputting the system utterance including a second response expression including a second response expression indicating a question for the one mission item or another mission item.

In an embodiment, any of the first response expressions that cause the learner to concentrate on the conversation may be, for example, “Let's try to concentrate.” in turn 2 of FIG. 6 .

In an embodiment, the second response expression including the second response expression indicating a question about any one mission item or another mission item is, for example, “What's your favorite sport?” in turn 2 of FIG. 6 . can

In an embodiment, after the step of outputting the system speech representing the extracted first and second response expressions, according to the system speech according to the second response expression, including words related to all of the remaining mission items. When the learner's voice is uttered, the voice recognition module 102 generating a second text corresponding to the learner's voice including words related to all of the remaining mission items, the mission item candidate selection module ( 103) selecting all of the remaining mission items as mission item candidates, and the conversation state prediction module 104 selecting the second text, the remaining mission items, and others unrelated to the remaining mission items. Using an item, predicting a dialogue state including a slot and a slot value connected to the slot, and the system speech generating module generates a system speech corresponding to a 'mission dialogue' based on the predicted dialogue state. An outputting step may be further included.

In an embodiment, predicting a conversation state including a slot and a slot value connected to the slot comprises configuring the remaining mission items as slots and corresponding to each remaining mission item among words included in the second text. and configuring the other item as a slot value representing 'other' and configuring 'false' as a slot value connected to the 'other'.

In an embodiment, in the step of outputting the system speech corresponding to the 'mission dialogue', if the slot value connected to the 'other' is 'false', it is assumed that the learner properly uttered the remaining mission items. Judging and recognizing the conversation type between the learner and the conversation device as the 'mission conversation' and as the learner appropriately utters all of the plurality of mission items including the remaining mission items, the conversation device and extracting a third response expression representing a greeting for terminating a conversation with the conversation corpus from the conversation corpus, and outputting the system utterance representing the extracted third response expression.

In an embodiment, the third response expression may be, for example, “It was nice talking to you!” in FIG. 6 .

The protection scope of the present invention is not limited to the description and expression of the embodiments explicitly described above. In addition, it is added once again that the scope of protection of the present invention cannot be limited due to obvious changes or substitutions in the technical field to which the present invention belongs.

Claims (1)

A method for learning to speak a foreign language performed by a conversation device,
recognizing, by a voice recognition module, a learner's voice uttered for any one mission item among a plurality of mission items given by the conversation device and generating text corresponding to the learner's voice;
selecting, by a mission item candidate selection module, a mission item candidate having the highest similarity with the text from among the plurality of mission items;
The dialog state prediction module uses the text, the mission item candidate, mission items other than the mission item candidate among the plurality of mission items, and other items unrelated to the plurality of mission items to determine the slot and predicting a conversation state including a slot value connected to the slot; and
outputting, by a system speech generating module, a system speech corresponding to 'mission dialogue' or 'free dialogue' based on the predicted dialogue state;
A method for learning to speak a foreign language comprising a.

Images