KR102383043B1 - Learning method and cognition method for omission restoration and apparatus for executing the method - Google Patents

Abstract

Disclosed are an omission restoration learning method, a recognition method, and an apparatus for performing the same. Omission restoration learning method according to an embodiment disclosed is a method performed in a computing device having a memory for storing one or more programs executed by one or more processors, and one or more processors, and input multi-turn dialog An operation of learning to receive a hidden state vector for each of the previous dialogs that exist before the user's question, and output information related to the restoration type related to the restoration type corresponding to the expression omitted from the user's question in each previous dialog; and and receiving the hidden state vector and restoration type-related information of each previous dialog, and learning to detect the boundary of the restoration value corresponding to the restoration type in each previous dialog.

Description

Omission restoration learning method, recognition method, and apparatus for performing the same

Disclosed embodiments relate to omission recovery techniques for query answering.

Recently, with the development of artificial intelligence technology, the field of providing chatbot services is increasing. In order to achieve continuous service while increasing user satisfaction with the chatbot service, it is necessary to accurately understand the user's query in a continuous conversation with a human (multi-turn conversation) and to provide an appropriate answer. However, conventional chatbots often do not provide a desired answer to a user's query.

This is because, when the user speaks, an omission phenomenon in which specific words are omitted from the user's utterance frequently occurs. For example, consider the following conversation between the user and the chatbot in an AI baseball chatbot.

User: Where did you play the Nexen Kia match yesterday?

Chatbot: It was done at Gocheok Sky Dome.

User: Who is the winning pitcher?

Here, when looking at the user's last question, information about which game is being played is omitted. The user thinks that the chatbot already understands the context of the conversation, such as in a human-to-human conversation, so omitting this information and uttering it occurs frequently. In this case, since the chatbot cannot give a correct answer to the user's question, a method for restoring the omitted expression from the user's question is required.

An embodiment disclosed is to provide an omission restoration learning method and recognition method capable of restoring an expression omitted from a user's question during a multi-turn conversation, and a computing device for performing the same.

Omission restoration learning method according to an embodiment disclosed is a method performed in a computing device having one or more processors, and a memory for storing one or more programs executed by the one or more processors, the input multi-turn Learning to receive a hidden state vector for each of the previous dialogs that exist before the user's question among dialogs, and to output restoration type-related information related to the restoration type corresponding to the expression omitted from the user's question in each of the previous dialogs action to do; and receiving a hidden state vector of each of the previous dialogues and information related to the restoration type, and learning to detect a boundary of a restoration value corresponding to the restoration type in each of the previous conversations.

The omission restoration learning method may further include generating a hidden state vector for each of the previous dialog sentences based on the input multi-turn dialog sentence.

The generating of the hidden state vector may include: generating a token sequence for each of the input multi-turn dialogues; receiving a token sequence for each of the multi-turn dialogues, and generating an embedding vector for the token sequence; converting an embedding vector of each previous dialogue existing before the user's question into a sentence vector; and receiving a sentence vector of each of the previous dialogues, and generating a hidden state vector for each of the previous dialogues.

The converting into the sentence vector includes, when an embedding vector for a user's question is input in the multi-turn dialog, an attention weight for the embedding vector of each previous dialog that exists before the user's question. deciding action; and converting the embedding vector of each previous dialogue into a sentence vector based on the attention weight.

The operation of generating the hidden state vector may include receiving the sentence vectors of each of the previous dialogue sentences in a chronological order from the user's question to generate the hidden state vector.

The restoration type is a type of an entity included in the multi-turn dialogue, and includes at least one of Person, Location, Organization, Date, and Time. can

The restoration type-related information may include at least one of a restoration type classification value for each restoration type of each of the previous dialogues and restoration type inclusion information on whether the restoration type is included in each of the previous conversations.

The operation of learning to detect the boundary of the restoration value may include learning the start position and the end position of the restoration value corresponding to the restoration type from the previous dialog sentences including the restoration type among the previous dialog sentences.

Omission restoration learning apparatus according to an embodiment disclosed is a computing device having a memory for storing one or more programs executed by one or more processors and the one or more processors, Restoration type learning that is learned to receive a hidden state vector for each of the previous dialogues that exist before the question, and to output restoration type-related information related to the restoration type corresponding to the expression omitted from the user's question in each of the previous dialogues wealth; and a restoration value learning unit that receives the hidden state vector of each of the previous dialogues and the restoration type-related information, and is trained to detect a boundary of restoration values corresponding to the restoration type in each of the previous conversations.

Omission restoration recognition method according to an embodiment disclosed is a method performed in a computing device having one or more processors, and a memory for storing one or more programs executed by the one or more processors, receiving a hidden state vector for each of the previous dialogues existing before the user's question among dialogues; and determining whether the expression omitted from the user's question needs to be restored based on the hidden state vectors of the previous dialogues.

The omission restoration recognition method may further include generating a hidden state vector for each of the previous dialogue sentences based on the input multi-turn dialogue sentence.

The generating of the hidden state vector may include: generating a token sequence for each of the input multi-turn dialogues; receiving a token sequence for each of the multi-turn dialogues, and generating an embedding vector for the token sequence; converting an embedding vector of each previous dialogue existing before the user's question into a sentence vector; and receiving a sentence vector of each of the previous dialogues, and generating a hidden state vector for each of the previous dialogues.

The converting into the sentence vector includes, when an embedding vector for a user's question is input in the multi-turn dialog, an attention weight for the embedding vector of each previous dialog that exists before the user's question. deciding action; and converting the embedding vector of each previous dialogue into a sentence vector based on the attention weight.

The operation of generating the hidden state vector may include receiving the sentence vectors of each of the previous dialogue sentences in a chronological order from the user's question to generate the hidden state vector.

The restoration type is a type of an entity included in the multi-turn dialogue, and includes at least one of Person, Location, Organization, Date, and Time. can

The operation of determining whether the omitted expression needs to be restored may include: outputting a restoration type classification value for each restoration type with respect to each previous dialog sentence based on a hidden state vector of each of the previous dialog sentences; outputting a comparison value by comparing a restoration type classification value for each restoration type of each of the previous dialogues with a preset threshold value; and outputting a restoration or not value for each restoration type according to the comparison value.

The outputting of the comparison value includes outputting High as the comparison value when the restoration type classification value is equal to or greater than a preset threshold value, and displaying a low value as the comparison value when the restoration type classification value is less than a preset threshold value. (Low) can be output.

The operation of outputting the restoration status value includes outputting High as the restoration status value if there is at least one case in which the comparison value is High for each restoration type, and the comparison for each restoration type If there is no case where the value is High, Low may be output as the restoration or not value.

The operation of determining whether the omitted expression needs to be restored may include determining that the omitted expression in the user question does not need to be restored when all of the restoration values of the respective restoration types are low. there is.

The omission restoration recognition method may include: recognizing a restoration type of an expression omitted from the user's question; and recognizing a restoration value corresponding to the restoration type of the omitted expression in the previous dialog sentence based on the hidden state vector of each of the previous dialog sentences and the restoration status value.

The operation of recognizing the restoration type of the omitted expression includes extracting the restoration value from a previous utterance having the highest restoration type classification value with respect to a restoration type in which the restoration status value is High among the restoration types. It can be determined by utterance.

The operation of recognizing the restoration value may include receiving the determined hidden state vector of the previous utterance and the determined information including the restoration type of the previous utterance, and outputting a restoration value for a corresponding restoration type in the determined previous utterance.

Omission restoration recognition apparatus according to an embodiment of the present disclosure is a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, A restoration type recognition unit that receives a hidden state vector for each of the previous dialogs existing before the question and checks whether the expression omitted from the user's question needs to be restored based on the hidden state vector of the previous dialogs do.

According to the disclosed embodiment, even if there is an omitted expression in the user's query in the multi-turn conversation between the user and the chatbot, the omitted expression in the user's query can be restored, so a more accurate answer to the user's query be able to provide

1 is a diagram showing the configuration of an omission restoration learning apparatus according to an embodiment of the present disclosure;
2 is a diagram illustrating a state learned according to a restoration type classification value in an omission restoration learning apparatus according to an embodiment of the present disclosure;
3 is a diagram illustrating a state in which a restoration value is learned in an omission restoration learning apparatus according to an embodiment of the present disclosure;
4 is a diagram showing the configuration of an omission restoration recognition apparatus according to an embodiment of the present disclosure;
5 is a block diagram illustrating a configuration of a restoration type recognition unit according to an embodiment of the present disclosure;
6 is a diagram schematically illustrating a restoration type recognition process according to an embodiment of the present disclosure;
7 is a diagram illustrating a state in which a restoration value is recognized in an apparatus for recognizing omitted restoration according to an embodiment of the present disclosure;
8 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments;

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, devices, and/or systems described herein. However, this is merely an example, and the present invention is not limited thereto.

The terminology used in the detailed description is for the purpose of describing the embodiments only, and should in no way be limiting. Unless explicitly used otherwise, expressions in the singular include the meaning of the plural. As used herein, expressions such as “comprising” or “comprising” are intended to indicate certain features, numbers, steps, operations, elements, some or combination thereof, and one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, acts, elements, or any part or combination thereof.

In the following description, the terms "transmission", "communication", "transmission", "reception" and other similar meanings of a signal or information are not only directly transmitted from one component to another component, but also a signal or information This includes passing through other components. In particular, to “transmit” or “transmit” a signal or information to a component indicates the final destination of the signal or information and does not imply a direct destination. The same is true for "reception" of signals or information. In addition, in this specification, when two or more data or information are "related", it means that when one data (or information) is acquired, at least a part of other data (or information) can be acquired based thereon.

Also, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The above terms may be used for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

The disclosed embodiment is for restoring an omitted expression in a user's query in a multi-turn conversation between a user and a chatbot. In the disclosed embodiment, a word or expression to be restored in a user's query may be set as an entity of a preset type. This is to perform omission restoration by focusing on important information (ie, core information) rather than additional information in order to give a correct answer to the user's question including the omitted expression.

Here, the object type may include Person, Location, Organization, Date, and Time. In the disclosed embodiment, the omitted restoration technique may be divided into restoration type recognition and restoration value recognition. The restored value is an expression omitted from the user's query (ie, an expression to be restored) and may mean the contents of the object. The restoration type is a type of restoration value and may mean the type of an object to be restored. For example, if the restoration value is “Dong-Soo Kim”, the restoration time is “Person”.

In this way, when the restoration value is recognized through the restoration type recognition, skip restoration can be performed more accurately and efficiently. For example, consider the following conversation between the user and the chatbot in an AI baseball chatbot.

User: Tell me about LG's major players in 1990.

Chatbot: Catcher Dong-soo Kim and shortstop Jae-Pak Kim were mainly active!

User: What was the age of Dongsoo Kim at that time?

Chatbot: I was 22 years old!

User: Your grades?

Here, the restoration type (Date) and restoration value “1990” are recognized rather than the restoration values “1990”, “LG”, and “Kim Dong-soo” for the user’s question “How are your grades?” Organization) and the restoration value “LG”, and recognizing the restoration type (Person) and restoration value “Kim Dong-soo” can perform more accurate omission restoration.

And, since the restored value and the restored type are recognized as a pair, data can be easily processed in various application fields. In addition, the method of recognizing the restoration type and recognizing the restoration value can be equally applied to the problem of reference resolution. Let's take a look at the dialogue below as an example.

User: Who is the Lotte starter on April 25th?

Chatbot: This is Seungjun Song.

User: Did Song Seung-jun win that day?

Here, "the day" in the user's last question is an expression indicating a date such as "April 25", and thus has a cross-reference relationship. However, the chatbot cannot know when "that day" is only from the user's question. After recognizing the restoration type Date from the user's question in the omission restoration process, the restored value "4" from "question sentence + restoration type" 25" per month can be recognized and processed.

In the disclosed embodiment, the omission restoration technique may be implemented by a deep learning technique. In this case, the omission restoration technique can be largely divided into a learning process and a recognition process.

The learning process 1) converts multi-turn dialogue sentences into vectors in vector space, 2) learns the restoration type omitted from the user question during multi-turn dialogue for each previous utterance, 3) from the user question during multi-turn dialogue It may consist of a process of learning the boundary of the omitted restoration value (the start position of the restoration value and the end position of the restoration value) from the previous utterance.

The recognition process 1) converts multi-turn dialogue sentences into vectors on a vector space, 2) recognizes a restoration type omitted from a user question during multi-turn conversation, and 3) a previous utterance from which restoration values are extracted for each recognized restoration type. Recognizing the location and 4) recognizing the boundary of the restored value omitted from the user's question during the multi-turn conversation in the previous utterance.

1 is a diagram illustrating a configuration of an omission restoration learning apparatus according to an embodiment of the present disclosure.

Referring to FIG. 1 , the omission restoration learning apparatus 100 includes a tokenization unit 102 , an embedding vector generation unit 104 , an attention unit 106 , a hidden state vector generation unit 108 , and a restoration type classification unit 110 . ) may be included.

Hereinafter, a case in which a multi-turn dialogue sentence in which correct answers to the restoration type and restoration value are tagged as follows is used as learning data will be described as an example.

User (U4): (Date =1990) (Team =LG) Tell me about the main players.

Chatbot (U3): Catcher Dong-soo Kim and shortstop Jae-Pak Kim were mainly active!

User (U2): What was your age at this time (Person = Dongsoo Kim)?

Chatbot (U1): I was 22 years old!

User (Q): What is your grade?

The tokenization unit 102 may receive a multi-turn dialogue between the user and the chatbot, respectively. The tokenization unit 102 may generate a token sequence for each multi-turn dialogue sentence (U4, U3, U2, U1, Q).

Specifically, the tokenization unit 102 divides each input multi-turn dialogue sentence (U4, U3, U2, U1, Q) into tokens of a preset division unit, and generates a token sequence including a plurality of divided tokens. can Here, the first previous dialogue (U1) is the dialogue located closest to the user's question (Q) among the multi-turn dialogues, and the fourth previous dialogue (U4) is the most in the user's question (Q) among the multi-turn dialogues. It becomes a dialogue that is located at a distance.

In an exemplary embodiment, the division unit may be a morpheme unit. That is, each token included in the token sequence may be each morpheme included in the multi-turn dialogue. The tokenization unit 102 may perform morphological analysis on each input multi-turn dialogue sentence to divide each multi-turn dialogue sentence into morpheme units. However, the present invention is not limited thereto, and the division unit may be a syllable or the like.

The embedding vector generation unit 104 may receive a token sequence for each multi-turn dialogue output from the tokenization unit 102 . The embedding vector generator 104 may convert the token sequence for each multi-turn dialogue into a vector representation of a preset dimension. The embedding vector generator 104 may generate an embedding vector for the corresponding token sequence in consideration of the left and right contexts of the sentence input in the multi-turn dialog.

In an exemplary embodiment, the embedding vector generator 104 may be a BERT (Bidirectional Encoder Representations from Transformers), but is not limited thereto, and various other deep learning models such as LSTM (Long Short Term Memory) may be used. .

The attention unit 106 may determine an attention weight for each of the previous dialogues U4, U3, U2, and U1 before the user's question Q among the multi-turn dialogues. That is, when the embedding vector (ie, the fifth embedding vector) for the user's question (Q) is input, the attention unit 106 includes a certain part ( That is, which token) to focus on can be expressed through the attention weight. Since such an attention mechanism is a known technique, a detailed description thereof will be omitted.

The attention unit 106 may convert the previous dialogue sentence into a sentence vector of a specific length by summing the attention weights of each token sequence in the embedding vectors of each previous dialogue sentence (U4, U3, U2, U1).

The hidden state vector generator 108 may receive a sentence vector of each previous dialogue output from the attention unit 106 . The hidden state vector generating unit 108 may sequentially receive previous dialogues from the previous dialogue that is close in time to the user's question Q. That is, the hidden state vector generating unit 108 generates the first previous dialog U1, the second previous dialog U2, the third previous dialog U3, and the fourth previous dialog U4 in the order (that is, in the reverse order). ) can be entered.

The hidden state vector generator 108 may receive the sentence vectors of each of the previous dialogues output from the attention unit 106 in a reverse order, and generate a hidden state vector for each of the previous dialogues. In an exemplary embodiment, the hidden state vector generator 108 may use a unidirectional LSTM, but is not limited thereto.

Here, since the hidden state vector generator 108 receives the sentence vectors of each previous dialog in the reverse order, the hidden state vector for the first previous dialog U1 is the second previous dialog U2 to the fourth previous dialog. (U4) is not reflected. That is, in the user question (Q), since there is a higher probability that there is an omitted expression in the previous dialogue at a close distance than in the previous dialogue at a long distance in time, the second in the hidden state vector for the first previous dialogue U1 The characteristics of the previous dialogue texts U2 to U4 of the previous dialogue texts may not be reflected.

In addition, in the hidden state vector for the second previous dialogue sentence U2, the characteristics of the first previous dialogue sentence U1 are reflected, but the characteristics of the third previous dialogue sentences U3 to the fourth previous dialogue sentences U4 are not reflected. . Also, in the hidden state vector for the third previous dialogue sentence U3, the characteristics of the first previous dialogue sentence U1 and the second previous dialogue sentence U2 are reflected, but the characteristics of the fourth previous dialogue sentence U4 are not reflected. . In addition, the hidden state vector for the fourth previous dialog text U4 reflects all the features of the first previous dialog text U1 to the third previous dialog text U3.

The reconstruction type learning unit 110 may receive a hidden state vector of each previous dialogue from the hidden state vector generating unit 108 . The reconstruction type learning unit 110 may be trained to classify the reconstruction type (ie, entity type) included in each previous dialog from the hidden state vector of each previous dialog. In an exemplary embodiment, the reconstruction type learner 110 may include a multi-label classifier.

The multi-label classifier may perform the task of assigning multiple labels to the input data. For example, a multi-label classifier may output a value between 0.0 and 1.0 as a classification value using a sigmoid function.

FIG. 2 is a diagram illustrating a learning state according to a restoration type classification value in the omission restoration learning apparatus according to the disclosed embodiment.

Referring to FIG. 2 , the reconstruction type learning unit 110 may receive the hidden state vectors of each of the previous dialogues U1 to U4 from the hidden state vector generating unit 108 . The restoration type learning unit 110 may learn a restoration type classification value of a corresponding previous dialog from a hidden state vector of each previous dialog. That is, the restoration type learning unit 110 may learn while adjusting parameters so that the restoration type classification values of previous dialogues match the labeled correct answer values (ie, restoration type information to be learned).

Here, each column of the matrix represents each previous dialogue sentence U1 to U4, and each row represents five restoration types (Person, Location, Team, Date, Time) to be learned. And, in each cell of the matrix, the learned parameter for the reconstruction type classification value is expressed as the density of the shade. That is, a dark cell indicates that the parameter for the restoration type classification value is learned close to 1.0, and the cell with a light shade indicates that the parameter for the restoration type classification value is learned close to 0.0.

In the case of the 4th previous dialogue (U4) "(Date = 1990) (Team = LG) tell me the main players", the restoration type contains the correct answer for the restoration type Date and Team, so the restoration type (Date, Team) Classification values were trained with high values (that is, dark shadows).

In the case of the second previous dialogue sentence (U2) “What was your age (Person = Dongsoo Kim) at this time?”, since it contains the correct answer for the restoration value for the restoration type Person, the classification value for the restoration type (Person) is high (that is, , dark shading).

In addition, in the case of the 3rd previous dialogue (U3) "Catcher Dong-soo Kim, shortstop Kim Jae-bak, etc. were mainly active!", the restoration value correct answer (Kim Dong-soo) is included for the restoration type Person, but the incorrect answer (Kim Jae-bak) is not the restoration value. Therefore, it was learned with a lower classification value (ie, normal shade) than the second previous dialogue (U2).

Here, the restoration type learning unit 110 may learn to check the restoration type information included in each previous dialog through the restoration type classification value of each previous dialog. That is, the restoration type learning unit 110 compares the restoration type classification value (ie, a value between 0.0 and 1.0) with a preset threshold value (eg, 0.5) to determine whether the restoration type of each previous dialogue is included. can be learned to confirm.

The restoration type learning unit 110 may be trained to determine that the previous dialogue includes the corresponding restoration type when the restoration type classification value is equal to or greater than a preset threshold value. When the restoration type classification value is less than a preset threshold value, the restoration type learning unit 110 may be trained to determine that the previous dialogue does not include the restoration type.

According to this, in FIG. 2 , the restoration type learning unit 110 may be trained to determine that the fourth previous dialogue text U4 includes the restoration types “Team” and “Date”. Although the third previous dialogue U3 indicates a restoration type classification value of a certain level with respect to the restoration type Person, it may be learned to determine that the restoration type does not include anything because it is less than a preset threshold value. The second previous dialogue U2 may be learned to determine that it includes the restoration type “Person”. The first previous dialogue U1 may be learned to determine that the restoration type contains nothing. The restoration type learning unit 110 may be trained to output restoration type information according to whether a restoration type is included for each previous dialogue.

3 is a diagram illustrating a state in which a restoration value is learned in the omission restoration learning apparatus according to an embodiment of the present disclosure; Referring to FIG. 3 , the omission restoration learning apparatus 100 may further include a restoration value learning unit 112 . The restoration value learning unit 112 may receive a hidden state vector of each of the previous dialogs U1 to U4 and information including the restoration type of each of the previous dialogs U1 to U4, respectively.

The restoration value learning unit 112 may receive the hidden state vectors of each of the previous dialogues U1 to U4 from the hidden state vector generating unit 108 . Also, the restoration value learning unit 112 may receive information including restoration types of each of the previous dialogues U1 to U4 from the restoration type learning unit 110 . Here, the restoration value learning unit 112 may input by concatenating restoration type-included information to the hidden state vectors of each previous dialogue sentences U1 to U4. Since the fourth previous dialogue U4 includes two restoration types, “Date” and “Team”, information including restoration types for each restoration type can be individually input and learned.

The restoration value learning unit 112 may be implemented to learn the boundary of the restoration value corresponding to the restoration type in the previous dialog including the restoration type (that is, the start position of the restoration value and the end position of the restoration value in the previous dialog). there is. In an exemplary embodiment, the restored value learning unit 112 may be formed of a Pointer Network, but is not limited thereto.

4 is a diagram illustrating a configuration of an omission restoration recognition apparatus according to an embodiment of the present disclosure. Referring to FIG. 4 , the omission restoration recognition apparatus 200 includes a tokenization unit 202 , an embedding vector generation unit 204 , an attention unit 206 , a hidden state vector generation unit 208 , and a restoration type recognition unit 210 . ) may be included.

Here, the tokenization unit 202, the embedding vector generation unit 204, the attention unit 206, and the hidden state vector generation unit 208 are the tokenization unit 102 and the embedding vector generation unit 104 shown in FIG. .

The restoration type recognition unit 210 may check whether the expression omitted from the user question (Q) of the multi-turn dialogue needs to be restored. The restoration type recognition unit 210 may recognize the restoration type of the expression omitted from the user question (Q). The restoration type recognition unit 210 may extract a previous utterance from which a restoration value is to be extracted from the previous utterance of the user question Q.

5 is a block diagram showing the configuration of the restoration type recognition unit 210 according to the disclosed embodiment, and FIG. 6 is a diagram schematically illustrating a restoration type recognition process according to the disclosed embodiment. In FIG. 6 , each column of the matrix represents each previous dialogue text U1 to U4 , and each row represents five restoration types (Person, Location, Team, Date, Time).

5 and 6 , the restoration type recognition unit 210 may include a classifier 221 , a threshold value comparison unit 223 , a restoration status confirmation unit 225 , and a utterance determination unit 227 . there is.

The classifier 221 may receive the hidden state vector of each previous dialogue from the hidden state vector generator 208 . The classifier 221 may classify the reconstruction type included in each previous dialog from the hidden state vector of each previous dialog. In an exemplary embodiment, the classifier 221 may output a value between 0.0 and 1.0 as a restoration type classification value using a sigmoid function. The classifier 221 may output a restoration type classification value for each restoration type (eg, Person, Location, Team, Date, Time) for each previous dialogue.

The threshold value comparison unit 223 may output a comparison value by comparing a restoration type classification value for each restoration type of each previous dialogue with a preset threshold value. For example, the threshold value comparison unit 223 may output 1 (ie, High) as the comparison value when the restoration type classification value for each restoration type of each previous dialogue sentence is equal to or greater than a preset threshold value. Also, the threshold value comparison unit 223 may output 0 (ie, Low) as a comparison value when the restoration type classification value for each restoration type of each previous dialogue sentence is less than a preset threshold value.

The restoration confirmation unit 225 may output a restoration or not value for each restoration type according to the comparison value output from the threshold value comparison unit 223 . If there is at least one case in which the comparison value is 1 for each restoration type, the restoration or not check unit 225 may output a restoration or not value of 1 (ie, High) indicating that the restoration type is restored. If there is no case where the comparison value is 1 for each restoration type, the restoration check unit 225 may output a restoration or not value of 0 (ie, Low) indicating that the restoration type is not restored.

For example, in FIG. 6 , the restoration check unit 225 outputs a restoration value of 1 for the restoration type “Person”, outputs a restoration value 0 for the restoration type “Location”, and outputs a restoration value of 0 for the restoration type “Team”. A restoration or not value of 1 may be outputted to , a restoration possibility value of 1 may be outputted to the restoration type “Date”, and a restoration value 0 may be outputted to the restoration type “Time”.

The restoration check unit 225 may determine that the expression omitted from the user question (Q) does not need to be restored when the restoration or not value is 0 (not restored) for all restoration types.

The utterance determiner 227 may determine a previous utterance having the highest restoration type classification value as a previous utterance from which a restoration value is to be extracted with respect to a restoration type having a restoration value of 1 . For example, in FIG. 6 , in the case of the restoration type “Person” in which the restoration value is 1, the utterance determiner 227 may determine the second previous utterance U2 as the previous utterance from which the restoration value is to be extracted. Also, the utterance determiner 227 may determine the fourth previous utterance U4 as the previous utterance from which the restoration value is to be extracted for the restoration type “Team” in which the restoration value is 1 . Also, the utterance determiner 227 may determine the fourth previous utterance U4 as the previous utterance from which the restoration value is to be extracted for the restoration type “Date” in which the restoration value is 1 .

7 is a diagram illustrating a state of recognizing a restoration value in an apparatus for recognizing omitted restoration according to an embodiment of the present disclosure. Referring to FIG. 7 , the omitted restoration recognition apparatus 200 may further include a restoration value recognition unit 212 .

The restored value recognition unit 212 receives the hidden state vectors of the previous utterances (eg, the second previous utterance U2 and the fourth previous utterance U4) determined by the utterance determiner 227 . can Also, the restoration value recognition unit 212 may receive information including restoration types (“Person”, “Team”, and “Date”) having a restoration value of 1 from the restoration confirmation unit 225 .

The restoration value recognition unit 212 may concatenate and input restoration type inclusion information to the hidden state vector of each previous utterance, and output the recognized restoration values, respectively. For example, the restored value recognition unit 212 may output the restored value “Dongsoo Kim” from the second previous utterance U2 for the restoration type “Person”. The restoration value recognition unit 212 may output the restoration value “LG” from the fourth previous utterance U4 for the restoration type “Team”. The restored value recognizing unit 212 may output the restored value “1990” from the fourth previous utterance U4 for the restoration type “Date”. In an exemplary embodiment, the restored value recognition unit 212 may be formed of a Pointer Network.

According to the disclosed embodiment, even if there is an omitted expression in the user's query in the multi-turn conversation between the user and the chatbot, the omitted expression in the user's query can be restored, so a more accurate answer to the user's query be able to provide

8 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12 . In one embodiment, the computing device 12 may be the omission recovery learning device 100 . Also, the computing device 12 may be the omission restoration recognition device 200 .

Computing device 12 includes at least one processor 14 , computer readable storage medium 16 , and communication bus 18 . The processor 14 may cause the computing device 12 to operate in accordance with the exemplary embodiments discussed above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16 . The one or more programs may include one or more computer-executable instructions that, when executed by the processor 14, configure the computing device 12 to perform operations in accordance with the exemplary embodiment. can be

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer readable storage medium 16 includes a set of instructions executable by the processor 14 . In one embodiment, computer-readable storage medium 16 includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other forms of storage medium accessed by computing device 12 and capable of storing desired information, or a suitable combination thereof.

Communication bus 18 interconnects various other components of computing device 12 , including processor 14 and computer readable storage medium 16 .

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . The input/output interface 22 and the network communication interface 26 are coupled to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output device 24 may include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or imaging devices. input devices and/or output devices such as display devices, printers, speakers and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12 , and may be connected to the computing device 12 as a separate device distinct from the computing device 12 . may be

Although representative embodiments of the present invention have been described in detail above, those of ordinary skill in the art to which the present invention pertains will understand that various modifications are possible within the limits without departing from the scope of the present invention with respect to the above-described embodiments. . Therefore, the scope of the present invention should not be limited to the described embodiments, and should be defined by the claims described below as well as the claims and equivalents.

100: game service system
102: user terminal
102a: remote game providing application
104: remote execution terminal
104a : emulator
104b: game software
104c: Web RTC Module
202: game execution module
204: Streaming module
211: first interface
213 : state agent
215: second interface
217 : Streaming Controller

Claims (23)

one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
generating a hidden state vector for each of the previous dialogues existing before the user's question among the multi-turn dialogues based on the input multi-turn dialogue;
learning to receive a hidden state vector for each of the previous dialogues and to output restoration type-related information related to a restoration type corresponding to an expression omitted from the user's question in each of the previous dialogues; and
receiving a hidden state vector of each of the previous dialogues and information related to the restoration type, and learning to detect a boundary of a restoration value corresponding to the restoration type in each of the previous conversations;
The restoration type-related information includes at least one of a restoration type classification value for each restoration type of each of the previous conversations and restoration type inclusion information on whether the restoration type is included in each of the previous conversations. Omit restoration learning method for . delete The method according to claim 1,
The operation of generating the hidden state vector comprises:
generating a token sequence for each of the input multi-turn dialogues;
receiving a token sequence for each of the multi-turn dialogues, and generating an embedding vector for the token sequence;
converting an embedding vector of each previous dialogue existing before the user's question into a sentence vector; and
and receiving the sentence vector of each of the previous dialogues and generating a hidden state vector for each of the previous dialogues. 4. The method according to claim 3,
The operation of converting to the sentence vector is,
determining an attention weight with respect to an embedding vector of each previous dialogue that exists before the user's question when an embedding vector for a user's question is input in the multi-turn dialogue; and
and converting the embedding vector of each previous dialogue into a sentence vector based on the attention weight. 4. The method according to claim 3,
The operation of generating the hidden state vector comprises:
The omission restoration learning method for a chatbot service, wherein the hidden state vector is generated by receiving the sentence vectors of the respective previous dialogues in the order closest in time to the user's question. The method according to claim 1,
The restoration type is
As a type of entity included in the multi-turn dialogue, a chatbot service that includes at least one of Person, Location, Organization, Date, and Time Omit Restoration Learning Method for. delete one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
Among the input multi-turn dialogues, a hidden state vector for each of the previous dialogues existing before the user's question is input, and restoration type related information related to the restoration type corresponding to the expression omitted from the user's question in each of the previous dialogues. learning to output and
receiving a hidden state vector of each of the previous dialogues and information related to the restoration type, and learning to detect a boundary of a restoration value corresponding to the restoration type in each of the previous conversations;
The operation of learning to detect the boundary of the restored value is,
Omit restoration learning method for a chatbot service, for learning the start position and the end position of the restoration value corresponding to the restoration type in the previous dialog including the restoration type among the previous dialog sentences. one or more processors, and
A computing device having a memory to store one or more programs executed by the one or more processors, the computing device comprising:
Among the input multi-turn dialogues, a hidden state vector for each of the previous dialogues existing before the user's question is input, and restoration type related information related to the restoration type corresponding to the expression omitted from the user's question in each of the previous dialogues. a restoration type learning unit that is learned to output ; and
and a restoration value learning unit that receives the hidden state vector of each of the previous dialogues and the restoration type-related information, and is trained to detect a boundary between restoration values corresponding to the restoration type in each of the previous conversations;
The restoration value learning unit may be configured to learn a start position and an end position of a restoration value corresponding to the restoration type from a previous conversation including the restoration type among the previous conversation sentences. one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
generating a hidden state vector for each of the previous dialogues existing before the user's question among the multi-turn dialogues based on the input multi-turn dialogue;
receiving a hidden state vector for each of the previous dialogues;
checking whether the expression omitted from the user's question needs to be restored based on the hidden state vectors of the previous dialogues;
recognizing a restoration type of the omitted expression in the user's question when the omitted expression needs to be restored; and
Omit for the chatbot service, comprising the operation of recognizing a restoration value corresponding to the restoration type of the omitted expression in the previous conversation sentence based on the hidden state vector of each of the previous conversation sentences and the restoration status value calculated for each restoration type How to recognize restoration. delete 11. The method of claim 10,
The operation of generating the hidden state vector comprises:
generating a token sequence for each of the input multi-turn dialogues;
receiving a token sequence for each of the multi-turn dialogues, and generating an embedding vector for the token sequence;
converting an embedding vector of each previous dialogue existing before the user's question into a sentence vector; and
and receiving a sentence vector of each of the previous dialogues and generating a hidden state vector for each of the previous dialogues. 13. The method of claim 12,
The operation of converting to the sentence vector is,
determining an attention weight with respect to an embedding vector of each previous dialogue that exists before the user's question when an embedding vector for a user's question is input in the multi-turn dialogue; and
and converting the embedding vector of each previous dialogue into a sentence vector based on the attention weight. 13. The method of claim 12,
The operation of generating the hidden state vector comprises:
A method for recognizing omission restoration for a chatbot service, wherein the hidden state vector is generated by receiving the sentence vectors of each of the previous dialogues in a chronologically close order from the user's question. 11. The method of claim 10,
The restoration type is
As a type of entity included in the multi-turn dialogue, a chatbot service that includes at least one of Person, Location, Organization, Date, and Time Skip for restoration recognition method. 11. The method of claim 10,
The operation of checking whether the omitted expression needs to be restored is,
outputting a restoration type classification value for each restoration type with respect to each of the previous dialog sentences based on the hidden state vector of each of the previous dialog sentences;
outputting a comparison value by comparing a restoration type classification value for each restoration type of each of the previous dialogues with a preset threshold value; and
An omitted restoration recognition method for a chatbot service, comprising outputting a restoration or not value for each restoration type according to the comparison value. 17. The method of claim 16,
The operation of outputting the comparison value is,
A chatbot service that outputs High as the comparison value when the restoration type classification value is greater than or equal to a preset threshold value, and outputs Low as the comparison value when the restoration type classification value is less than a preset threshold value Skip restoration recognition method for. 18. The method of claim 17,
The operation of outputting the restoration value is,
If there is more than one case in which the comparison value is High for each restoration type, High is output as the restoration status value, and there is no case in which the comparison value is High for each restoration type. If there is no restoration or not, outputting a low as the restoration value, omitting restoration recognition method for a chatbot service. 19. The method of claim 18,
The operation of checking whether the omitted expression needs to be restored is,
When the restoration value of each restoration type is all low, it is determined that the expression omitted from the user question does not need restoration. delete 20. The method of claim 19,
The operation of recognizing the restoration type of the omitted expression is,
Omit restoration recognition for a chatbot service, which determines the previous utterance having the highest restoration type classification value as the previous utterance from which the restoration value is to be extracted for the restoration type in which the restoration value is High among the restoration types Way. 22. The method of claim 21,
The operation of recognizing the restored value is,
A method for recognizing omission restoration for a chatbot service, receiving the determined hidden state vector of the previous utterance and the determined information including the restoration type of the previous utterance, and outputting a restoration value for the restoration type in the determined previous utterance. one or more processors, and
A computing device having a memory to store one or more programs executed by the one or more processors, the computing device comprising:
Among the input multi-turn dialogues, a hidden state vector for each previous dialogue that exists before the user's question is input, and based on the hidden state vector of the previous dialogues, whether the expression omitted from the user's question needs to be restored Includes a restoration type recognition unit to confirm,
The restoration type recognition unit,
Outputs a restoration type classification value for each restoration type for each previous dialog sentence based on the hidden state vector of each previous dialog sentence, and a restoration type classification value for each restoration type of each previous dialog sentence and a preset threshold value and outputs a comparison value, and outputs a restoration or not value for each restoration type according to the comparison value, an omission restoration recognition device for a chatbot service.

Images