WO2021095932A1

WO2021095932A1 - Input determining method and apparatus for dialogue prediction model, and text embedding method and apparatus

Info

Publication number: WO2021095932A1
Application number: PCT/KR2019/015609
Authority: WO
Inventors: 임덕규; 민충기
Original assignee: 주식회사 셀바스에이아이
Priority date: 2019-11-14
Filing date: 2019-11-15
Publication date: 2021-05-20

Abstract

An input determining method and apparatus for a dialogue prediction model, and a text embedding method and apparatus are disclosed. An input determining method for a dialogue prediction model, according to one embodiment of the present invention, is performed by a computing device comprising one or more processors and a memory for storing one or more programs to be executed by the one or more processors, and comprises the steps of: storing the embedding result of a previous text; receiving a current text; embedding the current text; identifying a correlation between the current text and the previous text; determining, on the basis of the correlation, whether the embedding result of the current text and the embedding result of the previous text are to be inputted into the dialogue prediction model, or whether the embedding result of the current text is to be inputted into the dialogue prediction model.

Description

Input determination method and device of conversation prediction model, text embedding method and device

The disclosed embodiments relate to a method and apparatus for determining an input of a conversation prediction model. More specifically, it relates to a method and apparatus for determining an input of a conversation prediction model and a method and apparatus for embedding a text for performing an easy conversation with a user.

A conversation prediction model is used in a chatbot to communicate with a user. Text is extracted from the user's voice, and the text is pre-processed and used as an input value of a conversation prediction model. Preprocessing text involves various steps such as noise removal and morpheme extraction. And, before the text is input into the conversation prediction model, the step of embedding the text is involved. This step is called sentence embedding. The sentence embedding step is usually performed by receiving each morpheme included in text, calculating a multidimensional vector matrix for each morpheme, and calculating a sum or average value of the multidimensional vector matrices. Then, the sum or average value of the multidimensional vector matrices is input into the conversation prediction model. In this way, a response to the user's voice can be obtained.

On the other hand, in order for the chatbot to easily communicate with the user, even if the current text is incomplete, it must be able to grasp the intention of the user more accurately by supplementing the previous text. In addition, in order for the chatbot to easily communicate with the user, it must be able to respond more appropriately to the user's situation. In addition, the chatbot needs to be able to more accurately grasp the atmosphere or context of the conversation.

Disclosed embodiments are to provide a method and apparatus for determining an input of a conversation prediction model and a method and apparatus for embedding a text for easy conversation with a user.

In order to be able to more accurately grasp the user's intention through supplementation of the previous text, even if the current text is incomplete, the present inventor is based on the correlation between the current text and the previous text, and the embedding result for the current text and the previous text. It has been found that there is a need for a method and apparatus for determining an input of a conversation prediction model that determines whether to input an embedding result for the text into a conversation prediction model or to input an embedding result for the current text into the conversation prediction model.

In order to implement a chatbot that can respond more appropriately to the user's situation, the present inventors input the morpheme included in the current text and the attribute value of the current text into the embedding model, so that the attribute value of the text is reflected in the current text. It has been found that there is a need for a text embedding method and apparatus that calculates the embedding result for.

In order to implement a chatbot that can more accurately identify the atmosphere or context of a conversation and communicate with a user more easily, the present inventors have a conversation prediction model that inputs the embedding result for the current text and the embedding result for the previous text into the conversation prediction model. It has been found that a method and apparatus for determining the input of is required.

The present inventors have found that even with little learning data, by selectively increasing the difference between the embedding results for different texts according to the keyword setting, the dialog prediction model can accurately respond to similar texts of the user. .

Disclosed is a method of determining an input of a conversation prediction model according to an embodiment. A method of determining an input of a conversation prediction model according to an embodiment is a method performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, comprising: Storing the embedding result; Receiving the current text; Embedding the current text; Determining a correlation between the current text and the previous text; And determining whether to input the embedding result for the current text and the embedding result for the previous text into a conversation prediction model or to input the embedding result for the current text into the conversation prediction model based on the correlation. Includes.

According to an embodiment, the step of determining the correlation between the current text and the previous text includes correlation between the current text and the previous text based on whether an indication pronoun morpheme exists among morphemes included in the current text. Figure out the relationship.

According to an embodiment, the step of determining the correlation between the current text and the previous text includes inputting the embedding result for the current text and the embedding result for the previous text into a text correlation prediction model, and the text correlation Based on the output value of the relationship prediction model, a correlation between the current text and the previous text is determined.

According to an embodiment, based on the correlation, whether to input the embedding result for the current text and the embedding result for the previous text into a conversation prediction model, or input the embedding result for the current text into the conversation prediction model The step of determining whether to do is, when it is determined that there is the correlation, it is determined to input the embedding result for the current text and the embedding result for the previous text into the conversation prediction model, and when it is determined that there is no correlation , It is determined to input the embedding result for the current text into the conversation prediction model.

An apparatus for determining an input of a conversation prediction model according to an embodiment is disclosed. An apparatus for determining an input of a conversation prediction model according to an embodiment includes: at least one processor; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the program comprising: receiving a current text; Embedding the current text; Determining a correlation between the current text and the previous text; And determining whether to input the embedding result for the current text and the embedding result for the previous text into a conversation prediction model or to input the embedding result for the current text into the conversation prediction model based on the correlation. Contains instructions for executing.

A text embedding method according to an embodiment is disclosed. A text embedding method according to an embodiment is a method performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, the method comprising: receiving a current text; Classifying the generation information of the current text according to a predetermined criterion to determine an attribute value of the current text; And calculating an embedding result for the current text reflecting the attribute value of the text by inputting the morpheme included in the current text and the attribute value of the current text into an embedding model.

A text embedding apparatus according to an embodiment is disclosed. A text embedding apparatus according to an embodiment includes at least one processor; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the program comprising: receiving a current text; Classifying the generation information of the current text according to a predetermined criterion to determine an attribute value of the current text; And instructions for executing the step of calculating an embedding result for the current text reflecting the attribute value of the text by inputting the morpheme included in the current text and the attribute value of the current text into the embedding model.

A method of determining an input of a conversation prediction model according to another embodiment is disclosed. A method of determining an input of a conversation prediction model according to another embodiment is a method performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, wherein a current text is input Receiving step; Embedding the current text; And inputting the embedding result for the current text and the embedding result for the previous text into a conversation prediction model.

An apparatus for determining an input of a conversation prediction model according to another embodiment is disclosed. An apparatus for determining an input of a conversation prediction model according to another embodiment includes: at least one processor; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the program comprising: receiving a current text; Embedding the current text; And instructions for executing the step of inputting the embedding result for the current text and the embedding result for the previous text into a conversation prediction model.

A text embedding method according to another embodiment is disclosed. A method according to an embodiment is a method performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, comprising: determining a keyword morpheme included in a text ; And embedding the text with a weight on the keyword morpheme.

According to an embodiment, in the determining of a keyword included in the text, a morpheme included in a pre-stored keyword list among the morphemes included in the text is determined as the keyword morpheme.

According to an embodiment, the determining of a keyword included in the text includes, when some morphemes included in a previously stored keyword list among the morphemes included in the text are partially overlapped, a morpheme having a maximum number of letters among the partially overlapped morphemes is selected. Determined by keyword morpheme.

According to an embodiment, in the embedding of the text, the morpheme included in the text is input into the embedding model, and the keyword morpheme among the morphemes included in the text is duplicated in the embedding model, thereby generating an embedding result for the text. Calculate.

According to an embodiment, the embedding of the text includes calculating a multidimensional vector matrix for each morpheme included in the text, and summing a multidimensional vector matrix for each morpheme included in the text, wherein the keyword among the morphemes The multidimensional vector matrix for the morpheme is weighted and summed to calculate the embedding result for the text.

A text embedding apparatus according to another embodiment is disclosed. An apparatus according to an embodiment includes one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the program comprising: determining a keyword morpheme included in the text; And instructions for executing the step of embedding the text with a weight on the keyword morpheme.

According to an embodiment, the determining of a keyword included in the text may include, when some of the morphemes included in the text are partially duplicated in a pre-stored keyword list, the keyword is based on the number of letters of the partially overlapped morphemes. Determined by morpheme.

According to an embodiment, a morpheme included in the text is input into an embedding model, and the keyword morpheme among the morphemes included in the text is repeatedly input into the embedding model to calculate an embedding result for the text.

Details of other embodiments are included in the detailed description and drawings.

According to an embodiment, based on the correlation between the current text and the previous text, whether to input the embedding result for the current text and the embedding result for the previous text into the conversation prediction model, or the embedding result for the current text to predict the conversation By deciding whether to input into the model, even if the current text is incomplete, it is possible to more accurately grasp the intention of the user by supplementing the previous text. If the current text is not incomplete, the previous text is assumed to be independent of the current text and the previous text. Can be prevented from affecting the response to the current text.

According to another embodiment, by calculating the embedding result for the current text reflecting the attribute value of the text by inputting the morpheme included in the current text and the attribute value of the current text into the embedding model, a response more appropriate to the user's situation is provided. It is possible to implement a chatbot that can do it.

According to another embodiment, by inputting the embedding result for the current text and the embedding result for the previous text into the conversation prediction model, a chatbot that can more accurately grasp the atmosphere or context of the conversation and communicate with the user more easily is implemented. I can.

In another embodiment, by determining a keyword morpheme included in the text and embedding the text with a weight on the keyword morpheme, the conversation prediction model may provide a more accurate response with less training data. Specifically, according to an embodiment, even with only a small amount of training data, by selectively increasing the difference between the embedding results for different texts according to the keyword setting, the dialogue prediction model improves the accuracy of the response even for similar texts. I can make it. Depending on the embodiment, the keyword can be freely set by the user, and accordingly, the keyword can be input to the site more quickly depending on the application purpose or purpose of the embodiment.

1 is a flowchart of a method for determining an input of a conversation prediction model according to an embodiment of the present invention.

2 is an exemplary diagram illustrating a process of determining an input of a conversation prediction model according to an embodiment of the present invention.

3 is an exemplary diagram for explaining a process of determining an indication pronoun morpheme according to an embodiment of the present invention.

4 is a flowchart of a text embedding method according to an embodiment of the present invention.

5 is a flowchart of a method for determining an input of a conversation prediction model according to another embodiment of the present invention.

6 is a flowchart of a text embedding method according to another embodiment of the present invention.

7 is an exemplary diagram illustrating a process of embedding text according to an embodiment of the present invention.

8 is an exemplary diagram for explaining a process of determining a keyword morpheme according to an embodiment of the present invention.

9 is an exemplary diagram for explaining a process of inputting a morpheme into an embedding model according to an embodiment of the present invention.

10 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, only these embodiments are intended to complete the disclosure of the present invention, and common knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to those who have, and the invention is only defined by the scope of the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining the embodiments of the present invention are exemplary, and thus the present invention is not limited to the illustrated matters. In addition, in describing the present invention, when it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. When'include','have','consists of' and the like mentioned in the present specification are used, other parts may be added unless'only' is used. In the case of expressing the constituent elements in the singular, it includes the case of including the plural unless specifically stated otherwise.

In interpreting the components, even if there is no explicit description, it is interpreted as including an error range.

Although the first, second, and the like are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

The same reference numerals refer to the same elements throughout the specification.

The size and thickness of each component shown in the drawings are illustrated for convenience of description, and the present invention is not necessarily limited to the size and thickness of the illustrated component.

Each of the features of the various embodiments of the present invention may be partially or entirely combined or combined with each other, and as a person skilled in the art can fully understand, technically various interlocking and driving are possible, and each of the embodiments may be independently implemented with respect to each other. It may be possible to do it together in a related relationship

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart of a text embedding method according to an embodiment of the present invention.

The method shown in FIG. 1 may be performed, for example, by the computing device 12 shown in FIG. 10.

First, the computing device 12 stores the embedding result for the previous text (S110).

Next, the computing device 12 receives the current text (S120).

The text may be, for example, text extracted from the user's voice.

However, the present invention is not limited thereto, and the text may be input from, for example, a user's keyboard typing.

For example, the text could be a food order such as “Give me coffee” or “Give me chicken”. However, the present invention is not limited thereto, and various texts may exist according to embodiments. For example, the text may be about symptoms of a disease such as “I have a stomachache” or “I have a tight shoulder”.

The current text refers to the currently input text, and the previous text refers to the text input before the current text is input. On the other hand, the current text is not a term for referring to a moment (a moment) at the time of implementation of the invention, but a term for distinguishing it from the previous text.

For example, the previous text could be a food order such as “make me coffee”, and the current text could be a food cancellation such as “cancel that”.

Meanwhile, the computing device 12 may extract one or more morphemes included in the current text.

For example, a morpheme included in the current text may be extracted by the morpheme analyzer shown in FIG. 2.

As shown in FIG. 3, when the current text is input to the morpheme analyzer, one or more morphemes included in the current text may be output.

For example, if “make coffee” is input to the morpheme analyzer, “coffee” and “make” may be output, and if “cancel that” is input to the morpheme analyzer, “it” and “cancel” are output. Can be.

Meanwhile, the computing device 12 can embed the current text.

For example, embedding of text may be performed by the embedding model shown in FIG. 2.

The embedding model may calculate an embedding result for the current text by calculating a multidimensional vector matrix for each morpheme included in the current text and summing the multidimensional vector matrix for each morpheme. However, the present invention is not limited thereto, and the embedding model may determine the average value of the multidimensional vector matrix for each of the morphemes as the embedding result for the current text.

For example, when the current text is "make coffee", the computing device 12 may calculate an embedding result for "make coffee" by inputting "make coffee" and "make coffee" in the embedding model. In this case, the embedding model may calculate a multidimensional vector matrix for each of “coffee” and “make”, and determine the sum of the multidimensional vector matrix for each of the morphemes as an embedding result for “make coffee”.

As another example, when the current text is "Cancel that", the computing device 12 may calculate an embedding result for "Cancel that" by inputting "it" and "cancel" into the embedding model. In this case, the embedding model may calculate a multidimensional vector matrix for each of “it” and “cancel”, and determine the sum of the multidimensional vector matrix for each of the morphemes as the embedding result for “cancel it”.

Next, the computing device 12 determines the correlation between the current text and the previous text (S130).

As an example, the computing device 12 may determine a correlation between the current text and the previous text based on whether or not an indication pronoun morpheme exists among morphemes included in the current text.

For example, the computing device 12 may determine the correlation between the current text and the previous text by comparing the morpheme included in the current text and a pre-stored indication pronoun list. The denoting pronoun list means a set of denoting pronouns.

Indicative pronouns in the indicative pronoun list may be set by a user, for example. For example, “this”, “that”, “it”, “this time”, “that time”, “then”, “here”, “there”, and “there” can be set as denoting pronouns. However, the present invention is not limited thereto, and the user may set all morphemes for grasping the correlation between the current text and the previous text as the indicating pronouns in the indicating pronoun list in consideration of the purpose of use.

That is, in the present invention, the indicative pronoun includes the indicative pronoun in the Korean language, but is not limited thereto, and includes all terms that can be set by the user to grasp the correlation between the current text and the previous text.

Meanwhile, the indication pronoun list may be continuously updated by the user. As the list of indication pronouns can be freely changed, the user can freely and quickly expand the range of use of the chatbot without accumulating learning data. That is, the computing device 12 can grasp the correlation between the current text and the previous text with only a small amount of learning data.

The computing device 12 may determine a correlation between the current text and the previous text based on whether or not a morpheme included in the pre-stored indication pronoun list among the morphemes included in the current text exists.

For example, when there is an indication pronoun morpheme among the morphemes included in the current text, the computing device 12 determines that there is a correlation between the current text and the previous text, and among the morphemes included in the current text, the indicating pronoun morpheme If is not present, it may be determined that there is no correlation between the current text and the previous text. In general, this is because when a denoting pronoun is included in the current text, the correlation between the current text and the previous text can be considered high.

As another example, the computing device 12 inputs the embedding result for the current text and the embedding result for the previous text into the text correlation prediction model, and based on the output value of the text correlation prediction model, the current text and the previous text are Correlation can be identified.

The correlation prediction model according to an embodiment may receive an embedding result for a current text and an embedding result for a previous text and output a correlation between the current text and the previous text. Meanwhile, in this case, the user can learn the correlation between the current text and the previous text with respect to the correlation prediction model.

At this time, when the output value of the text correlation prediction model is greater than or equal to a preset value, the computing device 12 determines that there is a correlation between the current text and the previous text, and the output value of the text correlation prediction model is a preset value. If it is less than, it may be determined that there is no correlation between the current text and the previous text.

According to an embodiment, when the output value of the text correlation prediction model is greater than or equal to a preset first value, the computing device 12 determines that there is a correlation between the current text and the previous text, and outputs the text correlation prediction model. If the value is less than the preset second value, it is determined that there is no correlation between the current text and the previous text, and if the output value of the text correlation prediction model is less than a preset first value and more than a preset second value, the current It can be determined that the correlation between the text and the previous text is neutral.

As another example, the computing device 12 may input the current text and the previous text into the text correlation prediction model, and determine a correlation between the current text and the previous text based on an output value of the text correlation prediction model.

In this case, the correlation prediction model may receive a current text and a previous text directly and output a correlation between the current text and the previous text. In this case, the user can directly learn the correlation between the current text and the previous text with respect to the correlation prediction model.

Next, based on the correlation between the current text and the previous text, the computing device 12 inputs the embedding result for the current text and the embedding result for the previous text into the conversation prediction model, or the embedding result for the current text. It is determined whether to input into the conversation prediction model (S140).

For example, when it is determined that there is a correlation between the current text and the previous text, the computing device 12 determines to input the embedding result for the current text and the embedding result for the previous text into the conversation prediction model, and When it is determined that there is no correlation between the text and the previous text, it may be determined to input the embedding result for the current text into the conversation prediction model.

For example, if the previous text is “make coffee” and the current text is “cancel that”, the computing device 12 determines that there is a correlation between the current text and the previous text, and embeds the current text. It can be determined by inputting the result and the embedding result for the previous text into the conversation prediction model.

Thereafter, when the computing device 12 inputs the embedding result for the previous text “make coffee” and the embedding result for the current text “cancel that” into the conversation prediction model, for example, the conversation prediction model is “ You can respond to “Cancel coffee”. Accordingly, even if the current text is incomplete, it may be possible to more accurately grasp the intention of the user by supplementing the previous text. Meanwhile, if the current text is not incomplete, it is possible to prevent the previous text from affecting the response to the current text by assuming that the current text and the previous text are independent. Accordingly, it is possible to implement a chatbot that can facilitate a conversation with a user selectively by varying the response method depending on whether the current text is complete or incomplete.

Specifically, according to the disclosed embodiment, even if the current text has an indication pronoun so that the indication object cannot be recognized only with the current text, the indication object can be clearly identified by supplementing the previous text with the current text. More specifically, according to the disclosed embodiment, even if the current text has an indication pronoun and the meaning of the indication cannot be clearly understood through the user's voice, the indication object included in the previous text is supplemented with the current text. It is possible to implement a chatbot that can more accurately grasp the user's inner intentions embedded in the current text.

On the other hand, according to an embodiment, when it is determined that the correlation between the current text and the previous text is neutral, the computing device 12 displays the embedding result for the current text and the embedding result for the previous text according to the user's policy or setting. It may be determined to input into the conversation prediction model or to input the embedding result for the current text into the conversation prediction model.

Meanwhile, in the illustrated flow chart, the method is described by dividing the method into a plurality of steps, but at least some of the steps are performed in a different order, combined with other steps, performed together, omitted, divided into detailed steps, or shown. One or more steps that have not been performed may be added and performed.

The computing device 12 includes a morpheme analyzer, an embedding model, and an indication pronoun list, and may further include a conversation prediction model.

First, the computing device 12 stores an embedding result for the previous text.

Next, the computing device 12 can receive the current text.

For example, the computing device 12 may receive text through a microphone (not shown). However, the present invention is not limited thereto, and the computing device 12 may receive text through a keyboard (not shown). Meanwhile, the text may be a result of removing noise from the user's voice.

The computing device 12 may extract one or more morphemes included in the current text through the morpheme analyzer.

For the morpheme analyzer, open source Hannanum morpheme analyzer, little morpheme analyzer, tweeter morpheme analyzer, Korea University morpheme analyzer, Halla morpheme analyzer, and Komer morpheme analyzer can be used. However, the present invention is not limited thereto, and other known morpheme analyzers and combinations thereof may be used as morpheme analyzers.

Computing device 12 can embed the current text through the embedding model.

The embedding model may calculate an embedding result for the text by calculating a multidimensional vector matrix for each morpheme included in the text and summing the multidimensional vector matrix for each morpheme. However, the present invention is not limited thereto, and the embedding model may determine an average value of the multidimensional vector matrix for each of the morphemes as a result of embedding text.

For the embedding model, open source CBow model, skip-gram model, fasttext model, word2vec model, send2vec model, etc. can be used. However, the present invention is not limited thereto, and other well-known sentence embedding models, morpheme embedding models, and combinations thereof may be used as the embedding model.

Next, the computing device 12 finds a correlation between the current text and the previous text.

For example, the computing device 12 may determine the correlation between the current text and the previous text by comparing the morpheme included in the current text and a pre-stored indication pronoun list.

For example, when there is an indication pronoun morpheme among the morphemes included in the current text, the computing device 12 determines that there is a correlation between the current text and the previous text, and among the morphemes included in the current text, the indicating pronoun morpheme If is not present, it may be determined that there is no correlation between the current text and the previous text.

The correlation prediction model according to an embodiment may receive an embedding result for a current text and an embedding result for a previous text and output a correlation between the current text and the previous text.

Next, based on the correlation between the current text and the previous text, the computing device 12 inputs the embedding result for the current text and the embedding result for the previous text into the conversation prediction model, or the embedding result for the current text. To input into the conversation prediction model.

Thereafter, the computing device 12 may input the embedding result for the previous text and the embedding result for the current text into the conversation prediction model.

The computing device 12 can currently receive text.

The computing device 12 may extract one or more morphemes included in the current text through the morpheme analyzer. For example, in the current text, as shown in FIG. 3, a first morpheme, a second morpheme, a third morpheme, ... The nth morpheme may be extracted.

The computing device 12 may check whether an indication pronoun morpheme exists among morphemes included in the current text.

The denoting pronoun list means a set of denoting pronouns.

Indicative pronouns in the indicative pronoun list may be set by a user, for example. For example, “this”, “that”, “it”, “this time”, “that time”, “then”, “here”, “there”, and “there” can be set as denoting pronouns. However, the present invention is not limited thereto, and the user may set all terms for grasping the correlation between the current text and the previous text as the indication pronouns in the indication pronoun list in consideration of the purpose of use.

Meanwhile, the indication pronoun list may be continuously updated by the user. As the list of indication pronouns can be freely changed, the user can freely expand the range of use of the chatbot without accumulating learning data.

For example, when the second morpheme is included as an indicative pronoun in the Rishi pronoun list, the computing device 12 may determine that the indicative pronoun morpheme exists among the morphemes included in the current text.

4 is a flowchart of a method for determining an input of a conversation prediction model according to another embodiment of the present invention.

The method shown in FIG. 4 may be performed, for example, by the computing device 12 shown in FIG. 10.

First, the computing device 12 receives the current text (S410).

The text may be, for example, text extracted from the user's voice.

Next, the computing device 12 may determine the attribute value of the current text by classifying the current text generation information according to a predetermined criterion (S420).

For example, when the computing device 12 receives the current text from the external device, the computing device 12 may simultaneously receive the current text generation information from the external device. As another example, when the current text is directly input, the computing device 12 may simultaneously generate generation information of the current text.

The text generation information may include, for example, text creator information, text generation time, text generation location, and the like. The information on the creator of the text may be, for example, identification information of the speaker of the target text. The creation time of the text may be, for example, the utterance time of the target text. The place where the text is generated may be, for example, the position of the speaker at the time the target text is uttered.

For example, the computing device 12 classifies the text generation information according to a predetermined criterion, according to the occupation, sex, and age group of the talker, according to whether the speech time is morning, lunch, or evening, and the speech time is spring, summer, or Depending on whether it is autumn or winter, whether the location of the talker is indoor or outdoor, it is possible to check whether the location of the talker is indoor or outdoor, and specifically what location (eg, beach, hospital, mart).

Specifically, the computing device 12 may classify the identification information of the text according to a predetermined criterion to identify the talker's occupation, gender, age range, and the like. For example, the computing device 12 may classify the generation time of the text according to a predetermined criterion to determine whether the utterance time is morning, lunch, evening, spring, summer, autumn, winter, or the like. For example, the computing device 12 may classify the text generation location according to a predetermined criterion to determine whether the speaker's location is indoors or outdoors, and specifically where it is.

The computing device 12 may determine the attribute value of the current text by classifying the current text generation information according to a predetermined criterion using an attribute value table corresponding to the text generation information.

The attribute value table corresponding to the text generation information may include, for example, an attribute value corresponding to the text creator information, an attribute value corresponding to a text generation time, an attribute value corresponding to a text generation location, and the like. .

The user can pre-designate an attribute value corresponding to the creator information of the text, an attribute value corresponding to the creation time of the text, and an attribute value corresponding to the generation location of the text in the attribute value table corresponding to the text generation information. .

At this time, the computing device 12 determines the attribute value corresponding to the text creator information, the attribute value corresponding to the text generation time, the attribute value corresponding to the text generation location, and a combination of the current text attribute values. You can decide.

Accordingly, the computing device 12 may have a location of the talker indoors, depending on whether the speech time is morning, lunch, or evening, and whether the speech time is spring, summer, autumn, or winter according to the occupation, gender, and age group of the talker. The attribute value of the current text may be determined according to whether the talker is outdoors, according to whether the speaker's location is indoors or outdoors, specifically which place, and a combination thereof.

Next, the computing device 12 calculates an embedding result for the current text by inputting the morpheme included in the current text and the attribute value of the current text into the embedding model (S430). Here, the embedding result for the current text is the embedding result for the current text in which the attribute value of the text is reflected.

For example, the computing device 12 calculates a multidimensional vector matrix for each morpheme included in the current text and a multidimensional vector matrix for attribute values of the current text, and the multidimensional vector matrix for each morpheme included in the current text And by summing the multidimensional vector matrix for the attribute values of the current text, an embedding result for the current text reflecting the attribute values of the text may be calculated.

Thereafter, the computing device 12 may input a result of embedding the current text in which the attribute value of the text is reflected into the conversation prediction model.

The conversation prediction model is based on the speaker's occupation, gender, and age, depending on whether the utterance time is in the morning, lunch, or evening, the utterance time is in the spring, summer, autumn, and winter, and the location of the talker is indoors or outdoors. , It is possible to respond to the current text by reflecting whether the location of the talker is indoor or outdoor, specifically which place, and combinations thereof.

For example, even if the current text is the same, the talker's job, gender, and age range, depending on whether the speech is in the morning, lunch, or evening, and whether the speech is in spring, summer, autumn, or winter. , Depending on whether the location of the talker is indoor or outdoor, the response may be different depending on whether the location of the talker is indoor or outdoor, and specifically, which place.

As a result, it is possible to implement a chatbot that can respond more appropriately to the user's situation.

According to an embodiment, the computing device 12 may input each of the embedding result for the morpheme included in the current text and the embedding result for the attribute value of the current text into the conversation prediction model. That is, the conversation prediction model may receive an embedding result for a morpheme included in the current text and an attribute value of the current text.

5 is a flowchart of a text embedding method according to an embodiment of the present invention.

The method shown in FIG. 5 may be performed, for example, by the computing device 12 shown in FIG. 10.

First, the computing device 12 receives the current text (S510).

The text may be, for example, text extracted from the user's voice.

According to an embodiment, the computing device 12 may store an embedding result for the previous text before receiving the current text.

Next, the computing device 12 embeds the current text (S520).

Next, the computing device 12 may input the embedding result for the current text and the embedding result for the previous text into the conversation prediction model (S530).

Accordingly, the conversation prediction model can respond by reflecting the correlation between the embedding result for the current text and the embedding result for the previous text by itself. In general, this is because there is some correlation between the current text and the previous text. As a result, it is possible to implement a chatbot that can more accurately identify the atmosphere or context of a conversation and communicate with a user more easily.

The method shown in FIG. 6 may be performed, for example, by the computing device 12 shown in FIG. 10.

First, the computing device 12 determines a keyword morpheme included in the text (S610). Here, the text may be, for example, text extracted from the user's voice. However, the present invention is not limited thereto, and the text may be input from, for example, a user's keyboard typing.

The computing device 12 may determine a morpheme included in a previously stored keyword list among morphemes included in the text as the keyword morpheme. Meanwhile, the morpheme included in the text may be extracted by the morpheme analyzer shown in FIG. 7.

As illustrated in FIG. 8, when text is input to a morpheme analyzer, one or more morphemes included in the text may be output.

For example, if “make coffee” is input to the morpheme analyzer, “coffee” and “sake” may be output, and if “make me chicken” is input to the morpheme analyzer, “chicken” and “switch” may be output. have.

The keyword list refers to a set of keywords to be weighted when embedding text. Keywords in the keyword list may be set by a user, for example.

For example, in the case of an ordering device, it may be important to distinguish an order object in the user's voice. At this time, the user may set “coffee”, “chicken”, “pizza”, and “cola” as keywords in the keyword list. However, the present invention is not limited thereto, and the user may set keywords in the keyword list in consideration of the purpose of use.

Meanwhile, the keyword list may be continuously updated by the user. As the keyword list can be freely changed, the user can freely expand the scope of use of the chatbot without accumulating learning data.

In the above case, the computing device 12 may determine “coffee” when the text is “make coffee”, and determine “chicken” as the keyword morpheme when the text is “make me chicken”.

Depending on the embodiment, keywords in the keyword list may be set only with nouns, may be set only with adjectives, or may be set with only verbs.

Meanwhile, among the morphemes included in the text, there may be a case that some morphemes included in the keyword list are overlapped.

For example, the user has set “Milk Coffee”, “Black Coffee”, “Coffee”, “Strawberry Milk”, “Chocolate Milk”, and “Milk” as keywords. In this case, “Milk Coffee” and “Coffee” may be output. At this time, the “coffee” part of “milk coffee” and “coffee” is partially overlapped.

In this case, the computing device 12 may determine one morpheme as the keyword morpheme based on the number of letters of some overlapping morphemes. For example, the computing device 12 may determine a morpheme having the largest number of characters among the number of characters of some overlapping morphemes as the keyword morpheme.

Next, the computing device 12 may embed text with a weight on the keyword morpheme (S620). The embedding of text may be performed by the embedding model shown in FIG. 7.

The computing device 12 inputs the morphemes included in the text into the embedding model, and may calculate the embedding result for the text by repeatedly inputting the keyword morphemes among the morphemes included in the text into the embedding model. On the other hand, the number of times (for example, 2 times) of duplicate input of the keyword morpheme may be set by the user.

For example, when the text is “make coffee”, the computing device 12 inputs “coffee” and “make” into the embedding model, but calculates the embedding result for “make coffee” by repeatedly entering “coffee”. I can. Specifically, the computing device 12 may input “coffee”, “coffee”, and “make” into the embedding model.

In this case, the embedding model may calculate a multidimensional vector matrix for each of “coffee”, “coffee”, and “make”, and determine the sum of the multidimensional vector matrix for each of the morphemes as an embedding result for “make coffee”.

On the other hand, the computing device 12 can calculate the embedding result for "make coffee" in another way. For example, when the text is "make coffee", the computing device 12 calculates a multidimensional vector matrix for each of "coffee" and "make", and the multidimensional vector matrix for each of the "coffee" and "make" The embedding result for "Give me coffee" can be calculated by summing up, but adding a weight (eg, 2 times) to the multidimensional vector matrix for "coffee". Specifically, when the multidimensional vector matrix for each of “coffee” and “to make” is a first multidimensional vector matrix and a second multidimensional vector matrix, the computing device 12 uses a first multidimensional vector matrix * 2 + a second multidimensional vector The summation of the matrix can be determined as the result of embedding for "make me coffee".

The embedding result may be used as an input value of the conversation prediction model shown in FIG. 7. As a result, the conversation prediction model can more accurately distinguish between similar voice inputs of the user and provide a more accurate response.

As an example, it is assumed that in the ordering apparatus, text is embedded without weighting a keyword to be ordered. The embedding result for "make coffee" is the sum of the multidimensional vector matrix for each of "coffee" and "make", and the embedding result for "let me chicken" is the sum of the multidimensional vector matrices for each of "chicken" and "make" to be.

As another example, in the ordering apparatus, it is assumed that text is embedded with a weight on a keyword to be ordered. The embedding result for "make coffee" is the sum of the multidimensional vector matrix for each of "coffee", "coffee", and "make", and the embedding result for "make a chicken" is "chicken", "chicken", and "make" It is the sum of the multidimensional vector matrices for each.

For ease of understanding, the multidimensional vector matrix is assumed to be a natural number, and the above two cases are compared. Specifically, it is assumed that the multidimensional vector matrix for “coffee” is 1, the multidimensional vector matrix for “chicken” is 2, and the multidimensional vector matrix for “to order” is 3.

In the case where no weight is given to the keyword to be ordered, the result of embedding “make me coffee” is 4, which is the sum of 1 (multidimensional vector matrix for “coffee”) + 3 (multidimensional vector matrix for “to order”). In the case where no weight is given to the keyword to be ordered, the result of embedding "let me chicken" is 5, which is the sum of 2 (multidimensional vector matrix for "chicken") + 3 (multidimensional vector matrix for "let me").

In the case of weighting the keyword to be ordered, the result of embedding “make me coffee” is 1 (multidimensional vector matrix for “coffee”) + 1 (multidimensional vector matrix for “coffee”) + 3 (multidimensional vector matrix for “coffee”) It is 5 which is the sum of the multidimensional vector matrix). In the case of weighting the keyword to be ordered, the embedding result of “Chicken me” is 2 (multidimensional vector matrix for “chicken”) + 2 (multidimensional vector matrix for “chicken”) + 3 (multidimensional vector matrix for “chicken”) It is 7 which is the sum of the multidimensional vector matrix).

The difference (2) between the embedding result (5) for "make coffee" and the embedding result for "make chicken" (7) in the case of weighting the keyword subject to order (2) is not weighted for the keyword subject to order. In the case, the difference (1) between the embedding result (4) and the embedding result (5) for “make coffee” is greater than (1).

Therefore, when a keyword-weighted work bedding result is used as an input value of a dialogue prediction model, a dialogue prediction model is more textual than when a keyword-weighted embedding result is used as an input value of the dialogue prediction model. Can be recognized more distinctly. That is, even with only a small amount of training data, by selectively increasing the difference between the embedding results for different texts according to the keyword setting, the dialogue prediction model can accurately respond to similar and similar texts.

On the other hand, when the embedding result weighted for a keyword is used as an input value of the conversation prediction model, texts that have similar conversation prediction models compared to the case where the embedding result weighted for the keyword is used as the input value of the conversation prediction model. More learning data is required to give an accurate response to.

Therefore, when the embedding result weighted with respect to the keyword is used as an input value of the conversation prediction model, it is more economical than the case where the embedding result weighted with respect to the keyword is used as the input value of the conversation prediction model.

The computing device 12 may include a morpheme analyzer, an embedding model, and a keyword list, and may further include a conversation prediction model.

First, the computing device 12 may receive text.

The text may be various texts, such as food orders, and disease symptoms, according to embodiments.

The computing device 12 may extract one or more morphemes included in the text through the morpheme analyzer.

The computing device 12 may determine a morpheme included in a previously stored keyword list among morphemes included in the text as the keyword morpheme. The keyword list refers to a set of keywords to be weighted when embedding text. Keywords in the keyword list may be set by a user, for example.

The computing device 12 may determine one morpheme as the keyword morpheme based on the number of letters of the partially overlapped morphemes when some of the morphemes included in the keyword list are partially overlapped among the morphemes included in the text.

For example, when the morphemes included in the keyword list among the morphemes included in the text are partially overlapped, the computing device 12 may determine a morpheme having the largest number of letters among the morphemes included in the text as the keyword morpheme.

The computing device 12 may embed text with a weight on the keyword morpheme through the embedding model.

Depending on the embodiment, the number of duplicate inputs may be different for each keyword morpheme. According to an embodiment, the computing device 12 may calculate an embedding result for the text by inputting only the keyword morphemes among the morphemes included in the text into the embedding model.

In another embodiment, the computing device 12 calculates a multidimensional vector matrix for each morpheme included in the text, and sums a multidimensional vector matrix for each morpheme included in the text, but includes a multidimensional vector for the keyword morpheme among the morphemes. By weighting the matrix and summing it, we can calculate the embedding result for the text.

The computing device 12 may use a result of embedding text as an input value of a conversation prediction model. In addition, the computing device 12 may output a response to a result of embedding text through a conversation prediction model. Meanwhile, before the embedding result is input into the conversation prediction model, other preprocessing may be performed according to embodiments.

The computing device 12 may receive text.

The computing device 12 may extract one or more morphemes included in the text through the morpheme analyzer. For example, in the text, as shown in FIG. 8, the first morpheme, the second morpheme, the third morpheme, ... The nth morpheme may be extracted.

For example, when the second morpheme and the third morpheme are included in the keyword list, the computing device 12 is the first morpheme, the second morpheme, the third morpheme, ... Among the n-th morphemes, the second morpheme and the third morpheme may be determined as keyword morphemes.

Specifically, when the second morpheme and the third morpheme are partially overlapped, the computing device 12 may determine a morpheme having the largest number of letters among the second morpheme and the third morpheme as the keyword morpheme.

For example, the first morpheme, the second morpheme, the third morpheme, ... When the second morpheme and the third morpheme among the n-th morphemes are keyword morphemes, the computing device 12 is the first morpheme, the second morpheme, the third morpheme, ... By inputting the nth morpheme into the embedding model, and inputting only the second morpheme and the third morpheme into the embedding model twice, the embedding result for the text may be calculated.

Depending on the embodiment, the number of times of overlapping input may be set differently for each keyword morpheme. For example, the computing device 12 has a first morpheme, a second morpheme, a third morpheme, ... By inputting the nth morpheme to the embedding model, the second morpheme is inputted twice and the third morpheme is inputted to the embedding model three times, so that the embedding result for the text may be calculated.

According to an embodiment, the computing device 12 may calculate an embedding result for the text by inputting only the keyword morphemes among the morphemes included in the text into the embedding model. For example, the computing device 12 is a first morpheme, a second morpheme, a third morpheme, ... By inputting only the second morpheme and the third morpheme among the n-th morphemes into the embedding model, an embedding result for the text may be calculated.

In the illustrated embodiment, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12. Computing device 12 includes at least one processor 14, a computer-readable storage medium 16, and a communication bus 18. Processor 14 may cause computing device 12 to operate according to the exemplary embodiments mentioned above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16. The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions are configured to cause the computing device 12 to perform operations according to an exemplary embodiment when executed by the processor 14. Can be.

The computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer-readable storage medium 16 includes a set of instructions executable by the processor 14. In one embodiment, computer-readable storage medium 16 includes memory (volatile memory such as random access memory, nonvolatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other types of storage media that can be accessed by computing device 12 and store desired information, or a suitable combination thereof.

The communication bus 18 interconnects the various other components of the computing device 12, including a processor 14 and a computer-readable storage medium 16.

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24. The input/output interface 22 and the network communication interface 26 are connected to the communication bus 18. The input/output device 24 may be connected to other components of the computing device 12 through the input/output interface 22. Exemplary input/output devices 24 include pointing devices (mouse or trackpad, etc.), keyboards, touch input devices (touch pads or touch screens, etc.), voice or sound input devices, various types of sensor devices, and/or photographing devices. Input devices, and/or output devices such as display devices, printers, speakers, and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12, and may be connected to the computing device 12 as a separate device distinct from the computing device 12. May be.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made without departing from the spirit of the present invention. . Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

[National R&D project that supported this invention]

[Task identification number] 2017-0-00255

[Ministry Name] Ministry of Science and Technology Information and Communication

[Research Management Agency] Information and Communication Technology Promotion Center

[Research Project Name] Autonomous Intelligence Digital Partner Technology Research

[Research Title] (Intelligent Information-General / Detailed) Autonomous Intelligence Digital Partner Framework and Application R&D

[Contribution rate] 1/1

[Organization] Electronic Components Research Institute

[Research Period] 2017.04.01 ~ 2020.12.31

Claims

One or more processors, and

A method performed in a computing device having a memory storing one or more programs executed by the one or more processors,

Storing an embedding result for the previous text;

Receiving the current text;

Embedding the current text;

Determining a correlation between the current text and the previous text; And

Based on the correlation, determining whether to input the embedding result for the current text and the embedding result for the previous text into a conversation prediction model, or input a result of embedding the current text into the conversation prediction model. Including, how.
The method of claim 1,

The step of determining a correlation between the current text and the previous text,

A method of determining a correlation between the current text and the previous text based on whether an indication pronoun morpheme exists among morphemes included in the current text.
The method of claim 1,

The step of determining a correlation between the current text and the previous text,

Input the embedding result for the current text and the embedding result for the previous text into a text correlation prediction model, and based on the output value of the text correlation prediction model, a correlation between the current text and the previous text To figure out, how.
The method of claim 1,

Based on the correlation, determining whether to input the embedding result for the current text and the embedding result for the previous text into a conversation prediction model or input the embedding result for the current text into the conversation prediction model, ,

When it is determined that there is the correlation, it is determined to input the embedding result for the current text and the embedding result for the previous text into the conversation prediction model, and if it is determined that there is no correlation, embedding for the current text Determining to input a result into the conversation prediction model.
One or more processors;

Memory; And

An apparatus comprising one or more programs,

The one or more programs are stored in the memory and configured to be executed by the one or more processors,

The above program,

Receiving the current text;

Embedding the current text;

Determining a correlation between the current text and the previous text; And

Based on the correlation, determining whether to input the embedding result for the current text and the embedding result for the previous text into a conversation prediction model, or input a result of embedding the current text into the conversation prediction model. An apparatus comprising instructions for executing.
The method of claim 5,

The step of determining a correlation between the current text and the previous text,

The apparatus for determining a correlation between the current text and the previous text based on whether an indication pronoun morpheme exists among morphemes included in the current text.
The method of claim 5,

The step of determining a correlation between the current text and the previous text,

Input the embedding result for the current text and the embedding result for the previous text into a text correlation prediction model, and based on the output value of the text correlation prediction model, a correlation between the current text and the previous text Grasping device.
The method of claim 1,

Based on the correlation, determining whether to input the embedding result for the current text and the embedding result for the previous text into a conversation prediction model or input the embedding result for the current text into the conversation prediction model, ,

When it is determined that there is the correlation, it is determined to input the embedding result for the current text and the embedding result for the previous text into the conversation prediction model, and if it is determined that there is no correlation, embedding for the current text Determining to input a result into the conversation prediction model.
One or more processors, and

A method performed in a computing device having a memory storing one or more programs executed by the one or more processors,

Receiving the current text;

Classifying the generation information of the current text according to a predetermined criterion to determine an attribute value of the current text; And

And calculating an embedding result for the current text reflecting the attribute value of the text by inputting a morpheme included in the current text and an attribute value of the current text into an embedding model.
One or more processors;

Memory; And

An apparatus comprising one or more programs,

The one or more programs are stored in the memory and configured to be executed by the one or more processors,

The above program,

Receiving the current text;

Classifying the generation information of the current text according to a predetermined criterion to determine an attribute value of the current text; And

The apparatus comprising instructions for executing the step of calculating an embedding result for the current text reflecting the attribute value of the text by inputting a morpheme included in the current text and an attribute value of the current text into an embedding model.
One or more processors, and

A method performed in a computing device having a memory storing one or more programs executed by the one or more processors,

Receiving the current text;

Embedding the current text; And

And inputting the embedding result for the current text and the embedding result for the previous text into a conversation prediction model.
One or more processors;

Memory; And

An apparatus comprising one or more programs,

The one or more programs are stored in the memory and configured to be executed by the one or more processors,

The above program,

Receiving the current text;

Embedding the current text; And

And instructions for executing the step of inputting the embedding result for the current text and the embedding result for the previous text into a conversation prediction model.
One or more processors, and

A method performed in a computing device having a memory storing one or more programs executed by the one or more processors,

Determining a keyword morpheme included in the text; And

Embedding the text with a weight on the keyword morpheme.
The method of claim 12,

Determining the keywords included in the text,

A method of determining a morpheme included in a previously stored keyword list among the morphemes included in the text as a keyword morpheme.
The method of claim 12,

Determining the keywords included in the text,

If the morphemes included in the previously stored keyword list among the morphemes included in the text are partially overlapped, the keyword morphemes are determined based on the number of letters of the partially overlapped morphemes.
The method of claim 12,

Embedding the text,

A method of calculating an embedding result for the text by inputting a morpheme included in the text into an embedding model, and repeatedly inputting the keyword morpheme among the morphemes included in the text into the embedding model.
The method of claim 12,

Embedding the text,

By calculating a multidimensional vector matrix for each morpheme included in the text, and summing a multidimensional vector matrix for each morpheme included in the text, among the morphemes, the multidimensional vector matrix for the keyword morpheme is weighted and summed, Calculating an embedding result for the text.
One or more processors;

Memory; And

An apparatus comprising one or more programs,

The one or more programs are stored in the memory and configured to be executed by the one or more processors,

The above program,

Determining a keyword morpheme included in the text; And

And instructions for performing the step of embedding the text with a weight on the keyword morpheme.
The method of claim 18,

Determining the keywords included in the text,

An apparatus for determining a morpheme included in a previously stored keyword list among the morphemes included in the text as a keyword morpheme.
The method of claim 18,

Determining the keywords included in the text,

When the morphemes included in the previously stored keyword list among the morphemes included in the text are partially overlapped, the keyword morphemes are determined based on the number of letters of the partially overlapped morphemes.
The method of claim 18,

Embedding the text,

An apparatus for calculating an embedding result for the text by inputting a morpheme included in the text into an embedding model, and repeatedly inputting the keyword morpheme among the morphemes included in the text into the embedding model.
The method of claim 18,

Embedding the text,

By calculating a multidimensional vector matrix for each morpheme included in the text, and summing a multidimensional vector matrix for each morpheme included in the text, among the morphemes, the multidimensional vector matrix for the keyword morpheme is weighted and summed, Calculating an embedding result for the text.