KR102261411B1 - Mehtod and apparatus for learning chatting model - Google Patents

Abstract

A chat model learning method and apparatus are disclosed. A chatting model learning method according to an embodiment includes: learning the chatting model using an unsupervised learning technique using a plurality of sentences as learning data; and additionally learning the learned chatting model using a supervised learning technique using a plurality of dialogue sentence pairs each including an utterance sentence and a response sentence as learning data.

Description

Chat model learning method and apparatus {MEHTOD AND APPARATUS FOR LEARNING CHATTING MODEL}

The disclosed embodiments relate to machine learning techniques.

A chat system is a conversation between a human and a computer using natural language, and can be divided into a search model and a generative model depending on the method. The search model finds the utterance most similar to the user input in the chat data of the utterance-response pair and outputs a response corresponding thereto. Because it outputs a fixed response, it can output an accurate output, but it cannot output an answer other than data. The generative model learns chat data and generates responses based on user input. The generative model can generate various responses that are not in the data, but there are problems in generating responses that do not conform to grammar or repeating the same word.

Disclosed embodiments are to provide a chat model learning method and apparatus.

A chatting model learning method according to an embodiment includes: learning the chatting model using an unsupervised learning technique using a plurality of sentences as learning data; and additionally learning the learned chatting model by using a supervised learning technique using a plurality of dialogue sentence pairs each including an utterance sentence and a response sentence as learning data.

The learning operation may train the chatting model to output the same sentence as the input sentence.

The chat model may be an encoder-decoder model including an encoder and a decoder configured with a recurrent neural network (RNN), respectively.

The chat model may be an attention-based encoder-decoder model to which an attention mechanism is applied.

In the additional learning operation, the learned chatting model may be additionally trained by using the weight of the learned chatting model as an initial weight.

Chatting model learning apparatus according to an embodiment, a memory for storing one or more instructions; and one or more processors executing the one or more instructions, wherein the one or more processors train the chatting model using an unsupervised learning technique using a plurality of sentences as learning data, each including an utterance sentence and a response sentence By using a supervised learning technique using a plurality of dialogue sentence pairs as learning data, the learned chatting model is further trained.

The one or more processors may train the chatting model to output the same sentence as the input sentence.

The chat model may be an encoder-decoder model including an encoder and a decoder configured with a recurrent neural network (RNN), respectively.

The chat model may be an attention-based encoder-decoder model to which an attention mechanism is applied.

The one or more processors may further train the learned chatting model by using the weight of the learned chatting model as an initial weight.

According to the disclosed embodiments, the chat model is trained using an unsupervised learning technique using a plurality of sentences as learning data to improve the sentence understanding ability and sentence generation ability of the chat model, and then a plurality of conversation sentence pairs are learned By further learning the chat model using the supervised learning technique used as data, it is possible to improve the sentence understanding ability, sentence generation ability, and conversation ability of the chat model even if a small number of conversation sentence pairs that are relatively difficult to collect are used as training data. have.

1 is a block diagram of an apparatus for learning a chatting model according to an embodiment.
2 is a diagram showing a schematic configuration of a chatting model according to an embodiment.
3 is a diagram illustrating a schematic configuration of a chatting model according to another embodiment.
4 is a flowchart of a chat model learning method according to an embodiment.
5 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments.

Hereinafter, specific embodiments will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, apparatus, and/or systems described herein. However, this is merely an example and the present invention is not limited thereto.

In describing the embodiments, if it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter, the detailed description thereof will be omitted. And, the terms to be described later are terms defined in consideration of functions, which may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the content throughout this specification. The terminology used in the detailed description is for the purpose of describing the embodiments only, and should in no way be limiting. Unless explicitly used otherwise, expressions in the singular include the meaning of the plural. In this description, expressions such as "comprising" or "comprising" are intended to indicate certain features, numbers, steps, acts, elements, some or a combination thereof, and one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, acts, elements, or any part or combination thereof.

1 is a block diagram of an apparatus for learning a chatting model according to an embodiment.

Referring to FIG. 1 , the chat model learning apparatus 100 according to an embodiment includes a first learning unit 110 and a second learning unit 120 .

Meanwhile, in one embodiment, the first learning unit 110 and the second learning unit 120 are implemented using one or more physically separated devices, or implemented by one or more processors or a combination of one or more processors and software. may be, and unlike the illustrated example, it may not be clearly distinguished in a specific operation.

The first learning unit 110 learns the chatting model using an unsupervised learning technique using a plurality of sentences as learning data.

In this case, the chatting model may be a deep neural network-based sentence generation model for generating a response sentence to a sentence input by the user in a chatting system that performs a conversation with the user using a natural language sentence.

Meanwhile, according to an embodiment, each of the plurality of sentences used by the first learning unit 110 for unsupervised learning of the chatting model is, for example, a sentence that is pre-collected from a news article or written by a user. can

Also, according to an embodiment, each of the plurality of sentences may be a sentence written in a specific style, such as a written style, but is not necessarily limited to a sentence written in a specific style.

Also, according to an embodiment, the first learning unit 110 may train the chatting model to generate the same sentence as the sentence input for learning using an unsupervised learning technique. For this, for example, an auto encoder may be used as an unsupervised learning technique.

Specifically, the first learning unit 110 may learn the weight of each hidden layer constituting the deep neural network of the chatting model so that, when a specific sentence is input, the sentence output from the chatting model is output by imitation of the inputted sentence. . Accordingly, the chat model learned by the first learning unit 110 can improve the sentence understanding ability to understand the input sentence and the sentence generation ability to generate a correct sentence without grammatical errors.

The second learning unit 120 is a chat model learned by the first learning unit 110 using a supervised learning technique using a plurality of dialogue sentence pairs each including an utterance sentence and a response sentence as learning data. to learn additionally.

Specifically, the second learning unit 120 uses the spoken sentence included in the conversation sentence pair used as learning data as input data of the chatting model, and uses the response sentence included in the conversation sentence pair as the target data of the chatting model. The chat model learned by the first learning unit 120 may be additionally trained by using it.

In this case, the second learning unit 120 may use the weight of the chat model learned by the first learning unit 120 as an initial weight for further learning.

Specifically, the initial weight of each hidden layer of the chatting model to be further learned by the second learning unit 120 may be determined by the unsupervised learning result performed by the first learning unit 110 . In addition, the second learning unit 120 receives the spoken sentence included in the conversation sentence pair used as learning data, and updates the initial weight of the chat model so that the response sentence included in the conversation sentence pair is output. You can learn more.

Accordingly, the chatting model additionally learned by the second learning unit 120 can improve the conversational ability while maintaining the sentence understanding ability and the sentence generating ability.

Meanwhile, a plurality of pairs of dialogue sentences used for supervised learning may be pre-collected from, for example, a database that stores contents of conversations performed between two or more people using natural language sentences or may be prepared by a user.

2 is a diagram showing a schematic configuration of a chatting model according to an embodiment.

Referring to FIG. 2 , the chat model 200 according to an embodiment is an encoder including an encoder 210 and a decoder 230 configured of a Recurrent Neural Network (RNN), respectively - a decoder-decoder can be a model.

Specifically, the encoder 210 generates a context vector 220 from the input sequence X={x ₁ , x ₂ , x ₃ , ..., x _n }, and the decoder 230 generates a context vector ( 220) can generate the output sequence Y={y ₁ , y ₂ , y ₃ ,..., y _m }.

At this time, the token values (ie, x ₂ , x ₃ , ..., x _n ) included in the input sequence X are each divided into token units such as morphemes and syllables, and then each divided The token may be, for example, an embedding vector converted through word embedding.

In addition, token values (ie, y ₁ , y ₂ , y ₃ ,..., y _m ) included in the output sequence Y may each be an embedding vector for a token to be included in the output sentence.

On the other hand, when the i-th token value (ie, x _i ) of the input sequence X to the encoder 210 is input to the encoder 210 , the hidden state vector h _i of the encoder 210 is calculated by the following math. It can be expressed as Equation 1.

[Equation 1]

In Equation 1, f represents a non-linear active function, and may be, for example, a Long-Short Term Memory (LSTM) or a Gated Recurrent Unit (GRU). Hereinafter, f is used with the same meaning.

Meanwhile, the context vector 220 c may be generated using Equation 2 below.

[Equation 2]

In Equation 2, q represents a nonlinear function for generating a context vector, and may be, for example, a function for extracting the last hidden state vector of the encoder 210 as shown in Equation 3 below.

[Equation 3]

Meanwhile, the decoder 230 may receive the context vector 220 , the hidden state vector of the decoder 230 at the previous output time, and the token value output at the previous output time, and determine the token value to be output at the current output time.

_{Specifically, the hidden state vector S j} of the decoder 230 at the current output time j may be expressed as Equation 4 below.

[Equation 4]

In addition, the decoder 230 calculates a conditional probability for a token value to be output at the current time point as shown in Equation 5 below, and determines the token value to be output at the current time point such that the calculated probability becomes the maximum. have.

[Equation 5]

3 is a diagram illustrating a schematic configuration of a chatting model according to another embodiment.

Referring to FIG. 3 , the chat model 300 according to an embodiment may be an attention-based encoder-decoder model to which an attention mechanism is applied.

In the embodiment shown in FIG. 3 , the encoder 310 may be configured as a bidirectional RNN (RNN) including a forward RNN (forward RNN) and a backward RNN (backward RNN).

와 역방향 은닉 상태 벡터

는 각각 아래의 수학식 6 및 7과 같이 표현될 수 있다.Specifically, the forward hidden state vector of the encoder 310 for the i-th token value (ie, x _{i ) of the input sequence X.}

and the reverse hidden state vector

may be expressed as Equations 6 and 7 below, respectively.

[Equation 6]

[Equation 7]

는 아래의 수학식 8과 같이

와

를 결합(concatenate)하여 생성될 수 있다.Also, the hidden state vector of encoder 310 for _{x i}

is as in Equation 8 below

Wow

It can be created by concatenating

[Equation 8]

Meanwhile, the attention layer 320 may determine an attention weight at the current output time of the decoder 330 for each hidden state vector generated by the encoder 310 . In this case, the attention weight indicates which token value among token values included in the input sequence should be focused in order to determine the token value to be output at the current output time.

에 대한 주의 집중 가중치

는 예를 들어, 수학식 9와 같은 소프트맥스(softmax) 함수를 이용하여 정규화된 값으로 산출될 수 있다. Specifically, the i-th hidden state vector of the encoder 310 at the current output time j

attention weight for

may be calculated as a normalized value using, for example, a softmax function as in Equation 9.

[Equation 9]

는 현재 출력 시점 j에서 인코더(310)의 i번째 은닉 상태 벡터

에 대한 주의 집중 스코어(attention score)이며, 이전 출력 시점에서 디코더(330)의 은닉 상태 벡터

와 인코더(310)의 i번째 은닉 상태 벡터

가 얼마나 유사한지를 나타낸다. 예를 들어, 주의 집중 스코어는 아래의 수학식 10에 의해 산출될 수 있다.In Equation 9, T _x represents the number of token values input to the decoder 330 . Also,

is the i-th hidden state vector of the encoder 310 at the current output time j

is the attention score for , and the hidden state vector of the decoder 330 at the previous output time.

and the i-th hidden state vector of the encoder 310

indicates how similar they are. For example, the attention score may be calculated by Equation 10 below.

[Equation 10]

는 스코어 함수를 나타낸다. 이때, 스코어 함수는 예를

와

를 입력으로 하고, tanh를 활성화 함수로 이용하는 앞 먹임 신경망(Feed-Forward Neural Network)로 구현될 수 있다. 그러나, 스코어 함수로서 앞 먹임 신경망 외에도

와

사이의 내적(dot product) 내지는 쌍선형 함수(bilinear function)가 이용될 수도 있다.in Equation 10

represents the score function. In this case, the score function is

Wow

It can be implemented as a Feed-Forward Neural Network using tanh as an input and tanh as an activation function. However, in addition to the forward-feeding neural network as a score function,

Wow

A dot product or a bilinear function between may be used.

Meanwhile, the decoder 330 may receive the hidden state vector of the decoder 330 at the previous output time, the token value output at the previous output time, and the context vector at the current output time, and determine the token value to be output at the current output time. .

는 아래의 수학식 11과 같이 산출될 수 있다.At this time, the context vector input to the decoder 330 at the current output time j

can be calculated as in Equation 11 below.

[Equation 11]

는 아래의 수학식 12와 같이 표현될 수 있다.In addition, the hidden state vector of the decoder 330 at the current output time j

can be expressed as in Equation 12 below.

[Equation 12]

4 is a flowchart of a chat model learning method according to an embodiment.

The method shown in FIG. 4 may be performed, for example, by the chat model learning apparatus 100 shown in FIG. 1 .

Referring to FIG. 4 , first, the chat model learning apparatus 100 trains the chat model using an unsupervised learning technique using a plurality of sentences as learning data ( 410 ).

In this case, according to an embodiment, the chat model learning apparatus 100 may train the chat model to generate the same sentence as the sentence input for learning.

Meanwhile, according to an embodiment, the chat model may be an encoder-decoder model or an attention-based encoder-decoder model to which an attention mechanism is applied.

Thereafter, the chat model learning apparatus 100 additionally learns the chat model learned in step 410 by using a supervised learning technique using a plurality of dialogue sentence pairs each including an utterance sentence and a response sentence as learning data ( 420 ).

At this time, according to an embodiment, the chat model learning apparatus 100 may use the weight of the chat model learned in step 410 as an initial weight for further learning.

On the other hand, in the flowchart shown in FIG. 4, the method is described by dividing the method into a plurality of steps, but it is performed together with other steps, omitted, performed divided into detailed steps, or one or more steps not shown are added. can be performed.

5 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those not described below.

The illustrated computing environment 10 includes a computing device 12 . In an embodiment, the computing device 12 may be one or more components included in the application information providing device 100 illustrated in FIG. 1 .

Computing device 12 includes at least one processor 14 , computer readable storage medium 16 , and communication bus 18 . The processor 14 may cause the computing device 12 to operate in accordance with the exemplary embodiments discussed above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16 . The one or more programs may include one or more computer-executable instructions that, when executed by the processor 14, configure the computing device 12 to perform operations in accordance with the exemplary embodiment. can be

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer readable storage medium 16 includes a set of instructions executable by the processor 14 . In one embodiment, computer-readable storage medium 16 includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other forms of storage medium accessed by computing device 12 and capable of storing desired information, or a suitable combination thereof.

Communication bus 18 interconnects various other components of computing device 12 , including processor 14 and computer readable storage medium 16 .

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . The input/output interface 22 and the network communication interface 26 are coupled to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output device 24 may include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or imaging devices. input devices, and/or output devices such as display devices, printers, speakers and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12 , and may be connected to the computing device 12 as a separate device distinct from the computing device 12 . may be

According to the disclosed embodiments, the icon of the game application displayed on the display screen of the gaming device is updated according to the game information of the user, so that the personalized icon can be displayed according to the game information of each user.

In addition, according to the disclosed embodiments, by displaying visual information corresponding to the user's game information through the icon of the game application, various information related to the game may be provided to the user through the icon of the game application, through which It can induce users to access the game.

Although the present invention has been described in detail through representative embodiments above, those of ordinary skill in the art to which the present invention pertains can make various modifications to the above-described embodiments without departing from the scope of the present invention. will understand Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims described below as well as the claims and equivalents.

10: Computing Environment
12: computing device
14: Processor
16: computer readable storage medium
18: communication bus
20: Program
22: input/output interface
24: input/output device
26: network communication interface
100: chat model training device
110: first learning unit
120: second learning unit
200, 300: chat model

Claims (10)

As a chat model learning method performed by the chat model learning device,
an operation of learning the chatting model using an unsupervised learning technique using a plurality of sentences as learning data, and learning the chatting model to output the same sentence as the inputted sentence; and
A method for learning a chatting model, comprising: additionally learning the learned chatting model using a supervised learning technique using a plurality of pairs of dialog sentences each including a spoken sentence and a response sentence as learning data. delete The method according to claim 1,
The chat model is an encoder-decoder model comprising an encoder and a decoder each configured with a recurrent neural network (RNN). 4. The method of claim 3,
The chat model is a chat model learning method that is an attention-based encoder-decoder model to which an attention mechanism is applied. The method according to claim 1,
The operation of the additional learning is a chat model learning method of further learning the learned chatting model by using the weight of the learned chatting model as an initial weight. a memory that stores one or more instructions; and
one or more processors executing the one or more instructions;
The one or more processors,
Learning the chatting model using an unsupervised learning technique using a plurality of sentences as learning data, learning the chatting model to output the same sentence as the inputted sentence,
A chat model learning apparatus for additionally learning the learned chatting model by using a supervised learning technique using a plurality of dialog sentence pairs each including an utterance sentence and a response sentence as learning data. delete 7. The method of claim 6,
The chat model is an encoder-decoder model including an encoder and a decoder configured of a recurrent neural network (RNN), respectively. 9. The method of claim 8,
The chat model is a chat model learning apparatus that is an attention-based encoder-decoder model to which an attention mechanism is applied. 7. The method of claim 6,
The one or more processors, chat model learning apparatus for further learning the learned chat model by using the weight of the learned chat model as an initial weight.

Images