KR102501869B1 - Document-level sentiment classification method and apparatus based on importance of sentences - Google Patents

Abstract

A document-level sentiment classification method and apparatus based on sentence importance are disclosed. A sentiment classification method according to an embodiment includes an operation of reinforcing first sentence embedding vectors for each sentence included in a document and outputting second sentence embedding vectors; calculating an emotional importance of each sentence in the document based on the second sentence embedding vectors; generating a document embedding vector by weighted summing sentences in the document based on the emotional importance; and classifying a sentiment class of the document based on the document embedding vector.

Description

Document-level sentiment classification method and apparatus based on sentence importance

The disclosure below relates to a method and apparatus for classifying emotion at a document level based on sentence importance.

Sentiment classification (e.g., sentiment classification model) refers to software (e.g., software module) that determines whether a given text has emotional elements (subjectivity) and emotionally positive or negative polarity.

Existing sentiment classification was mainly studied for sentence-level input, and the sentiment of the entire document was also determined for document-level input.

As a related prior art, there is Korean Patent Publication No. 2020-0107501 (Title of Invention: Emotion Scoring Apparatus and Method for Psychological Counseling).

The background art described above is possessed or acquired by the inventor in the process of deriving the disclosure of the present application, and cannot necessarily be said to be known art disclosed to the general public prior to the present application.

Various embodiments may provide a technique for determining the emotion of the entire input text by considering the emotional importance of each sentence when a text composed of a plurality of sentences (eg, document-level text) is input.

Various embodiments may improve document-level sentiment classification performance based on the emotional importance of each sentence constituting the document.

However, the technical challenges are not limited to the above-described technical challenges, and other technical challenges may exist.

A sentiment classification method according to an embodiment includes an operation of reinforcing first sentence embedding vectors for each sentence included in a document and outputting second sentence embedding vectors; calculating an emotional importance of each sentence in the document based on the second sentence embedding vectors; generating a document embedding vector by weighted summing sentences in the document based on the emotional importance; and classifying a sentiment class of the document based on the document embedding vector.

The outputting may include calculating class similarity embedding vectors representing a relation between a sentence and each emotion class; and strengthening the first sentence embedding vectors by connecting the first sentence embedding vectors and the class similarity embedding vectors.

Each of the class similarity embedding vectors may include a dot product value between a sentence and each emotion class.

The generating may include generating third sentence embedding vectors by reflecting the emotional importance on the second sentence embedding vectors; encoding the third sentence embedding vectors using respective encoders; calculating a context vector based on the output of each encoder; and generating the document embedding vector based on the context vector.

The calculating of the context vector may include calculating each attention weight based on the output of each encoder and the hidden state of the decoder; and calculating the context vector for performing a weighted sum of the output of each encoder and each attention weight.

The generating of the document embedding vector may include generating the document embedding vector based on a concealment state of a last encoder among the respective encoders and the context vector.

The classifying operation may include calculating a class similarity embedding vector indicating a relation between the document embedding vector and each emotion class; and determining a sentiment class of the document by connecting the document embedding vector and the class similarity embedding vectors.

The class similarity embedding vector may include a dot product value between the document embedding vector and each emotion class.

An apparatus according to one embodiment includes a memory for storing one or more instructions; and a processor configured to execute the instruction, when the instruction is executed, the processor reinforces first sentence embedding vectors for each sentence included in the document and outputs second sentence embedding vectors, and the second sentence embedding vectors are output. Calculate the emotional importance of each sentence in the document based on sentence embedding vectors, generate a document embedding vector by weighted summing sentences in the document based on the emotional importance, and generate the document embedding vector based on the document embedding vector can be classified into emotional classes.

The processor may calculate class similarity embedding vectors representing a relationship between a sentence and each emotion class, and reinforce the first sentence embedding vectors by connecting the first sentence embedding vectors and the class similarity embedding vectors.

Each of the class similarity embedding vectors may include a dot product value between a sentence and each emotion class.

The processor generates third sentence embedding vectors by reflecting the emotional importance on the second sentence embedding vectors, encodes the third sentence embedding vectors using each encoder, and based on the output of each encoder, A context vector may be calculated, and the document embedding vector may be generated based on the context vector.

The processor may calculate each attention weight based on the output of each encoder and the hidden state of the decoder, and calculate the context vector for performing a weighted sum of the output of each encoder and the attention weight.

The processor may generate the document embedding vector based on the hidden state of the last encoder among the respective encoders and the context vector.

The processor may calculate a class similarity embedding vector indicating a relationship between the document embedding vector and each sentiment class, and determine the sentiment class of the document by connecting the document embedding vector and the class similarity embedding vectors.

The class similarity embedding vector may include a dot product value between the document embedding vector and each emotion class.

1 is a diagram for explaining emotion classification.
2 illustrates an emotion classification device according to various embodiments.
3 is a diagram for explaining an example of a sentence encoder.
4 is a diagram for explaining an example of a document encoder, and FIG. 5 is a diagram for explaining an operation of the document encoder.
6 shows an example of a sentiment classifier.
7 shows a schematic block diagram of an emotion classification device according to various embodiments.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be changed and implemented in various forms. Therefore, the form actually implemented is not limited only to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea described in the embodiments.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted.

1 is a diagram for explaining emotion classification.

Sentiment classification (e.g. sentiment classification model, sentiment analysis model) is a natural language processing (NLP) method in which a given text (e.g. input text) is classified into a predefined sentiment class (e.g. positive, neutral, negative). It is a task. Sentiment classification models are divided into lexicon-based models and ML-based models.

The dictionary-based model first defines a sentiment dictionary containing sentiment words and their properties (eg, polarity and intensity). Then, the dictionary-based model predicts the sentiment class (eg, rating) of a given text using the number of sentiment words, the total strength of sentiment words, and the maximum strength of sentiment words. The dictionary-based model classifies sentences into a sentiment class having the strongest polarity based on discrete information such as polarity and strength of sentiment words. These dictionary-based models include emotional words and surrounding linguistic clues (e.g., amplifiers (e.g., very, extraordinary, most), and meaning-reducing adverbs (e.g., slightly, somewhat, pretty). ), there is also a model that proposes a rule set for calculating the polarity of input text based on negators (e.g., nobody, none, nothing).

Dictionary-based models are simple and efficient, but have limitations. First, constructing a sentiment lexicon manually is a time- and labor-intensive task. Second, although the polarity may be different depending on the application domain, each sentiment word should be assigned a fixed polarity with strength. For example, "The air conditioner is so hot" expresses a negative opinion because "hot" means "having mechanical trouble". In contrast, "The movie is so hot" expresses a positive opinion, because "hot" means "popular".

To overcome these limitations, ML-based models have been proposed. ML-based models predict the sentiment class of a given text based on statistical or algebraic information obtained from a large amount of training corpus. These ML-based models require large amounts of polarized annotated data to train. ML-based models include sentiment classification models based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), and DNNs. Previous ML-based models focused on sentence-level sentiment classification, but recent ML-based models focus on document-level sentiment classification (e.g., sentiment classification for documents containing multiple sentences).

Conventionally, sentiment classification at the document level does not consider the importance of each sentence in the input text, but instead treats the input text as a collection of sentences. However, while a human determines the polarity of a document, the human reads the entire document, removes ordinary sentences (eg unimpressive sentences) and determines the final polarity based on the impressive sentences. Table 1 is an example summarizing sentiment classification in the movie review domain.

[Table 1]

A document has only one polarity, but each sentence comprising a document can have a different polarity. The first sentence is opposite the polarity of the document, the second sentence does not contain the biased polarity. It can be seen that the third and fourth sentences have weak and strong effects on the polarity of the document, respectively. That is, sentences 3 and 4 are strong evidence supporting the polarity of the entire document.

As shown in Table 1, in order to classify the emotion of the entire document, the emotion classification of the entire document must be performed based on the emotion classification result for each sentence. That is, to effectively determine the polarity of a document, it is necessary to treat each sentence in the document with a different importance.

2 illustrates an emotion classification device according to various embodiments.

The emotion classification device 100 may include the emotion classification model 200 (eg, a document-level emotion classification model, a document-level emotion analysis model). The emotion classification apparatus 100 may classify (eg, determine) a sentiment class of an input document (eg, including a plurality of sentences) using the emotion classification model 200 .

The emotion classification model 200 may classify the emotion class of the document based on the emotional importance of each sentence in the document. The emotion classification model 200 may be a DNN-based emotion classification model. The sentiment classification model 200 may include a sentence encoder 210 , a document encoder 230 , and a sentiment classifier 250 .

Sentence encoder 210 may use ALBERT (eg, a light version of BERT) to generate a sentence embedding vector (eg, first sentence embedding vector) of each sentence contained in the document in a given document (eg, input document). there is. The sentence encoder 210 may include ALBERT (eg, a light version of BERT) for sentence embedding. The sentence encoder 210 may enhance the sentence embedding vector by adding an embedding vector of a sentiment class (eg, a class similarity embedding vector) to the sentence embedding vector. The sentence encoder 210 may output an enhanced sentence embedding vector (eg, a second sentence embedding vector) to the document encoder 230 .

The document encoder 230 may calculate the emotional importance (eg, the emotional importance of each sentence in determining the sentiment of the entire document) of each sentence in the document through a gate function (eg, gate layer). . The document encoder 230 may generate a document embedding vector (eg, document embedding) by weighted summing sentences according to the calculated emotional importance. After that, the document encoder 230 may enhance the document embedding vector by adding an embedding vector of a sentiment class (eg, a class similarity embedding vector) to the document embedding vector. The document encoder 230 may output an enhanced document embedding vector (eg, enhanced document embedding) to the sentiment classifier 250 .

The sentiment classifier 250 may determine the sentiment class of the input document through a fully connected neural network (FNN).

3 is a diagram for explaining an example of a sentence encoder.

Each sentence included in the document (eg, the input document) may be input to the sentence encoder 210 sentence by sentence. The sentence encoder 210 may include a language model 213 , a class embedding layer 215 , and a concatenation function 217 .

The language model 213 converts each sentence (eg, each input sentence) into a sentence embedding vector, and the sentence embedding vector may include contextual information. The language model 213 may be implemented with ALBERT.

는 언어 모델(213)로 입력되는 i번째 문장을 의미하며, 언어 모델(213)을 통해 각 단어가 임베딩 벡터(예: ALBERT 임베딩 벡터)로 표시(예: 표현)될 수 있다.

는 클래스 토큰(예: 분류 태스크를 위한 특수 토큰(special token))을 의미하고,

는 i번째 세퍼레이터(separator) 토큰(예: i번째 문장과 i+1번째 문장 사이의 문장 경계(sentence boundary)를 나타내는 특수 토큰(special token))일 수 있다.In Figure 3,

denotes the ith sentence input to the language model 213, and each word may be displayed (eg, expressed) as an embedding vector (eg, ALBERT embedding vector) through the language model 213.

denotes a class token (e.g., a special token for a classification task),

may be an i-th separator token (eg, a special token indicating a sentence boundary between the i-th sentence and the i+1-th sentence).

의 출력 벡터가 입력되는 각 문장의 표현(representation)을 전달할 수 있다. 문서 내 입력 문장들(예:

)은 언어 모델(213)을 통해 수학식 1과 같이 문장 임베딩 벡터들(예:

)로 변환될 수 있다.In language models such as BERT and ALBERT, an output vector of class tokens usually conveys the task-oriented semantics of an input document (e.g., a sequence of all input words), whereas in language model 213 a separator token

The output vector of can deliver the representation of each input sentence. Input sentences in the document (e.g.

) is the sentence embedding vectors (eg, Equation 1) through the language model 213.

) can be converted to

[Equation 1]

는 i번째 세퍼레이터 토큰

의 출력 벡터를 의미할 수 있다.here,

is the ith separator token

It can mean the output vector of

를 보완하기 위해, 클래스 임베딩 레이어(215)는 수학식 2와 같이 도메인 임베딩 방식을 수행할 수 있다. 클래스 임베딩 레이어(215)는 ReLU(Rectified Linear Activation Unit) 출력 함수가 포함된 FNN(fully connected neural network)을 포함할 수 있다. 클래스 임베딩 레이어(215)는 해당 문장의 감성적 중요도(예: 문서를 고려하지 않는 문장과 감성 클래스 사이의 관련성)를 계산할 수 있다. 클래스 임베딩 레이어(215)에 의해서 계산되는 문장의 감성적 중요도는 문장과 감성 클래스들(예: 타겟 클래스들) 간의 관련도(degrees of association)

(예: 클래스 유사도 임베딩 벡터)일 수 있다.Sentence embedding vector with domain knowledge of sentiment class

To complement , the class embedding layer 215 may perform a domain embedding method as shown in Equation 2. The class embedding layer 215 may include a fully connected neural network (FNN) including a Rectified Linear Activation Unit (ReLU) output function. The class embedding layer 215 may calculate an emotional importance (eg, a relationship between a sentence not considering a document and an emotional class) of a corresponding sentence. The emotional importance of the sentence calculated by the class embedding layer 215 is the degree of association between the sentence and sentiment classes (eg, target classes).

(eg, class similarity embedding vector).

[Equation 2]

는 i번째 문장 임베딩 벡터

를 타겟 클래스(예: 감성 분류 태스크에서 긍정 클래스, 중립 클래스, 부정 클래스)의 벡터 공간(vector space)에 매핑하기 위한 FNN을 의미할 수 있다.

는 감성 클래스들(예: 타겟 클래스들)의 무작위로 초기화된 도메인 임베딩 벡터들로 구성된 가중치 행렬(weight matrix)을 의미할 수 있다.

는 i번째 입력 문장과 각 감성 클래스(예: 타겟 클래스) 간의 내적(inner product) 값을 포함하는 클래스 유사도 임베딩(class similarity embedding) 벡터를 의미할 수 있다. 클래스 유사도 임베딩 벡터

는 입력 문장과 감성 클래스들(예: 타겟 클래스들) 간의 관련도(degrees of association)를 나타내는 것일 수 있다.here,

is the ith sentence embedding vector

It may mean an FNN for mapping to a vector space of a target class (eg, a positive class, a neutral class, and a negative class in a sentiment classification task).

may denote a weight matrix composed of randomly initialized domain embedding vectors of emotion classes (eg, target classes).

may denote a class similarity embedding vector including an inner product value between the i-th input sentence and each emotion class (eg, target class). class similarity embedding vector

may represent degrees of association between an input sentence and emotion classes (eg, target classes).

와 클래스 유사도 임베딩 벡터

를 연결하여(예: 추가하여) 문장 임베딩 벡터를 강화할 수 있다. 연결 함수(217)는 강화된 문장 임베딩 벡터(예: i번째 문장의 도메인-특정(domain-specific) 문장 임베딩 벡터)

를 생성하여 문서 인코더(230)로 출력할 수 있다.The link function 217 is a sentence embedding vector as shown in Equation 3

and class similarity embedding vector

You can strengthen the sentence embedding vector by concatenating (e.g., adding). The link function 217 is an enhanced sentence embedding vector (e.g., domain-specific sentence embedding vector of the ith sentence).

can be generated and output to the document encoder 230.

[Equation 3]

4 is a diagram for explaining an example of a document encoder, and FIG. 5 is a diagram for explaining an operation of the document encoder.

The document encoder 230 may include a gate layer 233, an RNN 235, an attention layer 237, and a decoder 239.

(예: 문서 전체의 감성을 판단하는데 각 문장의 감성적 중요도)를 계산하고, 문장의 감성적 중요도가 반영된 문장 임베딩 벡터(예: 제3 문장 임베딩 벡터)를 생성할 수 있다. 게이트 레이어(233)는 수학식 4와 같이 게이트 메커니즘을 이용할 수 있다.The gate layer 233 determines the emotional importance of sentences in the document.

(eg, the emotional importance of each sentence in determining the sentiment of the entire document) may be calculated, and a sentence embedding vector (eg, a third sentence embedding vector) reflecting the emotional importance of the sentence may be generated. The gate layer 233 may use a gate mechanism as shown in Equation 4.

[Equation 4]

는 무작위로 초기화된 가중치 행렬을 나타내고,

는 강화된 문장 임베딩 벡터

(예: 제2 문장 임베딩 벡터)의 감성적 중요도(importance degree)

를 계산하기위한 시그모이드 함수를 나타내며,

는 감성적 중요도가 반영된 문장 임베딩 벡터(예: 제3 문장 임베딩 벡터, 게이트된 문장 임베딩 벡터)일 수 있다.here,

denotes a randomly initialized weight matrix,

is the enhanced sentence embedding vector

(e.g. second sentence embedding vector) emotional importance (importance degree)

Represents the sigmoid function for calculating

may be a sentence embedding vector in which emotional importance is reflected (eg, a third sentence embedding vector, a gated sentence embedding vector).

The RNN 235 includes one or more GRU encoders, and may encode a sentence embedding vector (eg, a third sentence embedding vector, a gated sentence embedding vector) in which emotional importance is reflected through the GRU encoder. All sentence embedding vectors can be encoded through the GRU encoder represented by Equation 5.

[Equation 5]

는 GRU 인코더의 순방향 상태(forward state)로 인코딩된 i번째 인코딩된 문장 임베딩 벡터를 의미할 수 있다.here,

May mean the i th encoded sentence embedding vector encoded in the forward state of the GRU encoder.

Then, the document encoder 230 may generate a document embedding vector using the attention layer 237 and the decoder 239 (eg, a GRU decoder). An example of the operation of the attention layer 237 and the decoder 237 is shown in FIG. 5 .

과 디코더(239)의 첫 번째 은닉 상태(hidden state)

사이에 내적(inner product)을 수행하여 각 어텐션 가중치

를 계산할 수 있다. 각 어텐션 가중치

는 GRU 인코더의 각 출력

이 디코더(239)의 첫 번째 은닉 상태

와 얼마나 관련이 있는지를 나타내는 것일 수 있다. 어텐션 레이어(237)는 어텐션 가중치

와

의 가중치 합(weighted sum)을 수행하여 컨텍스트 벡터

를 계산할 수 있다.As shown in Figure 5, the attention layer 237 is each output of the GRU encoder

and the first hidden state of the decoder 239

Each attention weight by performing an inner product between

can be calculated. Each Attention Weight

is each output of the GRU encoder

The first hidden state of this decoder 239

It may indicate how much it is related to Attention layer 237 is the attention weight

and

The context vector by performing a weighted sum of

can be calculated.

, 시작 심볼(start symbol)

(예: 디코더(239)의 디코딩의 시작을 의미하는 값(벡터 값)), 및 컨텍스트 벡터

에 기초하여 문서 임베딩 벡터

를 생성할 수 있다. 문서 임베딩 벡터

는 수학식 6과 같이 표현될 수 있다.The decoder 239 is the last hidden state of the encoder (eg, the last GRU encoder) included in the RNN

, start symbol

(Example: a value indicating the start of decoding of the decoder 239 (vector value)), and a context vector

Document embedding vector based on

can create Document embedding vector

Can be expressed as in Equation 6.

[Equation 6]

6 shows an example of a sentiment classifier.

는 감성 분류기(250)에 입력될 수 있다. 감성 분류기(250)는 문서 임베딩 벡터

및 클래스 유사도 임베딩 벡터에 기초하여 문서의 감성 클래스를 분류할 수 있다. 감성 분류기(250)는 클래스 임베딩 레이어(253), 및 FNN(255)를 포함할 수 있다.Document embedding vector

may be input to the sentiment classifier 250. Sentiment classifier 250 is a document embedding vector

And based on the class similarity embedding vector, the sentiment class of the document may be classified. The sentiment classifier 250 may include a class embedding layer 253 and an FNN 255 .

를 계산할 수 있다. 수학식 7로 표현되는 것처럼, 클래스 유사도 임베딩 벡터는 문서 임베딩 벡터와 감성 클래스들(예: 타겟 클래스들) 간의 관련도를 나타낸다는 것일 수 있다. 클래스 유사도 임베딩 벡터가 문서 임베딩 벡터와 타겟 클래스들 간의 관련도를 나타낸다는 점을 제외하면, 클래스 유사도 임베딩 벡터의 설명은 문장 인코더(210)에서와 동일할 수 있다.The class embedding layer 253 may include a fully connected neural network (FNN) including a Rectified Linear Activation Unit (ReLU) output function. The class embedding layer 253 is a class similarity embedding vector

can be calculated. As expressed by Equation 7, the class similarity embedding vector may indicate a degree of relationship between the document embedding vector and emotion classes (eg, target classes). The description of the class similarity embedding vector may be the same as in the sentence encoder 210, except that the class similarity embedding vector represents the degree of relationship between the document embedding vector and the target classes.

[Equation 7]

는 문서 임베딩 벡터

를 감성 클래스들(예: 타겟 클래스)의 벡터 공간(vector space)에 매핑하기 위한 출력 함수가 포함된 FNN을 의미할 수 있다.

는 수학식 2에서와 동일한 감성 클래스들(예: 타겟 클래스들)의 무작위로 초기화된 도메인 임베딩 벡터들로 구성된 가중치 행렬일 수 있다. 즉, 감성 분류기(250)는 문장 인코더(210)와 가중치 행렬

을 공유할 수 있다.

는 문서 임베딩 벡터와 각 감성 클래스(예: 각 타겟 클래스) 사이의 내적 값(inner product value)을 포함하는 클래스 유사도 임베딩 벡터(class similarity embedding)를 나타낼 수 있다.here,

is the document embedding vector

It may mean an FNN including an output function for mapping to a vector space of emotion classes (eg, a target class).

may be a weight matrix composed of randomly initialized domain embedding vectors of the same emotion classes (eg, target classes) as in Equation 2. That is, the sentiment classifier 250 includes the sentence encoder 210 and the weight matrix.

can share

may represent a class similarity embedding including an inner product value between the document embedding vector and each sentiment class (eg, each target class).

The FNN 255 includes a sigmoid output function and can determine the sentiment class of the input document. The emotion class may be determined as shown in Equation 8.

[Equation 8]

는 문서 임베딩 벡터

과 클래스 유사도 임베딩 벡터

의 연결을 나타내는 것일 수 있다.here,

is the document embedding vector

and class similarity embedding vector

It may indicate a connection of

As described above, the emotion classification apparatus 100 automatically determines the emotional importance of sentences in a document using a gate function (e.g., the gate layer 233), and considers the emotional importance of each sentence differently in the input document. can be classified into predefined emotion classes.

7 shows a schematic block diagram of an emotion classification device according to various embodiments.

The emotion classification device 700 (eg, the machine reading device 100 of FIG. 2 ) may perform a document-level emotion classification operation reflecting the sentence importance described with reference to FIGS. 2 to 6 . The machine reading device 700 may include a memory 710 and a processor 730 . The emotion classification model 200 of FIG. 2 (eg, the sentence encoder 210, the document encoder 230, and the emotion classifier 250 of FIG. 2 ) is stored in the memory 710 . It may be loaded into the processor 730 and executed by the processor 730, or may be embedded in the processor 730.

Memory 710 may store instructions (or programs) executable by processor 730 . For example, the instructions may include instructions for executing an operation of the processor 730 and/or an operation of each component of the processor 730 .

The processor 730 may process data stored in the memory 710 . The processor 730 may execute computer readable code (eg, software) stored in the memory 710 and instructions triggered by the processor 730 .

The processor 730 may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, desired operations may include codes or instructions included in a program.

For example, a data processing unit implemented in hardware includes a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor. , Application-Specific Integrated Circuit (ASIC), and Field Programmable Gate Array (FPGA).

An operation performed by the processor 730 may be substantially the same as the emotion classification operation using the sentence encoder 210, the document encoder 230, and the emotion classifier 250 described with reference to FIGS. 2 to 6 . Accordingly, detailed descriptions will be omitted.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. A computer readable medium may store program instructions, data files, data structures, etc. alone or in combination, and program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in the art of computer software. there is. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (17)

In the operation method of the emotion classification device,
outputting second sentence embedding vectors by reinforcing first sentence embedding vectors for each sentence included in the document by the sentiment classification apparatus;
calculating, by the emotion classifier, emotional importance of each sentence in the document based on the second sentence embedding vectors through a gate function;
generating a document embedding vector by adding weighted sentences in the document based on the emotional importance, by the emotion classification device; and
Classifying, by the emotion classification device, the emotion class of the document based on the document embedding vector
including,
The generating operation is
generating third sentence embedding vectors by reflecting the emotional importance on the second sentence embedding vectors;
encoding the third sentence embedding vectors using respective encoders;
calculating a context vector based on the output of each encoder; and
Generating the document embedding vector based on the context vector
Including, operating method.
According to claim 1,
The outputting operation is
calculating class similarity embedding vectors indicating a relation between a sentence and each emotion class; and
An operation of reinforcing the first sentence embedding vectors by connecting the first sentence embedding vectors and the class similarity embedding vectors.
Including, operating method.
According to claim 2,
Each of the class similarity embedding vectors,
A method of operation, including an inner product value between a sentence and each emotion class.
delete According to claim 1,
The operation of calculating the context vector,
calculating each attention weight based on the output of each encoder and the hidden state of the decoder; and
An operation of calculating the context vector performing a weighted sum of the output of each encoder and each attention weight
Including, operating method.
According to claim 1,
The operation of generating the document embedding vector,
Generating the document embedding vector based on the concealment state of the last encoder among the respective encoders and the context vector
Including, operating method.
According to claim 1,
The classification operation,
calculating a class similarity embedding vector indicating a relation between the document embedding vector and each emotion class; and
An operation of determining a sentiment class of the document by connecting the document embedding vector and the class similarity embedding vectors.
Including, operating method.
According to claim 7,
The class similarity embedding vector,
And including a dot product value between the document embedding vector and each sentiment class.
a memory that stores one or more instructions; and
A processor to execute the instruction
including,
When the instruction is executed, the processor:
outputting second sentence embedding vectors by reinforcing first sentence embedding vectors for each sentence included in the document;
Calculate the emotional importance of each sentence in the document based on the second sentence embedding vectors through a gate function;
generating third sentence embedding vectors by reflecting the emotional importance on the second sentence embedding vectors;
Encoding the third sentence embedding vectors using each encoder;
Calculate a context vector based on the output of each encoder;
generate a document embedding vector based on the context vector;
Classifying a sentiment class of the document based on the document embedding vector.
According to claim 9,
the processor,
Calculate class similarity embedding vectors representing the relationship between the sentence and each emotion class;
and reinforcing the first sentence embedding vectors by concatenating the first sentence embedding vectors and the class similarity embedding vectors.
According to claim 10,
Each of the class similarity embedding vectors,
A device, containing dot products between sentences and each sentiment class.
delete According to claim 9,
the processor,
Calculate each attention weight based on the output of each encoder and the hidden state of the decoder;
The apparatus for calculating the context vector performing a weighted sum of the output of each encoder and each attention weight.
According to claim 9,
the processor,
Generating the document embedding vector based on the concealment state of a last encoder among the respective encoders and the context vector.
According to claim 9,
the processor,
Calculate a class similarity embedding vector representing the relationship between the document embedding vector and each emotion class;
and determining a sentiment class of the document by concatenating the document embedding vector and the class similarity embedding vectors.
According to claim 15,
The class similarity embedding vector,
and a dot product between the document embedding vector and each sentiment class.
A computer program stored in a computer readable recording medium to be combined with hardware to execute the method of any one of claims 1, 2, 3, and 5 to 8.

Images