KR20220142050A - Document-level sentiment classification method and apparatus based on importance of sentences - Google Patents

Abstract

A document-level emotion classification method and device based on sentence importance are disclosed. The emotion classification method according to one embodiment may include: an operation of outputting second sentence embedding vectors by reinforcing first sentence embedding vectors for each sentence included in the document; an operation of calculating the emotional importance of each sentence in the document based on the second sentence embedding vectors; an operation of generating a document embedding vector by weighted summing sentences in the document based on the emotional importance; and an operation of classifying an emotional class of the document based on the document embedding vector.

Description

Document-level sentiment classification method and apparatus based on sentence importance

The following disclosure relates to a document-level sentiment classification method and apparatus based on sentence importance.

Sentiment classification (eg, sentiment classification model) refers to software (eg, a software module) that determines whether a given text has an emotional element (subjectivity) and whether it is emotionally positive or negative.

Existing sentiment classification was mainly studied for sentence-level input, and the sentiment of the entire document was also determined for document-level input.

As a related prior art, there is Korean Patent Application Laid-Open No. 2020-0107501 (Title of the Invention: Apparatus and Method for Emotion Scoring for Psychological Counseling).

The background art described above is possessed or acquired by the inventor in the process of deriving the disclosure of the present application, and cannot necessarily be said to be a known technology disclosed to the general public prior to the present application.

Various embodiments may provide a technique for determining the sentiment of the entire input text in consideration of the emotional importance of each sentence when text (eg, text at a document level) composed of a plurality of sentences is input.

Various embodiments may improve document-level sentiment classification performance based on the emotional importance of each sentence constituting the document.

However, the technical problems are not limited to the above-described technical problems, and other technical problems may exist.

A sentiment classification method according to an embodiment includes: outputting second sentence embedding vectors by reinforcing first sentence embedding vectors for each sentence included in a document; calculating an emotional importance of each sentence in the document based on the second sentence embedding vectors; generating a document embedding vector by weighted summing sentences in the document based on the emotional importance; and classifying the sentiment class of the document based on the document embedding vector.

The outputting operation may include calculating class similarity embedding vectors indicating the relation between the sentence and each sentiment class; and concatenating the first sentence embedding vectors and the class similarity embedding vectors to reinforce the first sentence embedding vectors.

Each of the class similarity embedding vectors may include a dot product value between a sentence and each sentiment class.

The generating may include generating third sentence embedding vectors by reflecting the emotional importance in the second sentence embedding vectors; encoding the third sentence embedding vectors using respective encoders; calculating a context vector based on the output of each encoder; and generating the document embedding vector based on the context vector.

The calculating of the context vector may include calculating each attention weight based on the output of each encoder and the hidden state of the decoder; and calculating the context vector performing a weighted sum of the output of each encoder and the respective attention weights.

The generating of the document embedding vector may include generating the document embedding vector based on a hidden state of a last encoder among the encoders and the context vector.

The classifying may include: calculating a class similarity embedding vector indicating a relationship between the document embedding vector and each sentiment class; and determining the sentiment class of the document by concatenating the document embedding vector and the class similarity embedding vectors.

The class similarity embedding vector may include a dot product value between the document embedding vector and each sentiment class.

An apparatus according to one embodiment includes a memory for storing one or more instructions; and a processor for executing the instruction, wherein when the instruction is executed, the processor enhances the first sentence embedding vectors for each sentence included in the document to output second sentence embedding vectors, and the second Calculating the emotional importance of each sentence in the document based on the sentence embedding vectors, weighted summing the sentences in the document based on the emotional importance to generate a document embedding vector, and based on the document embedding vector, the document embedding vector can classify the emotional classes of

The processor may calculate class similarity embedding vectors indicating the relevance of each sentiment class to a sentence, and connect the first sentence embedding vectors and the class similarity embedding vectors to reinforce the first sentence embedding vectors.

Each of the class similarity embedding vectors may include a dot product value between a sentence and each sentiment class.

The processor generates third sentence embedding vectors by reflecting the emotional importance in the second sentence embedding vectors, encodes the third sentence embedding vectors using each encoder, and based on the output of each encoder The context vector may be calculated, and the document embedding vector may be generated based on the context vector.

The processor may calculate each attention weight based on the output of each encoder and the hidden state of the decoder, and calculate the context vector for performing a weighted sum of the output of each encoder and each attention weight.

The processor may generate the document embedding vector based on a hidden state of a last encoder among the encoders and the context vector.

The processor may calculate a class similarity embedding vector indicating the relationship between the document embedding vector and each sentiment class, and determine the sentiment class of the document by concatenating the document embedding vector and the class similarity embedding vectors.

The class similarity embedding vector may include a dot product value between the document embedding vector and each sentiment class.

1 is a diagram for explaining emotion classification.
2 illustrates a sentiment classification apparatus according to various embodiments.
3 is a diagram for explaining an example of a sentence encoder.
4 is a diagram for explaining an example of a document encoder, and FIG. 5 is a diagram for explaining an operation of the document encoder.
6 shows an example of a sentiment classifier.
7 is a schematic block diagram of a sentiment classification apparatus according to various embodiments.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the actual implementation form is not limited to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit described in the embodiments.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected” to another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted.

1 is a diagram for explaining emotion classification.

Sentiment classification (e.g. sentiment classification model, sentiment analysis model) is a natural language processing (NLP) method in which a given text (e.g. input text) is classified into predefined sentiment classes (e.g. positive, neutral, negative). It's a task. The sentiment classification model is divided into a lexicon-based model and an ML-based model.

The dictionary-based model first defines a sentiment dictionary including sentiment words and their attributes (eg, polarity and intensity). Then, the dictionary-based model predicts the sentiment class (eg, rating) of a given text using the number of sentiment words, the total strength of the sentiment words, and the maximum strength of the sentiment words. The dictionary-based model classifies a sentence into a sentiment class having the strongest polarity based on discrete information such as polarity and strength of the sentiment word. These dictionary-based models include emotional words and surrounding linguistic clues (e.g., amplifiers (e.g., very, extraordinary, most), and downtoners (e.g., slightly, somewhat, pretty) ) and negators (eg nobody, none, nothing), there is also a model that proposes a rule set for calculating the polarity of the input text.

Dictionary-based models are simple and efficient, but have limitations. First, constructing a sentiment lexicon manually is a time- and labor-intensive task. Second, although the polarity may be different depending on the application domain, each sentiment word should be assigned a fixed polarity with strength. For example, "The air conditioner is so hot" expresses a negative opinion because "hot" means "having mechanical trouble". In contrast, "The movie is so hot" expresses a positive opinion, because "hot" means "popular."

To overcome these limitations, an ML-based model is proposed. The ML-based model predicts the sentiment class of a given text based on statistical or algebraic information obtained from a large amount of training corpus (corpus). These ML-based models require large amounts of polarized annotated data for training. ML-based models include sentiment classification models based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), and DNNs. While previous ML-based models focused on sentence-level sentiment classification, recent ML-based models focus on document-level sentiment classification (eg, sentiment classification for documents containing multiple sentences).

Conventionally, document-level sentiment classification does not consider the importance of each sentence in the input text, but instead treats the input text as a collection of sentences. However, while humans determine the polarity of a document, humans read the entire document, remove ordinary sentences (eg, unimpressive sentences), and determine the final polarity based on the impressive sentences. Table 1 is an example of summarizing sentiment classification in the movie review domain.

[Table 1]

A document has only one polarity, but each sentence that makes up a document can have a different polarity. The first sentence is the opposite of the polarity of the document, and the second sentence contains no biased polarity. It can be seen that each of the third and fourth sentences had a weak and strong effect on the polarity of the document. That is, the third and fourth sentences are strong evidence to support the polarity of the entire document.

As shown in Table 1, in order to classify the emotion of the entire document, it is necessary to perform the emotion classification of the entire document based on the emotion classification result for each sentence. In other words, to effectively determine the polarity of a document, it is necessary to treat each sentence in the document with different importance.

2 illustrates a sentiment classification apparatus according to various embodiments.

The sentiment classification apparatus 100 may include a sentiment classification model 200 (eg, a document-level sentiment classification model, a document-level sentiment analysis model). The emotion classification apparatus 100 may classify (eg, determine) an emotion class of an input document (eg, including a plurality of sentences) using the emotion classification model 200 .

The emotion classification model 200 may classify the emotion class of the document based on the emotional importance of each sentence in the document. The sentiment classification model 200 may be a DNN-based sentiment classification model. The sentiment classification model 200 may include a sentence encoder 210 , a document encoder 230 , and a sentiment classifier 250 .

The sentence encoder 210 may generate a sentence embedding vector (eg, a first sentence embedding vector) of each sentence included in the document in a given document (eg, an input document) using ALBERT (eg, a lite version of BERT). have. The sentence encoder 210 may include an ALBERT (eg, a lite version of BERT) for sentence embedding. The sentence encoder 210 may strengthen the sentence embedding vector by adding an embedding vector of a sentiment class (eg, a class similarity embedding vector) to the sentence embedding vector. The sentence encoder 210 may output an enhanced sentence embedding vector (eg, a second sentence embedding vector) to the document encoder 230 .

The document encoder 230 may calculate the emotional importance of each sentence in the document (eg, the emotional importance of each sentence in determining the sentiment of the entire document) through a gate function (eg, a gate layer). . The document encoder 230 may generate a document embedding vector (eg, document embedding) by weighted summing sentences according to the calculated emotional importance. Thereafter, the document encoder 230 may enhance the document embedding vector by adding an embedding vector of a sentiment class (eg, a class similarity embedding vector) to the document embedding vector. The document encoder 230 may output an enhanced document embedding vector (eg, enhanced document embedding) to the sentiment classifier 250 .

The sentiment classifier 250 may determine the sentiment class of the input document through a fully connected neural network (FNN).

3 is a diagram for explaining an example of a sentence encoder.

Each sentence included in a document (eg, an input document) may be input to the sentence encoder 210 for each sentence. The sentence encoder 210 may include a language model 213 , a class embedding layer 215 , and a concatenation function 217 .

The language model 213 converts each sentence (eg, each input sentence) into a sentence embedding vector, and the sentence embedding vector may include contextual information. The language model 213 may be implemented with ALBERT.

는 언어 모델(213)로 입력되는 i번째 문장을 의미하며, 언어 모델(213)을 통해 각 단어가 임베딩 벡터(예: ALBERT 임베딩 벡터)로 표시(예: 표현)될 수 있다.

는 클래스 토큰(예: 분류 태스크를 위한 특수 토큰(special token))을 의미하고,

는 i번째 세퍼레이터(separator) 토큰(예: i번째 문장과 i+1번째 문장 사이의 문장 경계(sentence boundary)를 나타내는 특수 토큰(special token))일 수 있다.3,

denotes an i-th sentence input to the language model 213 , and each word may be displayed (eg, expressed) as an embedding vector (eg, ALBERT embedding vector) through the language model 213 .

means a class token (e.g. a special token for a classification task),

may be an i-th separator token (eg, a special token indicating a sentence boundary between the i-th sentence and the i+1-th sentence).

의 출력 벡터가 입력되는 각 문장의 표현(representation)을 전달할 수 있다. 문서 내 입력 문장들(예:

)은 언어 모델(213)을 통해 수학식 1과 같이 문장 임베딩 벡터들(예:

)로 변환될 수 있다.In language models such as BERT and ALBERT, an output vector of class tokens typically conveys the task-oriented meaning of an input document (e.g., a sequence of all input words), whereas in the language model 213, a separator token

A representation of each sentence to which the output vector of is input can be delivered. Input sentences in the document (e.g.

) is the sentence embedding vectors (eg, Equation 1) through the language model 213

) can be converted to

[Equation 1]

는 i번째 세퍼레이터 토큰

의 출력 벡터를 의미할 수 있다.here,

is the i-th separator token

may mean an output vector of .

를 보완하기 위해, 클래스 임베딩 레이어(215)는 수학식 2와 같이 도메인 임베딩 방식을 수행할 수 있다. 클래스 임베딩 레이어(215)는 ReLU(Rectified Linear Activation Unit) 출력 함수가 포함된 FNN(fully connected neural network)을 포함할 수 있다. 클래스 임베딩 레이어(215)는 해당 문장의 감성적 중요도(예: 문서를 고려하지 않는 문장과 감성 클래스 사이의 관련성)를 계산할 수 있다. 클래스 임베딩 레이어(215)에 의해서 계산되는 문장의 감성적 중요도는 문장과 감성 클래스들(예: 타겟 클래스들) 간의 관련도(degrees of association)

(예: 클래스 유사도 임베딩 벡터)일 수 있다.Sentence embedding vector with domain knowledge of sentiment class

In order to supplement , the class embedding layer 215 may perform a domain embedding method as shown in Equation (2). The class embedding layer 215 may include a fully connected neural network (FNN) including a Rectified Linear Activation Unit (ReLU) output function. The class embedding layer 215 may calculate the emotional importance of the corresponding sentence (eg, the relation between the sentence not considering the document and the sentiment class). The emotional importance of the sentence calculated by the class embedding layer 215 is the degree of association between the sentence and the sentiment classes (eg, target classes).

(eg class similarity embedding vector).

[Equation 2]

는 i번째 문장 임베딩 벡터

를 타겟 클래스(예: 감성 분류 태스크에서 긍정 클래스, 중립 클래스, 부정 클래스)의 벡터 공간(vector space)에 매핑하기 위한 FNN을 의미할 수 있다.

는 감성 클래스들(예: 타겟 클래스들)의 무작위로 초기화된 도메인 임베딩 벡터들로 구성된 가중치 행렬(weight matrix)을 의미할 수 있다.

는 i번째 입력 문장과 각 감성 클래스(예: 타겟 클래스) 간의 내적(inner product) 값을 포함하는 클래스 유사도 임베딩(class similarity embedding) 벡터를 의미할 수 있다. 클래스 유사도 임베딩 벡터

는 입력 문장과 감성 클래스들(예: 타겟 클래스들) 간의 관련도(degrees of association)를 나타내는 것일 수 있다.here,

is the i-th sentence embedding vector

may refer to an FNN for mapping to a vector space of a target class (eg, a positive class, a neutral class, and a negative class in a sentiment classification task).

may mean a weight matrix composed of randomly initialized domain embedding vectors of sentiment classes (eg, target classes).

may mean a class similarity embedding vector including an inner product value between the i-th input sentence and each sentiment class (eg, target class). Class Similarity Embedding Vector

may indicate degrees of association between the input sentence and sentiment classes (eg, target classes).

와 클래스 유사도 임베딩 벡터

를 연결하여(예: 추가하여) 문장 임베딩 벡터를 강화할 수 있다. 연결 함수(217)는 강화된 문장 임베딩 벡터(예: i번째 문장의 도메인-특정(domain-specific) 문장 임베딩 벡터)

를 생성하여 문서 인코더(230)로 출력할 수 있다.The link function 217 is a sentence embedding vector as shown in Equation 3

and class similarity embedding vector

can be concatenated (e.g. by adding) to strengthen the sentence embedding vector. The link function 217 is a reinforced sentence embedding vector (eg, a domain-specific sentence embedding vector of the i-th sentence).

may be generated and output to the document encoder 230 .

[Equation 3]

4 is a diagram for explaining an example of a document encoder, and FIG. 5 is a diagram for explaining an operation of the document encoder.

The document encoder 230 may include a gate layer 233 , an RNN 235 , an attention layer 237 , and a decoder 239 .

(예: 문서 전체의 감성을 판단하는데 각 문장의 감성적 중요도)를 계산하고, 문장의 감성적 중요도가 반영된 문장 임베딩 벡터(예: 제3 문장 임베딩 벡터)를 생성할 수 있다. 게이트 레이어(233)는 수학식 4와 같이 게이트 메커니즘을 이용할 수 있다.The gate layer 233 determines the emotional importance degree of sentences in the document.

(eg, the emotional importance of each sentence in determining the emotion of the entire document) may be calculated, and a sentence embedding vector (eg, the third sentence embedding vector) in which the emotional importance of the sentence is reflected may be generated. The gate layer 233 may use a gate mechanism as shown in Equation (4).

[Equation 4]

는 무작위로 초기화된 가중치 행렬을 나타내고,

는 강화된 문장 임베딩 벡터

(예: 제2 문장 임베딩 벡터)의 감성적 중요도(importance degree)

를 계산하기위한 시그모이드 함수를 나타내며,

는 감성적 중요도가 반영된 문장 임베딩 벡터(예: 제3 문장 임베딩 벡터, 게이트된 문장 임베딩 벡터)일 수 있다.here,

denotes a randomly initialized weight matrix,

is an enhanced sentence embedding vector

(e.g., the second sentence embedding vector) emotional importance (importance degree)

represents the sigmoid function for calculating

may be a sentence embedding vector (eg, a third sentence embedding vector, a gated sentence embedding vector) in which emotional importance is reflected.

The RNN 235 may include one or more GRU encoders, and may encode a sentence embedding vector (eg, a third sentence embedding vector, a gated sentence embedding vector) in which emotional importance is reflected through the GRU encoder. All sentence embedding vectors may be encoded through the GRU encoder expressed by Equation (5).

[Equation 5]

는 GRU 인코더의 순방향 상태(forward state)로 인코딩된 i번째 인코딩된 문장 임베딩 벡터를 의미할 수 있다.here,

may mean the i-th encoded sentence embedding vector encoded in the forward state of the GRU encoder.

Then, the document encoder 230 may generate a document embedding vector using the attention layer 237 and the decoder 239 (eg, a GRU decoder). An example of the operation of the attention layer 237 and the decoder 237 is shown in FIG. 5 .

과 디코더(239)의 첫 번째 은닉 상태(hidden state)

사이에 내적(inner product)을 수행하여 각 어텐션 가중치

를 계산할 수 있다. 각 어텐션 가중치

는 GRU 인코더의 각 출력

이 디코더(239)의 첫 번째 은닉 상태

와 얼마나 관련이 있는지를 나타내는 것일 수 있다. 어텐션 레이어(237)는 어텐션 가중치

와

의 가중치 합(weighted sum)을 수행하여 컨텍스트 벡터

를 계산할 수 있다.As shown in Figure 5, the attention layer 237 is each output of the GRU encoder.

and the first hidden state of the decoder 239

Each attention weight by performing an inner product between

can be calculated. weight of each attention

is each output of the GRU encoder

The first hidden state of this decoder 239

It may indicate how much it is related to The attention layer 237 is an attention weight

Wow

Context vector by performing a weighted sum of

can be calculated.

, 시작 심볼(start symbol)

(예: 디코더(239)의 디코딩의 시작을 의미하는 값(벡터 값)), 및 컨텍스트 벡터

에 기초하여 문서 임베딩 벡터

를 생성할 수 있다. 문서 임베딩 벡터

는 수학식 6과 같이 표현될 수 있다.The decoder 239 is the last hidden state of the encoder (eg, the last GRU encoder) included in the RNN.

, start symbol

(eg, a value indicating the start of decoding of the decoder 239 (vector value)), and a context vector

Document embedding vector based on

can create Document Embedding Vector

can be expressed as Equation (6).

[Equation 6]

6 shows an example of a sentiment classifier.

는 감성 분류기(250)에 입력될 수 있다. 감성 분류기(250)는 문서 임베딩 벡터

및 클래스 유사도 임베딩 벡터에 기초하여 문서의 감성 클래스를 분류할 수 있다. 감성 분류기(250)는 클래스 임베딩 레이어(253), 및 FNN(255)를 포함할 수 있다.Document Embedding Vector

may be input to the sentiment classifier 250 . Sentiment classifier 250 is a document embedding vector

And the sentiment class of the document may be classified based on the class similarity embedding vector. The sentiment classifier 250 may include a class embedding layer 253 , and an FNN 255 .

를 계산할 수 있다. 수학식 7로 표현되는 것처럼, 클래스 유사도 임베딩 벡터는 문서 임베딩 벡터와 감성 클래스들(예: 타겟 클래스들) 간의 관련도를 나타낸다는 것일 수 있다. 클래스 유사도 임베딩 벡터가 문서 임베딩 벡터와 타겟 클래스들 간의 관련도를 나타낸다는 점을 제외하면, 클래스 유사도 임베딩 벡터의 설명은 문장 인코더(210)에서와 동일할 수 있다.The class embedding layer 253 may include a fully connected neural network (FNN) including a Rectified Linear Activation Unit (ReLU) output function. The class embedding layer 253 is a class similarity embedding vector

can be calculated. As expressed by Equation 7, the class similarity embedding vector may indicate the degree of relevance between the document embedding vector and sentiment classes (eg, target classes). The description of the class similarity embedding vector may be the same as in the sentence encoder 210 , except that the class similarity embedding vector indicates the degree of relevance between the document embedding vector and target classes.

[Equation 7]

는 문서 임베딩 벡터

를 감성 클래스들(예: 타겟 클래스)의 벡터 공간(vector space)에 매핑하기 위한 출력 함수가 포함된 FNN을 의미할 수 있다.

는 수학식 2에서와 동일한 감성 클래스들(예: 타겟 클래스들)의 무작위로 초기화된 도메인 임베딩 벡터들로 구성된 가중치 행렬일 수 있다. 즉, 감성 분류기(250)는 문장 인코더(210)와 가중치 행렬

을 공유할 수 있다.

는 문서 임베딩 벡터와 각 감성 클래스(예: 각 타겟 클래스) 사이의 내적 값(inner product value)을 포함하는 클래스 유사도 임베딩 벡터(class similarity embedding)를 나타낼 수 있다.here,

is the document embedding vector

It may refer to an FNN including an output function for mapping , to a vector space of sentiment classes (eg, a target class).

may be a weight matrix composed of randomly initialized domain embedding vectors of the same sentiment classes (eg, target classes) as in Equation (2). That is, the sentiment classifier 250 includes the sentence encoder 210 and the weight matrix.

can be shared

may represent a class similarity embedding vector including an inner product value between the document embedding vector and each sentiment class (eg, each target class).

The FNN 255 includes a sigmoid output function and may determine the sentiment class of the input document. The sentiment class may be determined as in Equation (8).

[Equation 8]

는 문서 임베딩 벡터

과 클래스 유사도 임베딩 벡터

의 연결을 나타내는 것일 수 있다.here,

is the document embedding vector

and class similarity embedding vector

may indicate a connection of

As described above, the emotion classification apparatus 100 automatically determines the emotional importance of a sentence in a document using a gate function) (eg, the gate layer 233), and considers the emotional importance of each sentence differently in the input document can be classified into a predefined sentiment class.

7 is a schematic block diagram of a sentiment classification apparatus according to various embodiments.

The emotion classification apparatus 700 (eg, the machine reading apparatus 100 of FIG. 2 ) may perform a document-level emotion classification operation in which the importance of sentences described with reference to FIGS. 2 to 6 is reflected. The machine reading apparatus 700 may include a memory 710 and a processor 730 . The sentiment classification model 200 of FIG. 2 (eg, the sentence encoder 210 , the document encoder 230 , and the sentiment classifier 250 of FIG. 2 ) is stored in the memory 710 . It may be loaded into the processor 730 and executed by the processor 730 , or may be embedded in the processor 730 .

The memory 710 may store instructions (or programs) executable by the processor 730 . For example, the instructions may include instructions for executing an operation of the processor 730 and/or an operation of each component of the processor 730 .

The processor 730 may process data stored in the memory 710 . The processor 730 may execute computer readable code (eg, software) stored in the memory 710 and instructions induced by the processor 730 .

The processor 730 may be a hardware-implemented data processing device having a circuit having a physical structure for executing desired operations. For example, desired operations may include code or instructions included in a program.

For example, a data processing device implemented as hardware includes a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor. , an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

The operation performed by the processor 730 may be substantially the same as the sentiment classification operation using the sentence encoder 210 , the document encoder 230 , and the sentiment classifier 250 described with reference to FIGS. 2 to 6 . Accordingly, a detailed description will be omitted.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA) array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using a general purpose computer or special purpose computer. The processing device may execute an operating system (OS) and a software application running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer readable medium may store program instructions, data files, data structures, etc. alone or in combination, and the program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. have. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

The hardware devices described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited drawings, a person skilled in the art may apply various technical modifications and variations based thereon. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (17)

outputting second sentence embedding vectors by reinforcing the first sentence embedding vectors for each sentence included in the document;
calculating an emotional importance of each sentence in the document based on the second sentence embedding vectors;
generating a document embedding vector by weighted summing sentences in the document based on the emotional importance; and
Classifying the sentiment class of the document based on the document embedding vector
Including, sentiment classification method.
According to claim 1,
The output operation is
calculating class similarity embedding vectors indicating the relation between the sentence and each sentiment class; and
Reinforcing the first sentence embedding vectors by concatenating the first sentence embedding vectors and the class similarity embedding vectors
Including, sentiment classification method.
3. The method of claim 2,
Each of the class similarity embedding vectors is,
A sentiment classification method, including a dot product value between a sentence and each sentiment class.
According to claim 1,
The generating operation is
generating third sentence embedding vectors by reflecting the emotional importance in the second sentence embedding vectors;
encoding the third sentence embedding vectors using respective encoders;
calculating a context vector based on the output of each encoder; and
generating the document embedding vector based on the context vector
Including, sentiment classification method.
5. The method of claim 4,
The operation of calculating the context vector is,
calculating each attention weight based on the output of each encoder and the hidden state of the decoder; and
Calculating the context vector performing a weighted sum of the output of each encoder and the respective attention weights
Including, sentiment classification method.
5. The method of claim 4,
The operation of generating the document embedding vector comprises:
generating the document embedding vector based on the context vector and the hidden state of the last encoder among the encoders
Including, sentiment classification method.
According to claim 1,
The classification operation is
calculating a class similarity embedding vector indicating a relationship between the document embedding vector and each sentiment class; and
determining the sentiment class of the document by concatenating the document embedding vector and the class similarity embedding vectors
Including, sentiment classification method.
8. The method of claim 7,
The class similarity embedding vector is,
Sentiment classification method comprising a dot product value between the document embedding vector and each sentiment class.
a memory storing one or more instructions; and
a processor for executing the instructions
including,
When the instruction is executed, the processor
outputting second sentence embedding vectors by strengthening the first sentence embedding vectors for each sentence included in the document,
calculating the emotional importance of each sentence in the document based on the second sentence embedding vectors,
generating a document embedding vector by weighted summing sentences in the document based on the emotional importance,
Classifying the sentiment class of the document based on the document embedding vector.
10. The method of claim 9,
The processor is
Calculate the class similarity embedding vectors indicating the relationship between the sentence and each emotional class,
concatenating the first sentence embedding vectors and the class similarity embedding vectors to reinforce the first sentence embedding vectors.
11. The method of claim 10,
Each of the class similarity embedding vectors is,
A device comprising a dot product value between a sentence and each sentiment class.
10. The method of claim 9,
The processor is
generating third sentence embedding vectors by reflecting the emotional importance to the second sentence embedding vectors;
Encoding the third sentence embedding vectors using each encoder,
Calculate a context vector based on the output of each encoder,
and generate the document embedding vector based on the context vector.
13. The method of claim 12,
The processor is
calculating each attention weight based on the output of each encoder and the hidden state of the decoder;
and calculating the context vector for performing a weighted sum of the output of the respective encoders and the respective attention weights.
13. The method of claim 12,
The processor is
and generating the document embedding vector based on the context vector and a hidden state of a last encoder among the respective encoders.
10. The method of claim 9,
The processor is
calculating a class similarity embedding vector indicating the relationship between the document embedding vector and each sentiment class,
An apparatus for determining the sentiment class of the document by concatenating the document embedding vector and the class similarity embedding vectors.
16. The method of claim 15,
The class similarity embedding vector is,
and a dot product value between the document embedding vector and each sentiment class.
A computer program stored in a computer-readable recording medium in combination with hardware to execute the method of any one of claims 1 to 8.

Images