KR102326473B1 - Text sensibility analysis apparatus and method threrof - Google Patents

Abstract

A text sentiment analysis apparatus for performing sentiment analysis on text, receiving a text composed of at least one sentence including a plurality of words, and embedding the word based on the context of the at least one sentence A construction unit for constructing a sentiment analysis model including an embedding layer that calculates a value and at least one fully connected layer that receives the calculated embedding value and outputs sentiment information for the text, and the built sentiment analysis model and an analysis unit that analyzes sentiment information on text input in real time using

Description

Text sentiment analysis apparatus and method {TEXT SENSIBILITY ANALYSIS APPARATUS AND METHOD THREROF}

The present invention relates to a text sentiment analysis apparatus and method.

Due to the development of social media and the proliferation of online product markets, the amount of text generated in real time has exploded.

Such constantly generated text is called a text stream, and analyzing such text stream to understand public opinion enables effective decision-making.

On the other hand, sentiment analysis among text mining techniques is a good analysis method that can monitor the reaction of the public in real time.

In this case, sentiment analysis for text streams having various topics from moment to moment is meaningful when model construction and improvement are performed in real time.

This requires online learning, which executes instant learning on sequentially incoming data. However, batch learning, which stores a text stream and then executes iterative learning on the stored data, is not a suitable method for text stream analysis.

In addition, online learning is a learning technique that obtains a learning model by performing an inquiry on stream-type data only once. In order for this to be possible, it is necessary to extract good features from the data and identify patterns in the data with only a single inquiry into the data.

In addition, the learning process should be performed quickly to avoid delay in processing the stream data. In other words, a deep learning model capable of online learning should have a simple structure for rapid learning while extracting good features from data.

However, considering that feature extraction and pattern learning are performed together in deep learning, there is a limitation in that it is difficult to simultaneously extract good features with a simple structure.

Therefore, context-based word embedding is required to extract good features for online learning from text. After building a model, the focus has been on how to operate it in real time.

However, this method is not suitable for a text stream environment in which the subject is constantly changing, and it also has a disadvantage in that the text needs to be stored for batch learning.

The present invention is to solve the problems of the prior art described above, by pre-learning an embedding layer that calculates an embedding value by digitizing a word, and by learning the embedding layer to calculate an embedding value for a word based on the context, the same An object of the present invention is to provide a text sentiment analysis apparatus and method capable of improving the sentiment analysis accuracy of a text mining technique by calculating different embedding values according to context even in the case of words.

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the text sentiment analysis apparatus for performing sentiment analysis on text according to an aspect of the present invention inputs text composed of at least one sentence including a plurality of words. an embedding layer for calculating an embedding value for the word based on the context of the at least one sentence or more, and at least one fully connected layer for receiving the calculated embedding value and outputting sentiment information for the text a construction unit that builds a sentiment analysis model; and an analysis unit that analyzes sentiment information for text input in real time using the built sentiment analysis model, wherein the embedding layer is pre-learned to calculate an embedding value for a word based on the context of a plurality of sentences. can

In addition, according to an aspect of the present invention, a text sentiment analysis method for performing sentiment analysis on text includes: building a sentiment analysis model; and analyzing the emotion information on the text input in real time by using the built emotion analysis model, wherein the step of constructing the emotion analysis model includes at least one sentence including a plurality of words receiving an inputted text and calculating an embedding value for the word based on the context of the at least one sentence; and receiving the calculated embedding value and outputting sentiment information for the text, wherein the calculation of the embedding value for the word is to calculate the embedding value for the word based on the context of a plurality of sentences It can be based on prior learning.

The above-described problem solving means are merely exemplary, and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description.

According to any one of the above-described problem solving means of the present invention, the present invention pre-learns an embedding layer for calculating an embedding value by digitizing a word, by learning the embedding layer to calculate an embedding value for a word based on the context. , it is possible to provide a text sentiment analysis apparatus and method capable of improving the sentiment analysis accuracy of the text mining technique by calculating different embedding values according to context even for the same word.

In addition, the present invention uses an embedding layer learned in advance to calculate an embedding value for a word based on context, and builds a sentiment analysis model that performs sentiment analysis on text, thereby performing sentiment analysis on text with a simple structure. It is possible to provide an apparatus and method for analyzing text sentiment that can be performed.

In addition, the present invention learns the emotion analysis result for each text to enable faster emotion analysis on the input text, but in the case of the pre-learned embedding layer, by excluding the emotion analysis result for each text from the learning process, more quickly It is possible to provide a text sentiment analysis apparatus and method for enabling sentiment analysis learning on text.

1 is a diagram showing the configuration of a text sentiment analysis apparatus according to an embodiment of the present invention.
2 is a diagram showing the configuration of a sentiment analysis model according to an embodiment of the present invention.
3 is a diagram illustrating the configuration of a hidden layer according to an embodiment of the present invention.
4 is a diagram showing the configuration of a hidden layer according to an embodiment of the present invention.
5 is a diagram illustrating a weight allocation method of a weight allocator according to an embodiment of the present invention.
6 is a diagram illustrating a flow of a text sentiment analysis method according to an embodiment of the present invention.
7 is a diagram illustrating a flow of a method for constructing a sentiment analysis model according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware.

Some of the operations or functions described as being performed by the terminal or device in the present specification may be instead performed by a server connected to the terminal or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal or device connected to the server.

Hereinafter, detailed contents for carrying out the present invention will be described with reference to the accompanying configuration diagram or process flow diagram.

1 is a diagram showing the configuration of a text sentiment analysis apparatus according to an embodiment of the present invention, and FIG. 2 is a diagram showing the configuration of a sentiment analysis model according to an embodiment of the present invention.

First, referring to FIG. 1 , the apparatus 100 for analyzing text sentiment according to an embodiment of the present invention may include a construction unit 110 and an analysis unit 130 .

The construction unit 110 may build a sentiment analysis model 200 to be described later in order to analyze the sentiment of the input text. In addition, the construction unit 110 may include a learning unit 111 for learning the fully connected layer 230 constituting the emotion analysis model 200 to be described later.

Here, the sentiment analysis model may be a text mining model that determines and analyzes information related to the emotion of a user who has input the text with respect to text or a word included in the text.

Next, a sentiment analysis model according to an embodiment of the present invention will be described with reference to FIG. 2 .

Referring to FIG. 2 , the sentiment analysis model 200 according to the present invention outputs sentiment information on input text, and may include an embedding layer 210 and a fully connected layer 230 .

The embedding layer 210 may receive a text composed of one or more sentences including a plurality of words, and may calculate an embedding value for the word based on the context of the at least one or more sentences.

Here, the embedding value may mean a value obtained by digitizing text so that it can be used in a computer. Meanwhile, it should be noted that the embedding value may be calculated based on calculation formulas such as count vectorization and term frequency-inverse document frequency, but is not limited thereto.

The embedding layer 210 may be pre-learned to determine the context of a sentence based on a conjunction between a sentence including a word for which an embedding value is to be calculated and another sentence positioned at at least one before and after the corresponding sentence.

Here, prior learning means that only the embedding layer 210 is learned as a separate model before the building unit 110 builds the sentiment analysis model 200 including the embedding layer 210 and the fully connected layer 230. can mean

For example, the embedding layer 210 may include "but this song is plagiarism" between the sentence "this song is good" and the sentence "this song is plagiarism" in the sentence "this song is good. It may be pre-learned to determine the context between sentences based on the conjunction of ",".

In addition, the embedding layer 210 may be pre-learned to determine the context of a sentence based on a word for which an embedding value is to be calculated and a word located in at least one of before and after the corresponding word among the sentences.

For example, the embedding layer 210 is based on the word "song" for which the embedding value is to be calculated in the sentence "This song is plagiarism" and the word "plagiarism" located after the word "song" Thus, it can be pre-learned to determine the context of the sentence.

Accordingly, even in the case of the same word, the embedding layer 210 may calculate different embedding values for the corresponding word according to the context of the sentence including the word.

On the other hand, the embedding layer 210 may include at least one hidden layer 211 consisting of four hidden layers based on a transformer.

Here, the transformer may mean a model serving as a time sequence without using Recurrent Neural Networks (RNN) or Long Short Term Memory (LSTM) cells.

That is, the embedding layer 210 may calculate embedding values for a plurality of texts in parallel using at least one or more hidden layers 211 . This will be described in more detail with reference to FIG. 3 below.

The fully connected layer 230 may receive the calculated embedding value and output sentiment information on the text. Here, the sentiment information output from the fully connected layer 230 may include, for example, positive values and negative values for text.

More specifically, the fully connected layer 230 receives the embedding value calculated on the basis of the context from the above-described embedding layer 210, and based on the received embedding value, sentiment including positive and negative values for text information can be printed.

For example, when calculating the embedding value of the word "song" from the sentence "This song is plagiarism" in the embedding layer 210 described above, the embedding layer 210 is based on the negative word "plagiarism" After determining the context of the sentence, it is possible to calculate the embedding value of the word "song", and the fully connected layer 230 receives the calculated embedding value as input, and emotional information is provided in the text including the word "song". may be output, and in this case, the output emotion information may include a negative value.

On the other hand, the fully connected layer 230 may consist of three layers that process different number of dimensions, and sequentially receive the embedding values calculated from each layer to sequentially reduce the number of dimensions of the input embedding values. have.

For example, the embedding layer 210 may calculate embedding values for 100 texts in parallel using 100 hidden layers 211 .

In addition, the embedding layer 210 may calculate an embedding value of, for example, 768 dimensions for each text, and the fully connected layer 230 receives the embedded value calculated in this way, and then creates three different layers. can be used to sequentially decrease the number of dimensions of the embedding value.

More specifically, the fully connected layer 230 may receive the 768-dimensional embedding value calculated from the embedding layer 210 and sequentially decrease the dimension of the embedding value from 64 dimensions to 16 dimensions. To this end, the fully connected layer 230 may include a first fully connected layer that reduces the dimension of the embedding value to 64 dimensions and a second fully connected layer that reduces the dimension of the embedding value to 16 dimensions.

In this case, the number of hidden layers 211 of the embedding layer 210 may be to limit the size of the data of the embedding value in consideration of the text stream environment. Accordingly, the number of hidden layers 211 of the embedding layer 210 may be varied in consideration of the text stream environment.

Returning to FIG. 1 again, when the learning unit 111 is described, the learning unit 111 completely connects the emotional information output result according to the embedding value based on the emotional information output from at least one fully connected layer 230 . It can be taught in layers.

For example, by learning the fully connected layer 230 based on the embedding value input from the fully connected layer 230 by the learning unit 111 and the emotional information output result according to the embedding value, the fully connected layer 230 . Since the received embedding value is a pre-learned embedding value, emotional information can be output more quickly.

That is, the embedding layer 210 is pre-learned to determine the context of a sentence based on a conjunction between sentences or words before and after, and may calculate an embedding value for a word based on the pre-learned context, and the fully connected layer 230 ) may output emotion information according to the input embedding value based on the result of outputting emotion information for each text learned by the learning unit 111 .

Also, referring to FIG. 1 , the analysis unit 130 may analyze emotion information on the text input in real time by using the emotion analysis model.

More specifically, the analysis unit 130 may input the text input in real time to the sentiment analysis model 200 built by the construction unit 110 , and the sentiment analysis model 200 includes the embedding layer 210 and An embedding value for a word included in the input text may be calculated using the fully connected layer 230 , and emotion information for the corresponding text may be output based on the calculated embedding value, and the analysis unit 130 is Based on the output emotion information, emotion information on the corresponding text may be analyzed.

Hereinafter, a hidden layer according to an embodiment of the present invention will be described with reference to FIG. 3 .

3 is a diagram illustrating the configuration of a hidden layer according to an embodiment of the present invention.

Referring to FIG. 3 , the hidden layer 211 constituting the embedding layer 210 may be formed of a combination of at least one or more hidden layers 300 .

Here, the hidden layer 211 according to the present invention may be composed of at least one hidden layer among hidden layers included in a text mining model preset for text mining.

For example, the preset text mining model has 12 hidden layers, and the context of a sentence is determined based on a conjunction between a sentence including a word for which an embedding value is to be calculated and another sentence positioned at least one of the front and rear of the corresponding sentence. It can be pre-trained to judge.

In this case, the hidden layer 211 according to the present invention may be composed of a combination of the last four hidden layers 300 among the 12 hidden layers.

Here, the hidden layer 211 according to the present invention is made of a combination of the last four hidden layers 300 among the 12 hidden layers of the preset text mining model, as a result of the experiment, the combination of the last four hidden layers 300 has a high sensitivity This is because the analysis accuracy was indicated.

For example, referring to Table 1, it can be seen that the hidden layer 211 according to the present invention is formed of a combination of the last four hidden layers 300, thereby exhibiting the highest sensitivity analysis accuracy.

hidden layer combination Online 1000 Online 10000 Batch 1000 Combination of the last one hidden layer 0.7796 0.8183 0.8282 Combination of the last two hidden layers 0.7831 0.8238 0.8341 Combination of the last 4 hidden layers 0.7923 0.8241 0.8365 Combination of a total of 12 hidden layers 0.7170 0.8200 0.8166

That is, the text sentiment analysis apparatus 100 according to the present invention calculates an embedding value by combining the last four hidden layers 300 of a preset text mining model, thereby constructing a sentiment analysis model capable of more accurate sentiment analysis. .

Here, each hidden layer 300 may calculate, for example, 768-dimensional embedding values, respectively. Accordingly, the embedding layer 210 may calculate a 768-dimensional embedding value for each text in parallel using the four hidden layers 300 included in the hidden layer 211 .

Hereinafter, a hidden layer according to an embodiment of the present invention will be described with reference to FIG. 4 .

4 is a diagram showing the configuration of a hidden layer according to an embodiment of the present invention.

Referring to FIG. 4 , the hidden layer 300 may include an input unit 410 , a weight assignment unit 430 , a first standardization processing unit 450 , a feed forward unit 470 , and a second standardization processing unit 490 . .

The input unit 410 may receive a plurality of words.

In this case, the input unit 410 may receive a plurality of words in parallel, and by receiving order information to be described later together, may receive the input in a state in which a plurality of words input in parallel are arranged in order.

The above-described order information will be described in more detail by the weight allocator 430 below.

Here, the order information is a periodic function and may be implemented by positional encoding.

The weight allocator 430 may generate order information between the plurality of words by learning the input weights between the plurality of words. Here, the learning of the weight by the weight allocator 430 will be described in more detail with reference to FIG. 5 below.

For example, the weight allocator 430 may learn weights for each other's words based on the relationship between the word "credit" and the word "loan", and based on the learned weight, You can create order information for words.

In this case, the generated order information may be input to the above-described input unit 410 . That is, the input unit 410, even when the words "loan", "repayment" and "credit" are input regardless of the order, based on the input order information, "loan", "repayment" input regardless of the order " and "credit" may be input by sorting in the order of "credit", "loan" and "repayment".

Hereinafter, a weight allocator according to an embodiment of the present invention will be described with reference to FIG. 5 .

5 is a diagram illustrating a weight allocation method of a weight allocator according to an embodiment of the present invention.

Referring to FIG. 5 , when the embedding value of one sentence is 512-dimensional, the weight allocator 430 according to the present invention divides it into 8 equal parts, and then uses the three weight processing units 510 , 530 , and 550 to assign 64 values, respectively. By taking the vector and processing it, and inputting the processed vector processing value to the operation unit 570 , it can be learned to calculate a weight between a plurality of words.

That is, as shown in FIG. 5, if the embedding vector of the sentence "Je suis ιtudiant" is 512-dimensional, it is divided into 8 equal parts, and then 64 vectors are handled by one weight processing unit (510, 530, 550). and input the processed vector processing value to the operation unit 570 .

Here, as shown in Equation 1 below, the calculation unit 570 does the dot product of the vector processing values of the two weight processing units 510 and 530 , and then inputs them to the activation function, and then the value output from the weight processing unit 550 . The weight can be calculated by multiplying by .

Referring back to FIG. 4 , the first standardization processing unit 450 may calculate a word value by performing a primary standardization process on a plurality of words whose weights have been learned. That is, the first standardization processing unit 450 may standardize the plurality of words input through the input unit 410 as word values usable in the computer program.

The feed forward unit 470 may transmit the standardized word value to a second standardization processing unit 490 to be described later.

In this case, the feed forward unit 470 may control the secondary standardization processing of the second standardization processing unit 490 based on the first standardized word value.

That is, when the word value processed by the second standardization of the second standardization processing unit 490 does not belong to the target value, the feed forward unit 470 may control it more quickly to prevent an error in the word value.

The second standardization processing unit 490 may perform secondary standardization processing on the first standardized word value transmitted through the feed forward unit 470 .

That is, the second standardization processing unit 490 may standardize the word value first standardized through the first standardization processing unit 450 into a word value used in an actual computer program.

Hereinafter, a text sentiment analysis method according to an embodiment of the present invention will be described with reference to FIG. 6 .

6 is a diagram illustrating a flow of a text sentiment analysis method according to an embodiment of the present invention.

Referring to FIG. 6 , in the text sentiment analysis method according to an embodiment of the present invention, first, the building unit 110 builds the sentiment analysis model 200 in step S610 .

In step S630 , the analysis unit 130 analyzes emotion information on the text input in real time using the built emotion analysis model 200 .

In the above description, steps S610 to S630 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be changed.

Hereinafter, a method of constructing an emotion model according to an embodiment of the present invention will be described with reference to FIG. 7 .

7 is a diagram illustrating a flow of a method for constructing a sentiment analysis model according to an embodiment of the present invention.

Referring to FIG. 7 , in the method for constructing a sentiment analysis model according to an embodiment of the present invention, first, in step S710, the embedding layer 210 receives a text composed of at least one sentence including a plurality of words, An embedding value for a word is calculated based on the context of at least one sentence or more.

In step S730, the fully connected layer 230 receives the calculated embedding value and outputs emotional information about the text.

In step S750, the learning unit 111 learns at least one fully connected layer 230 based on the emotional information output result according to the embedding value.

In the above description, steps S710 to S750 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be changed.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. .

100: text sentiment analysis device
110: construction unit
111: study department
130: analysis unit
200: sentiment analysis model
210: embedding layer
211: hidden layer
230: fully connected layer
300: hidden layer
410: input unit
430: weight allocator
450: first standardization processing unit
470: feed forward unit
490: second standardization processing unit

Claims (14)

In the text sentiment analysis apparatus for performing sentiment analysis on text,
Receives a text composed of one or more sentences including a plurality of words, receives an embedding layer for calculating an embedding value for the word based on the context of the at least one or more sentences, and receives the calculated embedding value, a construction unit for constructing a sentiment analysis model including at least one fully connected layer for outputting sentiment information for the text; and
Comprising an analysis unit that analyzes emotion information about the text input in real time using the built-up emotion analysis model,
The embedding layer is pre-trained to calculate an embedding value for a word based on the context of a plurality of sentences, a text sentiment analysis apparatus.
The method of claim 1,
Wherein the building unit comprises a learning unit for learning only the at least one fully connected layer so that the at least one fully connected layer outputs the emotional information for the text.
The method of claim 1,
The embedding layer is pre-learned to determine the context of the sentence based on a conjunction between the sentence including the word for which the embedding value is to be calculated and the sentence positioned at least one before and after the sentence. Sentiment Analysis Device.
The method of claim 1,
The embedding layer is pre-learned to determine the context of the sentence based on the word for which the embedding value is to be calculated among the sentences and the word positioned before and after at least one of the word, the text sentiment analysis apparatus .
The method of claim 1,
The embedding layer calculates the embedding value, which is a value obtained by digitizing the word using any one of count vectorization and term frequency-inverse document frequency, text sentiment analysis apparatus .
The method of claim 1,
The embedding layer will include at least one or more hidden layers consisting of four transformer-based hidden layers, the text sentiment analysis apparatus.
7. The method of claim 6,
The hidden layer may include an input unit for receiving the plurality of words;
a weight allocator for learning weights between the plurality of words; and
a first standardization processing unit for performing primary standardization of the plurality of words whose weights have been learned; and
The text sentiment analysis apparatus comprising a feed forward unit for transmitting word values for the plurality of standardized words.
8. The method of claim 7,
The weight allocator learns a weight between the plurality of words to generate order information between the plurality of words,
Wherein the input unit additionally receives the order information between the plurality of words, text sentiment analysis apparatus.
8. The method of claim 7,
The hidden layer further includes a second standardization processing unit that performs secondary normalization processing on the first standardized word value transmitted through the feed forward unit,
Wherein the feed forward unit controls the secondary standardization processing of the second standardization processing unit based on the first standardized word value, text sentiment analysis apparatus.
7. The method of claim 6,
The hidden layer is a text sentiment analysis apparatus that parallelly processes a plurality of texts input using the four hidden layers.
A text sentiment analysis method for performing sentiment analysis on text performed by a text sentiment analysis apparatus, the method comprising:
building a sentiment analysis model; and
Including the step of analyzing sentiment information about the text input in real time using the built-up sentiment analysis model,
The step of constructing the sentiment analysis model is,
receiving a text composed of one or more sentences including a plurality of words, and calculating an embedding value for the word based on a context of the at least one or more sentences; and
receiving the calculated embedding value and outputting emotional information about the text;
The calculation of the embedding value for the word is based on dictionary learning to calculate the embedding value for the word based on the context of a plurality of sentences.
12. The method of claim 11,
Wherein the constructing comprises learning only the at least one fully connected layer so that the at least one fully connected layer outputs sentiment information for the text.
12. The method of claim 11,
Calculation of the embedding value for the word is a conjunction between the sentence including the word for which the embedding value is to be calculated, and a sentence positioned at least one of front and back of the sentence or the word for which the embedding value is to be calculated; The text sentiment analysis method, which is based on dictionary learning to determine the context of the sentence based on a word positioned at least one of the front and rear of the word.
12. The method of claim 11,
Calculating the embedding value for the word includes calculating the embedding value for the word using at least one or more hidden layers including four hidden layers based on a transformer,
The hidden layer receives a plurality of words from the text and additionally receives order information between the plurality of words,
learning a weight between the plurality of words, generating order information between the plurality of words,
Primary standardization processing of the plurality of words whose weights have been learned,
transfer the word values for the plurality of standardized words,
A text sentiment analysis method of secondary standardization of the first standardized word value.

Images