KR20200137290A - Text sensibility analysis apparatus and method threrof - Google Patents

Abstract

The present invention relates to a text sentiment analysis device which performs sentiment analysis on text, which comprises: a construction unit constructing a sentiment analyzing model including an embedding layer receiving input of text composed of at least one sentence including a plurality of words, and calculating an embedding value for the word based on context of at least one sentence, and at least one complete connection layer outputting the sentiment information on the text; and an analysis unit analyzing the sentiment information on the text input in real time by using the constructed sentiment analyzing model. The embedding layer may be pre-trained to calculate the embedding value for a word based on the context of a plurality of sentences.

Description

Text sentiment analysis apparatus and method {TEXT SENSIBILITY ANALYSIS APPARATUS AND METHOD THREROF}

The present invention relates to a text sentiment analysis apparatus and method.

Due to the development of social media and the spread of the online product market, the amount of text generated in real time has exploded.

This constantly generated text is called a text stream, and analyzing the text stream to understand the opinions of the public enables effective decision-making.

On the other hand, sentiment analysis among text mining techniques is a good analysis method that can monitor public reactions in real time.

At this time, it is meaningful only when the model is built and improved in real time to analyze the sentiment of a text stream that has various topics.

For this, online learning is required to perform immediate learning on sequentially incoming data. However, batch learning in which the text stream is stored and then iterative learning is performed on the stored data is not suitable for text stream analysis.

In addition, online learning is a learning technique that obtains a learning model by performing a single inquiry on data in the form of a stream. In order for this to be possible, it is necessary to extract good features from the data and grasp the pattern of the data with only a single inquiry for the data.

In addition, the learning process must be made quickly to avoid delays in processing of stream data. That is, a deep learning model capable of online learning must have a simple structure for rapid learning while extracting good features from data.

However, considering that feature extraction and pattern learning are performed together in deep learning, there is a limitation that it is difficult to satisfy at the same time to extract good features while having a simple structure.

Therefore, in order to extract good features for online learning from text, context-based word embedding is required.In the case of deep learning research for emotional analysis in the existing stream environment, deep learning through batch learning through word embedding that does not depend on context After building a model, it has focused on how to operate it in real time.

However, this method is not suitable for a text stream environment in which the subject is constantly changing, and it has a disadvantage that texts must be stored for batch learning.

The present invention is to solve the problems of the prior art described above, by pre-training an embedding layer that calculates an embedding value by numerically converting a word, and by learning the embedding layer to calculate an embedding value for a word based on context, the same Even in the case of words, it is intended to provide a text sentiment analysis apparatus and method capable of improving the sensitivity analysis accuracy of a text mining technique by calculating different embedding values according to context.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, the text sentiment analysis device for performing sentiment analysis on text according to an aspect of the present invention inputs text composed of at least one sentence including a plurality of words Receiving, and calculating an embedding value for the word based on the context of the at least one sentence, and at least one fully connected layer receiving the calculated embedding value and outputting sentiment information for the text A construction unit for building a sentiment analysis model; And an analysis unit for analyzing emotion information for text input in real time using the built emotion analysis model, wherein the embedding layer is pre-trained to calculate embedding values for words based on the context of a plurality of sentences. I can.

In addition, a text sentiment analysis method for performing sentiment analysis on a text according to an aspect of the present invention includes: establishing a sentiment analysis model; And analyzing the sentiment information for the text input in real time using the built sentiment analysis model, wherein the step of constructing the sentiment analysis model comprises at least one sentence including a plurality of words Receiving text to be input, and calculating an embedding value for the word based on the context of the at least one sentence; And receiving the calculated embedding value and outputting sentiment information for the text, wherein the embedding value for the word is calculated to calculate the embedding value for the word based on the context of a plurality of sentences. It can be based on prior learning.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, the present invention pre-trains an embedding layer for calculating an embedding value by quantifying a word, but by learning the embedding layer to calculate an embedding value for a word based on the context. , Even in the case of the same word, it is possible to provide a text sentiment analysis apparatus and method capable of improving the sentiment analysis accuracy of the text mining technique by calculating different embedding values according to context.

In addition, the present invention uses a pre-learned embedding layer to calculate an embedding value for a word based on the context, and constructs a sentiment analysis model that performs sentiment analysis on text, thereby performing sentiment analysis on text with a simple structure. It is possible to provide an apparatus and method for analyzing text sentiment that can be performed.

In addition, the present invention learns the sentiment analysis result for each text to enable a more rapid sentiment analysis for the input text, but in the case of the pre-learned embedding layer, by excluding it from the learning process of the sentiment analysis result for each text It is possible to provide an apparatus and method for analyzing text sentiment to enable learning of sentiment analysis on text.

1 is a diagram showing the configuration of a text sentiment analysis apparatus according to an embodiment of the present invention.
2 is a diagram showing the configuration of an emotion analysis model according to an embodiment of the present invention.
3 is a diagram showing the configuration of a hidden layer according to an embodiment of the present invention.
4 is a diagram showing the configuration of a hidden layer according to an embodiment of the present invention.
5 is a diagram illustrating a weight allocation method of a weight allocation unit according to an embodiment of the present invention.
6 is a diagram illustrating a flow of a text sentiment analysis method according to an embodiment of the present invention.
7 is a diagram illustrating a flow of a method for constructing an emotion analysis model according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, or two or more units may be realized using one hardware.

In the present specification, some of the operations or functions described as being performed by the terminal or device may be performed instead by a server connected to the terminal or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal or device connected to the server.

Hereinafter, with reference to the accompanying configuration diagram or processing flow chart, specific details for the implementation of the present invention will be described.

1 is a view showing the configuration of a text sentiment analysis device according to an embodiment of the present invention, Figure 2 is a view showing the configuration of a sentiment analysis model according to an embodiment of the present invention.

First, referring to FIG. 1, the text sentiment analysis apparatus 100 according to an embodiment of the present invention may include a construction unit 110 and an analysis unit 130.

The construction unit 110 may build a sentiment analysis model 200 to be described later in order to analyze the sentiment of the input text. In addition, the construction unit 110 may include a learning unit 111 for learning the fully connected layer 230 constituting the emotion analysis model 200 to be described later.

Here, the sentiment analysis model may be a text mining model that determines and analyzes information related to the emotion of a user who has input the text for a text or a word included in the text.

Next, in the following, an emotion analysis model according to an embodiment of the present invention will be described with reference to FIG. 2.

Referring to FIG. 2, the sentiment analysis model 200 according to the present invention outputs sentiment information on an input text, and may include an embedding layer 210 and a fully connected layer 230.

The embedding layer 210 may receive text composed of at least one sentence including a plurality of words, and calculate an embedding value for a word based on the context of the at least one sentence.

Here, the embedding value may mean a value obtained by converting text to be usable in a computer. Meanwhile, it should be noted that the embedding value may be calculated based on a calculation formula such as count vectorization and term frequency-inverse document frequency, but is not limited thereto.

The embedding layer 210 may be pre-trained to determine the context of a sentence based on a conjunction between a sentence including a word for which an embedding value is to be calculated and another sentence positioned at least one of the front and back of the sentence.

Here, pre-learning means that only the embedding layer 210 is trained as a separate model before the building unit 110 builds the sentiment analysis model 200 including the embedding layer 210 and the fully connected layer 230 It can mean.

For example, the embedding layer 210 is "But this song is good. However, this song is plagiarism" between the sentence "This song is good" and the sentence "This song is plagiarism." ," may be pre-trained to determine the context between sentences based on the conjunction.

In addition, the embedding layer 210 may be pre-trained to determine the context of a sentence based on a word for which an embedding value is to be calculated and a word positioned at least one of the front and back of the corresponding word.

For example, the embedding layer 210 is based on the word "song" for which the embedding value is to be calculated in the sentence "This song is plagiarism" and the word "plagiarism" positioned after the word "song". Thus, it can be pre-learned to determine the context of the sentence.

Accordingly, even in the case of the same word, the embedding layer 210 may calculate different embedding values for the corresponding word according to the context of the sentence including the word.

On the other hand, the embedding layer 210 may include at least one or more hidden layers 211 formed of four transformer-based hidden layers.

Here, the transformer may refer to a model that does not use Recurrent Neural Networks (RNN) or Long Short Term Memory (LSTM) cells and serves as a time sequence.

That is, the embedding layer 210 may calculate embedding values for a plurality of texts in parallel using at least one hidden layer 211. This will be described in more detail with reference to FIG. 3 below.

The fully connected layer 230 may receive the calculated embedding value and output emotion information for text. Here, the sentiment information output from the fully connected layer 230 may include, for example, a positive value and a negative value for text.

More specifically, the fully connected layer 230 receives the embedding value calculated based on the context from the embedding layer 210 described above, and includes a positive value and a negative value for text based on the input embedding value. Information can be printed.

For example, when the embedding value of the word "song" is calculated from the sentence "This song is plagiarism" in the embedding layer 210 described above, the embedding layer 210 is based on the negative word "plagiarism". After determining the context of the sentence, the embedding value of the word "song" can be calculated, and the fully connected layer 230 receives the calculated embedding value, and emotional information in the text including the word "song" May be output, and in this case, a negative value may be included in the output emotion information.

On the other hand, the fully connected layer 230 may consist of three layers that process different number of dimensions, and may sequentially receive the embedding values calculated from each layer, and sequentially decrease the number of dimensions of the input embedding values. have.

For example, the embedding layer 210 may calculate embedding values for 100 texts in parallel using 100 hidden layers 211.

In addition, the embedding layer 210 may calculate an embedding value of, for example, 768 dimensions for each text, and the fully connected layer 230 receives the calculated embedding value and then selects three different layers. By using, the number of dimensions of the embedding value can be sequentially decreased.

More specifically, the fully connected layer 230 may receive an embedding value of 768 dimensions calculated from the embedding layer 210, and may sequentially decrease the dimension of the embedding value from 64 to 16 dimensions. To this end, the fully connected layer 230 may include a first fully connected layer reducing the dimension of the embedding value to 64 dimensions and a second fully connected layer reducing the embedding value to 16 dimensions.

In this case, the number of hidden layers 211 of the embedding layer 210 may be for limiting the size of data of the embedding value in consideration of the text stream environment. Accordingly, the number of hidden layers 211 of the embedding layer 210 may be varied in consideration of the text stream environment.

Returning to FIG. 1 again, when the learning unit 111 is described, the learning unit 111 completely connects the output result of emotion information according to the embedding value based on the emotion information output from at least one fully connected layer 230 Can be learned in the hierarchy.

For example, the learning unit 111 learns the fully connected layer 230 based on the embedding value input from the fully connected layer 230 and the sentiment information output result according to the embedding value, so that the fully connected layer 230 Since the inputted embedding value is a pre-learned embedding value, it is possible to output sentiment information more quickly.

That is, the embedding layer 210 is pre-learned to determine the context of a sentence based on conjunctions between sentences or words before and after the sentence, and can calculate an embedding value for a word based on the pre-learned context, and the fully connected layer 230 ) May output emotion information according to an input embedding value based on a result of outputting emotion information for each text learned by the learning unit 111.

In addition, referring to FIG. 1, the analysis unit 130 may analyze emotion information for text input in real time using an emotion analysis model.

More specifically, the analysis unit 130 may input text input in real time into the emotion analysis model 200 built by the construction unit 110, and the emotion analysis model 200 includes the embedding layer 210 and Using the fully connected layer 230, an embedding value for a word included in the input text may be calculated, and sentiment information for the corresponding text may be output based on the calculated embedding value, and the analysis unit 130 The sentiment information for the text may be analyzed based on the output sentiment information.

Hereinafter, a hidden layer according to an embodiment of the present invention will be described with reference to FIG. 3.

3 is a diagram showing the configuration of a hidden layer according to an embodiment of the present invention.

Referring to FIG. 3, the hidden layer 211 constituting the embedding layer 210 may be formed of a combination of at least one hidden layer 300.

Here, the hidden layer 211 according to the present invention may be composed of at least one hidden layer among the hidden layers included in the text mining model preset for text mining.

For example, the preset text mining model has 12 hidden layers, and the context of the sentence is determined based on the conjunction between a sentence including a word for which an embedding value is to be calculated, and another sentence positioned at least one of the front and back of the sentence. It can be pre-learned to judge.

In this case, the hidden layer 211 according to the present invention may be configured as a combination of the last four hidden layers 300 of the 12 hidden layers.

Here, the hidden layer 211 according to the present invention is composed of a combination of the last 4 hidden layers 300 among the 12 hidden layers of the preset text mining model, as a result of the experiment, the combination of the last 4 hidden layers 300 is highly emotional. This is because it showed the accuracy of the analysis.

For example, referring to Table 1, it can be seen that the hidden layer 211 according to the present invention is formed of a combination of the last four hidden layers 300, thereby showing the highest sensitivity analysis accuracy.

Hidden floor combination Online 1000 Online 10000 Batch 1000 Combination of the last one hidden layer 0.7796 0.8183 0.8282 Combination of the last two hidden layers 0.7831 0.8238 0.8341 Combination of the last 4 hidden layers 0.7923 0.8241 0.8365 Combination of a total of 12 hidden layers 0.7170 0.8200 0.8166

That is, the text sentiment analysis apparatus 100 according to the present invention may construct a sentiment analysis model capable of more accurate sentiment analysis by calculating the embedding value by combining the last four hidden layers 300 of the preset text mining model. .

Here, each of the hidden layers 300 may calculate 768-dimensional embedding values, for example. Accordingly, the embedding layer 210 may calculate 768-dimensional embedding values for each text in parallel by using the four hidden layers 300 included in the hidden layer 211.

Hereinafter, a hidden layer according to an embodiment of the present invention will be described with reference to FIG. 4.

4 is a diagram showing the configuration of a hidden layer according to an embodiment of the present invention.

Referring to FIG. 4, the hidden layer 300 may include an input unit 410, a weight allocation unit 430, a first standardization processing unit 450, a feed forward unit 470, and a second standardization processing unit 490. .

The input unit 410 may receive a plurality of words.

In this case, the input unit 410 may receive a plurality of words in parallel, but by receiving order information to be described later together, the input unit 410 may be input in a state in which a plurality of words input in parallel are arranged in order.

For the above-described order information, the weight allocation unit 430 will be described below and will be described in more detail.

Here, the order information is a periodic function and may be implemented with positional encoding.

The weight assignment unit 430 may learn weights between the input words and generate order information between the plurality of words. Here, the weight allocation unit 430 learning the weight will be described in more detail with reference to FIG. 5 below.

For example, based on the relationship between the word "credit" and the word "loan", the weight allocation unit 430 may learn weights for each other's words, and based on the learned weights, You can generate order information for words.

In this case, the generated order information may be input to the above-described input unit 410. That is, even when the words "loan", "repayment" and "credit" are input regardless of the order, the input unit 410 is based on the input order information, and the input "loan", "repayment" The words "and "credit" can be input by sorting in the order of "credit", "loan" and "repayment".

Hereinafter, a weight allocation unit according to an embodiment of the present invention will be described with reference to FIG. 5.

5 is a diagram illustrating a weight allocation method of a weight allocation unit according to an embodiment of the present invention.

Referring to FIG. 5, when the embedding value of one sentence is 512 dimensional, the weight allocation unit 430 according to the present invention divides the embedding value into 8 equal parts, and then the three weight processing units 510, 530, and 550 A vector is handled and processed, and by inputting the processed vector processing value to the operation unit 570, the learning may be performed to calculate a weight between a plurality of words.

That is, as shown in FIG. 5, if the embedding vector of the sentence "Je suis ιtudiant" is 512 dimensions, it is divided into 8 equal parts, and then 64 vectors are handled by one weight processing unit (510, 530, 550). Then, the processed vector processing value is input to the operation unit 570.

Here, as shown in Equation 1 below, the calculation unit 570 performs a dot product of the vector processing values of the two weight processing units 510 and 530, inputs it to the activation function, and then outputs the value from the weight processing unit 550 The weight can be calculated by multiplying by.

Referring back to FIG. 4, the first standardization processing unit 450 may calculate a word value by first standardizing a plurality of words whose weights have been learned. That is, the first standardization processing unit 450 may standardize a plurality of words input through the input unit 410 into word values usable in a computer program.

The feed forward unit 470 may transfer the standardized word value to the second standardization processing unit 490 to be described later.

In this case, the feed forward unit 470 may control the second normalization processing of the second normalization processing unit 490 based on the first normalized word value.

That is, the feed forward unit 470 may prevent occurrence of an error in the word value by controlling it more quickly when the word value subjected to the second normalization process by the second standardization processing unit 490 does not belong to the target value.

The second normalization processing unit 490 may perform the second normalization processing of the first normalized word values transmitted through the feed forward unit 470.

That is, the second standardization processing unit 490 may standardize a word value that has been subjected to the first standardization process through the first standardization processing unit 450 to a word value used in an actual computer program.

Hereinafter, a text sentiment analysis method according to an embodiment of the present invention will be described with reference to FIG. 6.

6 is a diagram illustrating a flow of a text sentiment analysis method according to an embodiment of the present invention.

Referring to FIG. 6, in the text sentiment analysis method according to an embodiment of the present invention, first, in step S610, the construction unit 110 constructs the sentiment analysis model 200.

In step S630, the analysis unit 130 analyzes the sentiment information for the text input in real time using the built sentiment analysis model 200.

In the above description, steps S610 to S630 may be further divided into additional steps or may be combined into fewer steps according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, or the order between steps may be changed.

Hereinafter, a method of constructing an emotion model according to an embodiment of the present invention will be described with reference to FIG. 7.

7 is a diagram illustrating a flow of a method for constructing an emotion analysis model according to an embodiment of the present invention.

Referring to FIG. 7, in the method of constructing a sentiment analysis model according to an embodiment of the present invention, first, in step S710, the embedding layer 210 receives a text composed of at least one sentence including a plurality of words, An embedding value for a word is calculated based on the context of at least one sentence.

In step S730, the fully connected layer 230 receives the calculated embedding value and outputs sentiment information for the text.

In step S750, the learning unit 111 learns at least one fully connected layer 230 based on a result of outputting emotion information according to the embedding value.

In the above description, steps S710 to S750 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, or the order between steps may be changed.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. .

100: text sentiment analysis device
110: construction
111: Learning Department
130: analysis unit
200: sentiment analysis model
210: embedding layer
211: hidden layer
230: fully connected layer
300: hidden layer
410: input
430: weight allocation unit
450: first standardization processing unit
470: feed forward unit
490: second standardization processing unit

Claims (14)

In the text sentiment analysis device for performing sentiment analysis on text,
An embedding layer for calculating an embedding value for the word and the calculated embedding value is input, receiving text consisting of at least one sentence including a plurality of words, based on the context of the at least one sentence, A construction unit for constructing a sentiment analysis model including at least one fully connected layer that outputs sentiment information for the text; And
Including an analysis unit that analyzes the sentiment information about the text input in real time by using the constructed sentiment analysis model,
The embedding layer is pre-learned to calculate an embedding value for a word based on the context of a plurality of sentences.
The method of claim 1,
The construction unit includes a learning unit for learning only the at least one fully connected layer so that the at least one fully connected layer outputs emotion information for the text.
The method of claim 1,
The embedding layer is pre-learned to determine the context of the sentence based on a conjunction between the sentence including the word for which the embedding value is to be calculated and the sentence positioned at least one of the front and back of the sentence Sentiment analysis device.
The method of claim 1,
The embedding layer is pre-learned to determine the context of the sentence based on the word for which the embedding value is to be calculated, and a word positioned at least one of the front and back of the word. .
The method of claim 1,
The embedding layer calculates the embedding value, which is a value obtained by digitizing the word using any one of count vectorization and term frequency-inverse document frequency. .
The method of claim 1,
The embedding layer includes at least one hidden layer consisting of four transformer-based hidden layers.
The method of claim 6,
The hidden layer may include an input unit receiving the plurality of words;
A weight allocation unit for learning weights between the plurality of words; And
A first standardization processing unit that performs a first standardization process on a plurality of words whose weights are learned; And
And a feed forward unit for transmitting word values for the plurality of normalized words.
The method of claim 7,
The weight allocation unit learns the weights between the plurality of words to generate order information between the plurality of words,
The input unit further receives order information between the plurality of words.
The method of claim 7,
The hidden layer further includes a second normalization processing unit for performing a second normalization processing on the first normalized word value transmitted through the feed forward unit,
Wherein the feed forward unit controls the second normalization processing of the second normalization processing unit based on the first normalized word value.
The method of claim 6,
The hidden layer is to parallel-process a plurality of texts input by using the four hidden layers.
In the text sentiment analysis method for performing sentiment analysis on text,
Building a sentiment analysis model; And
Including the step of analyzing the sentiment information for the text input in real time using the built sentiment analysis model,
Building the sentiment analysis model,
Receiving a text composed of at least one sentence including a plurality of words, and calculating an embedding value for the word based on the context of the at least one sentence; And
Receiving the calculated embedding value and outputting sentiment information for the text,
The calculation of the embedding value for the word is based on prior learning in which the embedding value for the word is calculated based on the context of a plurality of sentences.
The method of claim 11,
The constructing step comprises the step of learning only the at least one fully connected layer so that at least one fully connected layer outputs sentiment information for the text.
The method of claim 11,
The calculation of the embedding value for the word includes a conjunction between the sentence including the word for which the embedding value is to be calculated, and a sentence positioned at least one of the front and back of the sentence or the word for which the embedding value is to be calculated, The text sentiment analysis method is based on prior learning to determine the context of the sentence based on a word positioned at least one of the front and back of the word.
The method of claim 11,
The step of calculating the embedding value for the word includes calculating an embedding value for the word using at least one hidden layer consisting of four hidden layers based on a transformer,
The hidden layer receives a plurality of words from the text, and additionally receives order information between the plurality of words,
Learning weights between the plurality of words, generating order information between the plurality of words,
First-order standardization processing of a plurality of words whose weights are learned,
Delivering word values for the plurality of normalized words,
The text sentiment analysis method to perform the second normalization processing of the first normalized word value.

Images