CN116150378A

CN116150378A - Text emotion extraction method and system based on pellet calculation and electronic equipment

Info

Publication number: CN116150378A
Application number: CN202310257830.1A
Authority: CN
Inventors: 陈子忠; 陈涛; 夏书银; 王国胤
Original assignee: Chongqing University of Post and Telecommunications
Current assignee: Chongqing University of Post and Telecommunications
Priority date: 2023-03-16
Filing date: 2023-03-16
Publication date: 2023-05-23

Abstract

The invention relates to the technical field of computers, and discloses a text emotion extraction method and system based on pellet calculation and electronic equipment, wherein the method comprises the following steps: s1, extracting advanced semantic representation information of an input text through a Bert model, carrying out cluster division on the input text according to the advanced semantic representation information of the input text in a clustering mode, dividing the input text into a plurality of pellets, and classifying the pellets so as to extract aspect words and viewpoint words of the input text; s2, according to the distance between the aspect words and the viewpoint words, embedding the word pairs formed by each aspect word and the viewpoint word into a BLSTM model, and connecting hidden states from the aspect words and the viewpoint words after encoding the BLSTM model, so as to perform emotion classification. The invention provides a pellet-based calculation method for extracting aspect words and viewpoint words, and solves the problem of word overlapping.

Description

Text emotion extraction method and system based on pellet calculation and electronic equipment

Technical Field

The invention relates to the technical field of computers, in particular to a text emotion extraction method and system based on pellet calculation and electronic equipment.

Background

In recent years, with the gradual maturation of natural language technology and the rapid development of artificial intelligence technology, network information of each social platform grows exponentially every day, and because data volume is huge and content is complex, screening of the information by manual mode is time-consuming and labor-consuming, so that the requirement of computer automation for network quality management and emotion demand analysis becomes important. Therefore, mining the emotion of the user in the mass data, analyzing the personal tendency of the user, and screening out the content with quality becomes an important subject for most scientific researchers to study.

At present, emotion analysis is an important research task in the field of natural language processing, and can help a platform or a merchant to know emotion requirements and attitudes of consumers or users, so that improvement and optimization are performed on products and after-sales services, the core competitiveness of the platform or the merchant is improved, and meanwhile, government departments can be helped to know mass requirements, so that public opinion guidance is mastered, and related policies are formulated. The traditional emotion analysis method is mainly a method based on rules or dictionary, the method cannot accurately analyze the semantics in sentences, the adaptability to new fields and new words is poor, the recognition rate is low, and through aspect emotion triple extraction, specific aspect words can be aimed at, so that viewpoint words corresponding to aspects and emotion polarities expressed by the viewpoint words can be extracted, and tasks can be defined as (aspects, viewpoints and emotions). In the existing emotion extraction methods, most research methods only can excavate shallow semantic information, classification effect is poor, and word extraction effect aiming at span level is poor; secondly, most sentences now contain a plurality of aspects or perspective words, the recognition performance of the data with the word overlapping is poor, and the overlapping elements cannot be effectively extracted in the current technical scheme.

Disclosure of Invention

The invention provides a text emotion extraction method and system based on pellet calculation and electronic equipment, so that the problem of word overlapping is solved.

The invention is realized by the following technical scheme:

a text emotion extraction method based on pellet calculation comprises the following steps:

s1, extracting advanced semantic representation information of an input text through a Bert model, carrying out cluster division on the input text according to the advanced semantic representation information of the input text in a clustering mode, dividing the input text into a plurality of pellets, and classifying the pellets so as to extract aspect words and viewpoint words of the input text;

s2, according to the distance between the aspect words and the viewpoint words, embedding the word pairs formed by each aspect word and the viewpoint word into a BLSTM model, and connecting hidden states from the aspect words and the viewpoint words after encoding the BLSTM model, so as to perform emotion classification.

As optimization, before extracting an input text through a Bert model, performing sequence marking on all words in the input text according to word labels, inputting the marked input text into an input layer of the Bert model to obtain word vectors, and inputting the obtained word vectors into the Bert model.

Preferably, each word label comprises one of an emotion label, an aspect label or a view label or a nonsensical label.

As optimization, clustering is specifically performed by using a K-means clustering method to perform clustering on the input text according to emotion labels of the input text to obtain a plurality of pellets, wherein each pellet is provided with a viewpoint label or an aspect label or a nonsensical label.

As optimization, comparing the purity of each pellet with a preset purity threshold value when dividing, stopping splitting the pellets if the purity of the pellets is not less than the purity threshold value, otherwise, continuing splitting until the purity of all pellets is not less than the purity threshold value.

As optimization, the specific process of classifying the pellets is as follows:

a1, for use by softmax function

Calculating aspect probability P marked as aspect tag in each pellet _i ^(ap) Then use +.>

To calculate the viewpoint probability P marked as the viewpoint label in each pellet _i ^(op) The aspect probability and the viewpoint probability are both called tag probability, and specifically are:

wherein W is _t ^(ap) And

for the weight and bias of aspect labels, W _t ^(op) 、/>

The weight and deviation of the viewpoint labels are respectively;

a2, judging the pellet as an aspect word or a viewpoint word according to the tag probability of the pellet.

As optimization, the specific steps of S2 are:

s2.1, screening out pellets belonging to aspect words and viewpoint words;

s2.2, combining the balls which are screened and belong to aspect words and viewpoint words to form aspect-viewpoint word pairs, setting text formats of positions in the input text according to the aspect-viewpoint words, and simultaneously, setting text formats which are different from the aspect words and the viewpoint words according to positions of non-viewpoint words and non-aspect words in the input text to convert the input text into specific text formats;

s2.3, inputting the input text in the specific text format and the word vector corresponding to the input text into the BLSTM model for prediction to obtain the emotion label of the aspect-viewpoint word pair, and then screening out the aspect-viewpoint-emotion triplet meeting the condition to finish emotion classification, and screening out the attribute-emotion pair meeting the condition to finish emotion classification.

As an optimization, the text formats of the aspect words and the viewpoint words are set to 1, and the text formats of the non-viewpoint words and the non-aspect words are set to 0.

The invention also discloses a text emotion extraction system based on pellet calculation, which comprises:

the extraction module is used for extracting high-level semantic representation information of an input text through a Bert model, carrying out cluster division on the input text according to the high-level semantic representation information of the input text in a clustering mode, dividing the input text into a plurality of pellets, and classifying the pellets so as to extract aspect words and viewpoint words of the input text;

and the classification module is used for embedding the word pairs formed by each aspect word and each viewpoint word into a BLSTM model according to the distance between the aspect word and the viewpoint word, and connecting the hidden states from the aspect word and the viewpoint word after the encoding of the BLSTM model is completed, so as to perform emotion classification.

The invention also discloses an electronic device, which comprises at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a text emotion extraction method based on a pellet calculation as described above.

Compared with the prior art, the invention has the following advantages and beneficial effects:

the invention provides a pellet-based calculation method for extracting aspect words and viewpoint words, so that the problem of word overlapping is solved, meanwhile, classification performance and accuracy can be improved through pellet calculation, and finally, emotion classification accuracy is improved.

Drawings

In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the drawings that are needed in the examples will be briefly described below, it being understood that the following drawings only illustrate some examples of the present invention and therefore should not be considered as limiting the scope, and that other related drawings may be obtained from these drawings without inventive effort for a person skilled in the art. In the drawings:

FIG. 1 is a diagram of a text emotion extraction system based on pellet computation according to the present invention;

fig. 2 is an exemplary diagram of extracting aspect words and perspective words by pellet calculation.

Detailed Description

For the purpose of making apparent the objects, technical solutions and advantages of the present invention, the present invention will be further described in detail with reference to the following examples and the accompanying drawings, wherein the exemplary embodiments of the present invention and the descriptions thereof are for illustrating the present invention only and are not to be construed as limiting the present invention.

Example 1

in this embodiment, before an input text is extracted through a Bert model, all words in the input text are sequentially labeled according to word labels, then the labeled input text is input into an input layer to obtain word vectors, and the obtained word vectors are input into the Bert model.

Specifically, step one: data of the input text is first preprocessed.

All data of the training set of the input text are subjected to sequence marking, and the main types of the data are four: emotion tags, aspect words, opinion words, and nonsensical words.

The emotion labels mainly comprise { NEG, POS, NEU, NOT } (positive, negative, neutral and none), then the aspect word attribute is marked as aspect, the aspect word attribute is marked as opiion, and the nonsensical word is marked as N.

For example, the input text is "my mobile phone screen is large and good", the useless word is marked by n, the beginning of the aspect word is marked by B-aspect, the interior of the aspect word is marked by I-aspect, the beginning of the viewpoint word is marked by B-aspect, the interior of the viewpoint word is marked by I-aspect, then the sentence is marked as:

n,n,B-aspect,I-aspect,I-aspect,I-aspect B-opinion,I-opinion,n,n,B-opinion,I-opinion。

step two: word vector generation (word unbedding)

And carrying out word embedding operation on the input text. In the input layer, all words are mapped to a high-dimensional vector space, which is a pre-trained 300-dimensional word vector, through which all words are contextualized. All words herein refer to all words that would make up a chinese sentence, and each word if english text. The mapping is that in the prior art, the pre-trained word vector information can be obtained through Glove, which is not described herein.

After the word vector of the input text is obtained, performing a step three:

step three: extracting high-level semantic representation of text through Bert model

The word vector obtained by the input layer is input into a Bert model, and high-level semantics of the text can be extracted through the Bert model.

When the high-level semantic representation information (semantic vector of the whole sentence, including all the words in the sentence) of the input text is extracted, the following steps are performed.

Step four: generating the advanced semantic representation information into pellets, namely dividing the input text into a plurality of pellets according to the advanced semantic representation information of the input text in a clustering mode.

The step mainly comprises the step of generating a plurality of pellets by using the high-level semantic representation information obtained in the step three. The pellet mainly divides semantic information with the same word label (the word label only comprises three groups of labels of view, aspect and nonsensical) by a K-means clustering method, and after a plurality of clusters are divided, each cluster has a cluster center and a radius. Here, the pellet radius, expressed in terms of the maximum euclidean distance from all points in the pellet to the center of the pellet, is defined as follows:

(V _i for the semantic vector of each word, N is the number of samples in the pellet

r ₁ ＝max(dis(x _i ,c))

r ₁ Maximum radius C-sphere center

dis(x _i C) -semantic vector x in sentence _i Euclidean distance to center of sphere c

In the generation process of the pellets, the whole sentence is one pellet, each pellet is split through a random initialization center, and the splitting is stopped until the purity threshold T of the pellet is greater than 0.9. Here, the purity threshold T is used as a stop condition for pellets, and is used as a criterion for determining whether pellets are produced or not. By calculating whether the purity of each pellet is less than the purity threshold T, if so, the partitioning is continued, and if so, the pellet is retained. And carrying out cluster division on the input text according to the emotion labels of the input text by a K-means clustering method to obtain a plurality of pellets, wherein each pellet comprises at least one word pair consisting of emotion-aspect-views, and the emotion corresponding to the aspect-view word pair in the same pellet is consistent.

Step five: classifying pellets with a softmax function

The specific process is as follows:

a1, for use by softmax function

To calculate the viewpoint probability P marked as the viewpoint label in each pellet _i ^(op) The method specifically comprises the following steps:

wherein W is _t ^(ap) And

for the weight and bias of aspect labels, W _t ^(op) 、/>

The weight and deviation of the viewpoint labels are respectively; the weights and deviations are coefficients of the softmax classification function, which are randomly initialized, but are optimized and adjusted according to the learning of each round of data, which is the prior art and will not be described here.

A2, judging the pellet as an aspect word or a viewpoint word according to the probability of the pellet. For example, if aspect probability P _i ^(ap) Greater than the opinion probability P _i ^(op) The pellet is defined as an aspect probability.

For this step

To calculate the probability P of marking as aspect in each pellet _i ^(ap) Then use +.>

To calculate the probability P of marking each pellet as a point of view _i ^(op) Wherein W is _t ^(op) ,W _t ^(ap) And->

Is a learnable weight and bias.

Specifically, the method comprises the following steps: emotion classification of ideas and aspect words

S2.1, screening out pellets belonging to aspect words and viewpoint words; as shown in FIG. 2, the selected pellets have 3 viewpoint words and 2 aspect words.

S2.2, combining the balls which are screened and belong to the aspect words and the viewpoint words to form aspect-viewpoint word pairs, setting text formats according to the positions of the aspect words and the viewpoint words in the input text, and simultaneously, setting text formats which are different from the aspect words and the viewpoint words according to the positions of non-viewpoint words and non-aspect words in the input text to convert the input text into specific text formats; as shown in FIG. 2, the non-opinion and aspect words herein are "My, but also,". "these 3".

S2.3, inputting the input text in the specific text format and the word vector corresponding to the input text into the BLSTM model for prediction to obtain the emotion label of the aspect-viewpoint word pair, and then screening out the aspect-viewpoint-emotion triples meeting the conditions, thereby completing emotion classification.

These terms and terms are found in the sentence and set to 1, and the non-terms are set to 0, resulting in some 0,1,0,1,0,0 such sequence, which is then input into the Bilstm model along with its term vector.

In this step, the N viewpoint words and the M aspect words obtained in the fifth step are arbitrarily combined, where n×m combination methods are obtained, and then these multiple methods are screened. After the BLSTM model is encoded, hidden states from the aspect words and the perspective words are connected (here, the BLSTM model learns itself, learns semantic information between word pairs, and then splices together, and the splicing here refers to dimension splicing), so that emotion classification is performed. In this stage, it is mainly from these combinations (triples of emotion-aspect-view) that the eligible attribute-emotion pairs are screened, e.g. the BLSTM model prediction results are retained (aspect-view-positive) and discarded if they are (aspect-view-none). When emotion predictions are positive, negative or neutral, the emotion pairs are kept against the true tag pairs, and a loss function is calculated, thereby optimizing the BLSTM model.

After the BLSTM model calculates the loss function of the predicted value and the real value, the loss function is optimized by deriving by using a gradient descent method, so that the parameters of the BLSTM model are reversely updated.

Because the semantic information is clustered by the pellets, gradient interruption can be caused in the back propagation process. For example, the size before the pellet clustering is 300×32, and becomes 128×32 after the clustering, and only 128 gradients are transmitted back to the pellet layer, so that gradient propagation is interrupted. The part of the pellet is calculated and described as a pellet layer in the neural network, so that the gradient at the center of the pellet is copied n times to replace the gradient of other samples in the pellet in the process of counter propagation, and n is specifically the number of words aggregated in the pellet, so that the normal propagation of the gradient is ensured.

For example, in the back propagation, if 10 words in a sentence become 4 pellets after clustering, the dimension will change for the back propagation, so the invention returns n copies of vectors in the centers of 4 words in one pellet center, so 4 becomes 10 again, and the gradient can normally propagate calculation.

With the word vector and the sequence of 0,1,0,1 representing the aspect and perspective words, they are entered into the BLSTM model together. (it will be appreciated that 0,1,0,1 is equivalent to a weight representation, 0 is meaningless, the BLSTM model will not take into account, and 1 is meaningful), and then the BLSTM model will predict these aspect-perspective pairs. The four classes of positive, negative, neutral and nonsensical emotion labels of these aspect-viewpoint pairs are predicted, then the emotion labels of the predicted result are nonsensical and will be discarded, while the other emotion labels will be paired with the aspect and viewpoint words of the first stage to form a triplet (aspect-viewpoint-positive), which is also the final objective of the prediction of the present invention.

When the sequence is marked, correct triples are marked, then the triples are compared with triples predicted by the BLSTM model, and finally the BLSTM model is optimized.

Example 2

Example 3

The invention provides a pellet-based calculation method for extracting aspect words and viewpoint words, which is characterized in that a pellet-based calculation method is used for extracting span-level words, words belonging to an aspect tag are clustered together, for example, a 'mobile phone screen', four words are clustered together to form a pellet, words with span level can be extracted through the method, then aspect tags and viewpoint tags are combined into NxM types, so that the extraction of overlapping words can be solved, for example, the 'mobile phone screen' is large and very good, the 'mobile phone screen' is paired with the 'large' and the 'very good', and the types in the NxM types comprise the two cases, thereby solving the problem of word overlapping.

Because the dimension of sentences in classification is reduced, if one sentence is formed by 10 words, each word is predicted, but vectors which can form semantic information are clustered together by pellet calculation to finally form 4 balls, and then prediction classification is performed on the 4 balls, so that classification performance and accuracy can be improved by pellet calculation, and finally the accuracy of emotion classification is improved.

The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.

Claims

1. The text emotion extraction method based on the pellet calculation is characterized by comprising the following steps of:

2. The text emotion extraction method based on pellet computation according to claim 1, wherein before an input text is extracted through a Bert model, all words in the input text are sequentially labeled according to word labels, then the labeled input text is input into an input layer of the Bert model to obtain word vectors, and the obtained word vectors are input into the Bert model.

3. The method of claim 2, wherein each word label comprises one of an emotion label, an aspect label, or a point of view label, or a nonsensical label.

4. The text emotion extraction method based on pellet calculation of claim 3, wherein the clustering is specifically that a plurality of pellets are obtained by clustering the input text according to word labels of the input text through a K-means clustering method, and each pellet has a viewpoint label or an aspect label or a nonsensical label.

5. The text emotion extraction method based on pellet calculation of claim 4, wherein the purity of each pellet is compared with a preset purity threshold value when dividing, if the purity of the pellet is not less than the purity threshold value, the pellet stops splitting, otherwise, splitting is continued until the purity of all pellets is not less than the purity threshold value.

6. The text emotion extraction method based on pellet calculation of claim 1, wherein the specific process of classifying a plurality of pellets is as follows:

a1, for use by softmax function

Calculating aspect probability P marked as aspect tag in each pellet _i ^(ap) Then using

wherein W is _t ^(ap) And

for the weight and bias of aspect labels, W _t ^(op) 、/>

The weight and deviation of the viewpoint labels are respectively;

7. The text emotion extraction method based on pellet calculation of claim 1, wherein the specific steps of S2 are as follows:

s2.1, screening out pellets belonging to aspect words and viewpoint words;

8. The text emotion extraction method based on pellet calculation of claim 7, wherein the text formats of the aspect words and the viewpoint words are set to 1, and the text formats of the non-viewpoint words and the non-aspect words are set to 0.

9. A text emotion extraction system based on pellet computation, comprising:

10. An electronic device comprising at least one processor, and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of text emotion extraction based on a pellet calculation as claimed in any of claims 1 to 8.