CN104965822A - Emotion analysis method for Chinese texts based on computer information processing technology - Google Patents

Abstract

The invention discloses an emotion analysis method for Chinese texts based on computer information processing technology. Comments on Chinese products are subjected to word segmentation. By utilizing a bag-of-words model, vector representations of product comments are generated. The vector of every comment is inputted to a visible unit of a limited Boltzmann machine (RBM) in deep learning. Sentimental characteristics of Chinese texts are extracted by the RBM and the extracted emotional characteristics are inputted to a SVM for text emotion classification. The emotion analysis method for the Chinese texts based on computer information processing technology is capable of improving relevance of emotional semantics of characteristics while the SVM is capable of improving accuracy of emotion classification of comments on Chinese products.

Description

A kind of Chinese text sentiment analysis method based on Computerized Information Processing Tech

Technical field

The present invention relates to natural language processing technique, particularly a kind of Chinese text sentiment analysis method based on Computerized Information Processing Tech.

Background technology

Along with the develop rapidly of internet and ecommerce, people more and more like in shopping online and deliver oneself on the net to the suggestion of certain product and view.Sentiment analysis is the popular research field of of artificial intelligence aspect, and its target makes computing machine extract the emotion tendency of author to evaluation object from the subjective texts that user produces, i.e. forward or negative sense.Forward refers to user's liking this product, and they hold the attitude praised and affirm to this product; Negative sense refers to that user holds negative attitude to this product because a certain product exists certain defect or discontented place.

At present, the main method for cyber comment sentiment analysis has in the method based on emotion knowledge and the method based on machine learning.

Method based on emotion knowledge mainly relies on some existing sentiment dictionary and linguistries, classifies to the Sentiment orientation of text.As: people such as Polanyi are absorbed in, several sentiment dictionary and emotion rule are built to polarity word and relevant negator lexical word; The people such as Velikovich illustrate and how learn a large sentiment dictionary with a seed dictionary and generation figure framework and then carry out sentiment analysis etc. to text.The people such as Lee differentiate the emotion tendency of text by the number comparing forward vocabulary and negative sense vocabulary, and namely more which side of number is as the Sentiment orientation of whole text.Sentiment dictionary depends on the formulation of expert, can not reflect the comprehensive of its.

Method based on machine learning includes supervision and unsupervised learning.The people such as Turney propose a kind of unsupervised learning that uses and carry out the method for sentiment analysis to documentation level text; The people such as Tang Huifeng use n-gram model, extract the feature as text such as noun, verb, adverbial word, adjective, KNN, Bayes and SVM classifier is used again to classify to text respectively, train in different scales and different feature quantity simultaneously, thus identify the emotional orientation analysis of text; The people such as Sun Yan propose a kind of unsupervised theme emotion mixture model, this model is on the basis do not marked text, gather affective tag and the theme label of each sentence, thus obtain the emotion word of this text, and then realize classifying to the emotion of documentation level.But the product review of Chinese text, towards the Chinese of multi-semantic meaning, more rules, because the feature selected or extract is incorrect when carrying out emotion automatic analysis with machine learning to them, often causes accuracy not high.

Based in the sentiment analysis of machine learning, feature extraction is the key factor determining analytical effect.Degree of depth study forms more abstract high level by combination low-level feature and represents attribute classification or feature, to find that the distributed nature of data represents.Degree of depth study is the neural network set up, simulation human brain carries out analytic learning, and the mechanism that it imitates human brain carrys out decryption, such as image, sound and text.A kind of algorithm that RBM learns as the degree of depth, has degree of depth study Level by level learning, obtains the characteristic of data structure information, therefore may be used for extracting the feature representing feeling polarities.

Summary of the invention

Technical matters to be solved by this invention is, not enough for prior art, provides a kind of Chinese text sentiment analysis method based on Computerized Information Processing Tech.

For solving the problems of the technologies described above, the technical solution adopted in the present invention is: a kind of Chinese text sentiment analysis method based on Computerized Information Processing Tech, comprises the following steps:

1) Chinese product review text set is inputted, Chinese word segmentation is carried out with the accurate model in stammerer participle, and determine part of speech, filter according to part of speech, reservation part of speech is the vocabulary of noun, adjective, adverbial word and conjunction, obtains the corpus after participle, calculate the text being cut into word by Chinese word segmentation by Chinese word segmentation, obtain the weights that each word is commented at every bar, produce comment vector;

2) limited Boltzmann machine is adopted to carry out feature selecting to above-mentioned comment vector; Described limited Boltzmann machine is made up of visible layer V and hidden layer H, and layer interior joint neuron is binary random number, and the node of visible layer is separate, separate between hidden layer node; Connection weights between visible layer node and hidden layer node are W; Above-mentioned steps 1) the comment vector input visible element layer that produces;

3) limited Boltzmann machine is by training sdpecific dispersion method, the correlation parameter of the limited Boltzmann machine of initialization, after iteration 100 times, obtains hidden layer node h _i=1 conditional probability p (h under visual layers v _i=1/v) and visual layers node v _i=1 conditional probability p (v under hidden layer h _i=1/h); Wherein, $p(h_j=1|v)=\mathrm{\σ}(b_j+\underset{i}{\Σ}{v}_i{w}_{ij});p(v_i=1|h)=\mathrm{\σ}(a_i+\underset{i}{\Σ}{h}_j{w}_{ij});$ Function amount of bias a _iand b _jinitial value is 0; w _ijfor the connection weight weight values between the node i of visible layer and the node j of hidden layer, w _ijobeying average is 0, and standard deviation is the normal distribution of 1;

4) according to hidden layer to visual layers node v _ibe reconstructed; If v _ireconstruction value be less than or equal to 0.01, then think visible node v _ican not high-level structure be represented, give up v _icorresponding word; If v _ireconstruction value be greater than 0.01, then by v _ireconstruction value with and input value subtract each other and obtain difference value; The maximum affective characteristics number that every section is commented on is set to 30, difference value is arranged from small to large, front 30 word v _idifference value less than word below, v is described _ido not change after reconstruct, then find out v _icorresponding Chinese word, and using this word as Feature Words; The difference that relatively above 30 words are corresponding of word is below large, and v is described _iafter reconstruct, result changes a lot, then by v _icorresponding to non-Feature Words; If in certain comment, the quantity that reconstruction value is greater than the visible elements of 0.01 is less than 30, then the word that these visible nodes are corresponding all retains as Feature Words; The Feature Words retained forms the vector space of each comment jointly;

5) by through step 4) the new comment vector produced is input in support vector machine and classifies after process, obtains the emotion classification that each comment is vectorial; Described emotion classification is forward or negative sense.

Described step 1) in, the production process of comment vector is: adopt the word bag model in natural language processing, if certain word occurs in certain comment, this word respective weights value in comment vector is just set to 1, otherwise value is set to 0, the length of comment vector is the size of the dictionary extracting word.

The nodes of visible layer is 476, and the nodes of hidden layer is 50.

Compared with prior art, the beneficial effect that the present invention has is: the present invention adopts the primitive character information of degree of depth learning algorithm RBM to Chinese product review to carry out higher level of abstraction, according to high layer information reconstruct primitive character information, and carry out feature selecting according to reconstruction value, obtain the Feature Words more expressing high-level structure information.This method can extract the feature that more can show product review emotion essence; RBM carries out the combination that new feature that feature extraction can obtain is former feature, and adopts RBM to carry out feature selecting to be compare according to the comment of reconstruct and former comment, to select the special testimony of this comment.Compared with feature extraction, the feature chosen has more intuitive; Carry out feature selecting at Chinese product review primitive character by RBM, then the comment vector produced by Feature Words is input in SVM, carries out emotional semantic classification.Experimental result shows, the method has very high precision.

Accompanying drawing explanation

Fig. 1 is RBM-SVM algorithm flow chart;

Fig. 2 is bag-of-words text vector expression figure;

Fig. 3 is the word space after RBM reconstruct;

Fig. 4 contrasts for before and after certain review information feature selecting dimensionality reduction.

Embodiment

Specific implementation process of the present invention is as follows:

Step 1: extract 1706 sections of comment texts in the Chinese comment corpus that electronic product is correlated with from data hall, wherein have 853 forwards comments, 853 negative senses comments.

Step 2: Chinese word segmentation is carried out to the accurate model in these 1706 sections of comment text stammerer participles, judges part of speech.Remove stop words, define part of speech filtering rule simultaneously.Definition part of speech filtering rule is exactly:

A. the tendentious part of speech set of performance emotion is set up: { noun, adjective, adverbial word and conjunction },

B. judge the part of speech of each word in comment, this part of speech of certain word not in part of speech set, then deletes this word, otherwise retains this word.

Step 3: adopt the word bag model in natural language processing, if this word occurs in this comment, be just set to 1, otherwise be set to 0, the term vector in this, as text represents.The length of vector is the size of the dictionary of the word extracted.As: there are two to comment on d1, d2, through processing these two comments, extract the noun in them, adjective, adverbial word, conjunction etc., being labeled as w1 respectively, w2, w3, w4, w5.Then the term vector of word bag method model representation is as follows:

Step 4: set up two layers of limited Boltzmann machine (RBM), the nodes of visible layer is 476, and the nodes of hidden layer is 50.Each visible element in visible layer is separate.There is connection between the node i of visible layer with the node j of hidden layer, connection weight weight values is W _ji.Limited Boltzmann machine has an energy of combining configuration between visual layers with hidden layer, can be expressed as:

$E(v,h;\mathrm{\θ})=-\underset{ij}{\Σ}{w}_{ji}{v}_i{h}_j-\underset{i}{\Σ}{b}_i{v}_i-\underset{j}{\Σ}{a}_j{h}_j$

Wherein i is the number of the node of visual layers unit, and j is the number of hidden layer cell node, θ={ w _ij, a _i, b _jbe model parameter, w _ijfor the connection weight weight values between the node i of visible layer and the node j of hidden layer, a _iand b _ibe respectively the amount of bias of visible layer and hidden layer.Their joint probability distribution is:

$\begin{array}{c}{p}_{\mathrm{\θ}}(v,h)=\frac{1}{z\left(\mathrm{\θ}\right)}\exp (-E(v,h;\mathrm{\θ}\left)\right)\\ =\frac{1}{z\left(\mathrm{\θ}\right)}\underset{ij}{\mathrm{\Π}}{e}^{w_{ij}{v}_i{h}_j}\underset{i}{\mathrm{\Π}}{e}^{b_i{v}_i}\underset{j}{\mathrm{\Π}}{e}^{a_j{h}_j}\end{array}$

Wherein joint probability is the sample probability in order to try to achieve hidden layer and visual layers.

Due in RBM, directly not connecting between hidden layer unit, is namely conditional sampling between hidden layer node, namely the data being then easy to the unbiased sample obtaining a Data distribution8 are expected.Given a visual layers v selected at random, hidden layer unit h _jbinary condition be 1 probability be:

$p(h_j=1|v)=\mathrm{\σ}(b_j+\underset{i}{\Σ}{v}_i{w}_{ij})---(3-3)$

Wherein function

$\mathrm{\σ}\left(x\right)=\frac{1}{1+\exp (-x)}---(3-4)$

Now v _i, h _jbe a unbiased sample, this function is the same with the function in neural network, is a condition Bei Nuli distribution.Owing to also directly not connecting between visible layer unit, be then easy to the sample obtaining visible element, the probability being 1 in the binary condition of given visual layers unit is:

$p(v_i=1|h)=\mathrm{\σ}(a_i+\underset{i}{\Σ}{h}_j{w}_{ij})$

Step 5: the comment of the Chinese of cutting is input in RBM, a node of the corresponding visible layer of each word in dictionary.RBM is adopted to carry out unsupervised learning.

Step 6:RBM is called sdpecific dispersion (Contrastive Divergence by one, CD) method rapidly unsupervised training .RBM in sdpecific dispersion (Contrastive Divergence, CD) carry out k and walked Gibbs sampling, namely adopt k-CDgibbs to learn.All a random initial state with a training sample or visible elements in k-CDgibbs algorithm, the state v of initialization visible elements ⁰, the sampling hocketed below, wherein P (h|v ₀) be h ₀sampled probability, wherein P (v|h ₀) be v ₁sampled probability, all the other are similar.

h ₀～p(h|v ₀),v ₁～p(v|h ₀) (3-13)

h ₁～p(h|v ₁),v ₂～p(v|h ₁) (3-14)

......,v _k+1～p(v|h _k) (3-15)

Suppose that S is training set, (v ₁..., v _m, h ₁... h _n) be respectively visible elements and the concealed nodes of RBM.K-CD Gibbs specific implementation flow process is as follows, and the parameter declaration in flow process is as follows: Δ w _ijfor W _ijthe increment of weight, Δ a _j=be amount of bias a _jincrement, Δ b _ifor amount of bias b _iincrement

Step 7: the parameter of the limited Boltzmann machine of initialization (RBM): for step 4, the weights W in RBM is obedience average is 0, and standard deviation is the normal distribution of 1; Amount of bias a _jand b _jinitial value is all 0 respectively.Learning rate is 0.01

Step 8: iterations is set to the weights W that 100, CD-k gibbs algorithm finally obtains between visible node i and concealed nodes j _ij, amount of bias a _jand b _iend value.And with reference to the formulae discovery p (v listed in step 4 _i=1|h)

Step 9: to the word space of each comment according to p (v _i=1|h) be reconstructed.If by v _ireconstruction value be less than or equal to 0.01, then think visible node v _ican not high-level structure be represented, give up v _icorresponding word.If v _ireconstruction value be greater than 0.01, then by v _ireconstruction value with and input value subtract each other and obtain difference value.If the difference of correspondence is less, illustrate that this word does not change after reconstruct, what the Chinese word then found out corresponding to it is, then just can using this word as character representation; On the contrary, if the difference of correspondence is comparatively large, illustrate that this word result after reconstruct changes a lot, this word just can not as the character representation of this comment like this.Therefore, difference value around this principle arranges by from small to large, select front 30 as Feature Words.If in certain comment, the quantity that reconstruction value is greater than the visible elements of 0.01 is less than 30, then the word that these visible nodes are corresponding all retains as Feature Words.Dimensionality reduction has been carried out by the term vector space of feature selecting to comment.

Step 10: we select by experiment to Gibbs frequency in sampling K.In experiment, 1706 sections of comment texts are pressed training set and test set by 3:1 random choose.The node of the hidden layer of RBM is set to 2 by us, with 0,1 represents, wherein 0 represents that the product review emotion now judged is that namely negative sense represents that user holds negative attitude to this product, and 1 represents that the product review emotion now judged is that namely forward represents that user holds positive attitude to this product, by RBM as cluster, export the emotion classification of comment. K=5 is set respectively, 10,15,20,25.Along with the change of K, RBM, to the different recognition results of product review, weighs by accuracy rate, recall rate and F average 3 indexs.

Table 1 RBM is to the different recognition results of product review

When experimental result shows K=10, F average is the highest, and show that combination property is best, therefore K is set to 10.

Step 11: the comment through dimensionality reduction is input in support vector machine (SVM) by we, carries out emotional semantic classification.The linear classifier that interval on SVM feature space is maximum, its learning strategy makes margin maximization, is finally converted into solving of a convex quadratic programming problem.Here is the objective function of SVM:

$\max W\left(\mathrm{\α}\right)={\mathrm{\Σ}}_{i=1}^N{\mathrm{\α}}_i-\frac{1}{2}{\mathrm{\Σ}}_{i,j=1}^N{\mathrm{\α}}_i{\mathrm{\α}}_j{y}_i{y}_{j}K(x_i,x_j)$

$s.t.{\mathrm{\Σ}}_{i=1}^N{\mathrm{\α}}_i{y}_i=0;C\≥\mathrm{\α}>>0,i=1,....$

Ask this objective function, obtain:

According to ${\mathrm{\&alpha;}}^{*}=({\mathrm{\&alpha;}}_1^{*},......{\mathrm{\&alpha;}}_1^{*}),$ Calculate $w^{*}={\mathrm{\&Sigma;}}_{i=1}^l{y}_i{\mathrm{\&alpha;}}_i^{*}{x}_i,$ Select α ^*one-component meet $0<{\mathrm{\&alpha;}}_j^{*}<C,$

And calculate accordingly $b^{*}=y_j-{\mathrm{\&Sigma;}}_{i=1}^l{y}_i{\mathrm{\&alpha;}}_i^{*}K(x_i,x_j)$

Wherein, α={ α ₁α _n; K (x _i, x _j) be Chinese comment x _iand x _jkernel function, y _ifor Chinese comment x _iemotion classification, C is penalty factor, and we select C value to be 0.0001 in an experiment.

In SVM, Chinese comment x ₁and x ₂kernel function we adopt be radial basis function RBF, see following formula.By kernel function, feature space is mapped to higher dimensional space by low-dimensional.

$K(x_1,x_2)=\exp (-\frac{\left|\right|x_1-x_2|{|}^2}{2{\mathrm{\&sigma;}}^2})$

Step 12: the sentiment analysis method that RBM and SVM is combined, and be used alone the Measures compare of SVM or RBM, calculate accuracy rate, recall rate and F average 3 indexs according to by the result exported and the result manually marked.Experimental result sees the following form:

The recognition result of table 2 distinct methods

As seen from Table 2, RBM and SVM combines and greatly improves sentiment analysis accuracy.Be used alone compared with SVM, RBM and SVM combines not only accuracy rate and improves, and recall rate also improves.Be used alone compared with RBM, RBM and SVM improves greatly in conjunction with accuracy rate, although recall rate reduces a some point, it is much higher that F average ratio is used alone RBM, shows that combination property should the sentiment analysis method of RBM and SVM combination.

Claims (3)

1., based on a Chinese text sentiment analysis method for Computerized Information Processing Tech, it is characterized in that, comprise the following steps:

1) Chinese product review text set is inputted, Chinese word segmentation is carried out with the accurate model in stammerer participle, and determine part of speech, filter according to part of speech, reservation part of speech is the vocabulary of noun, adjective, adverbial word and conjunction, obtains the corpus after participle, calculate the text being cut into word by Chinese word segmentation by Chinese word segmentation, obtain the weights that each word is commented at every bar, produce comment vector;

2) limited Boltzmann machine is adopted to carry out feature selecting to above-mentioned comment vector; Described limited Boltzmann machine is made up of visible layer V and hidden layer H, and layer interior joint neuron is binary random number, and the node of visible layer is separate, separate between hidden layer node; Connection weights between visible layer node and hidden layer node are W; Above-mentioned steps 1) the comment vector input visible element layer that produces;

3) limited Boltzmann machine is by training sdpecific dispersion method, the correlation parameter of the limited Boltzmann machine of initialization, after iteration 100 times, obtains hidden layer node h _i=1 conditional probability p (h under visual layers v _j=1/v) and visual layers node v _i=1 conditional probability p (v under hidden layer h _i=1/h); Wherein, $p(h_j=1|v)=\mathrm{\&sigma;}(b_j+\underset{i}{\&Sigma;}{v}_r{w}_{ij});$ $p(v_i=1|h)=\mathrm{\&sigma;}(a_i+\underset{i}{\&Sigma;}{h}_j{w}_{ij});$ Function amount of bias a _iand b _jinitial value is 0; w _ijfor the connection weight weight values between the node i of visible layer and the node j of hidden layer, w _ijobeying average is 0, and standard deviation is the normal distribution of 1;

4) according to visual layers node v _i=1 conditional probability p (v under hidden layer h _i=1/h) to visual layers node v _ibe reconstructed; If v _ireconstruction value be less than or equal to 0.01, then think visible node v _ican not high-level structure be represented, give up v _icorresponding word; If v _ireconstruction value be greater than 0.01, then by v _ireconstruction value with and input value subtract each other and obtain difference value; The maximum affective characteristics number that every section is commented on is set to 30, difference value is arranged from small to large, front 30 word v _idifference value less than word below, v is described _ido not change after reconstruct, then find out v _icorresponding Chinese word, and using this word as Feature Words; The difference that relatively above 30 words are corresponding of word is below large, and v is described _iafter reconstruct, result changes a lot, then by v _icorresponding to non-Feature Words; If in certain comment, the quantity that reconstruction value is greater than the visible elements of 0.01 is less than 30, then the word that these visible nodes are corresponding all retains as Feature Words; The Feature Words retained forms the vector space of each comment jointly;

5) by through step 4) the new comment vector produced is input in support vector machine and classifies after process, obtains the emotion classification that each comment is vectorial; Described emotion classification is forward or negative sense.

2. the Chinese text sentiment analysis method based on Computerized Information Processing Tech according to claim 1, it is characterized in that, described step 1) in, the production process of comment vector is: adopt the word bag model in natural language processing, if certain word occurs in certain comment, this word respective weights value in comment vector is just set to 1, otherwise value is set to 0, and the length of comment vector is the size of the dictionary extracting word.

3. the Chinese text sentiment analysis method based on Computerized Information Processing Tech according to claim 2, is characterized in that, the nodes of visible layer is 476, and the nodes of hidden layer is 50.