CN106599933A - Text emotion classification method based on the joint deep learning model - Google Patents

Abstract

The invention provides a text emotion classification method based on the joint deep learning model which relates to the text emotion classification method. The method is designed with the object of solving the problems with the dimension disaster and sparse data incurred from the existing support vector machine and other shallow layer classification methods. The method comprises: 1) processing each word in the text data; using the word2vec tool to train each processed word in the text data so as to obtain a word vector dictionary; 2) obtaining the matrix M of each sentence; training the matrix M by the LSTM layer and converting it into vector with fixed dimensions; improving the input layer; generating d-dimensional h word vectors with context semantic relations; 3) using a CNN as a trainable characteristic detector to extract characteristics from the d-dimensional h word vectors with context semantic relations; and 4) connecting the extracted characteristics in order; outputting to obtain the probability of each classification wherein the classification with the maximal probability value is the predicated classification. The invention is applied to the natural language processing field.

Description

A kind of text sentiment classification method based on combined deep learning model

Technical field

The present invention relates to be based on the text sentiment classification method of combined deep learning model.

Background technology

Text emotion classify, as the term suggests, be judge one evaluation text representation be commendation or derogatory sense emotion, Its Chinese version can be documentation level, Sentence-level or key element level.Emotional semantic classification both can be regarded as a classification problem, it is also possible to Regard a regression problem as.If regression problem, as long as doing after the emotion fraction for calculating text once arrive emotion class Other mapping.When a classification problem is considered, the technology that there are a large amount of text classifications is available, but with it is general Text classification is compared, and the difficult problem that emotional semantic classification has its exclusive needs specially treated, is available for the characteristics of also have its exclusive utilizing.Emotion Analysis text can be documentation level, Sentence-level or key element level, and the present invention is analyzed based on Sentence-level.Sentiment analysis are certainly After 2002 are proposed by Bo Pang, significantly paid close attention to, particularly obtained on the sentiment analysis of product review etc. Obtained and developed on a large scale very much.The method of early stage sentiment analysis research is mainly based upon the method and the method based on statistical learning of rule. Most sentiment analysis research is all based on statistical learning method, the main feature by learning objective sample, according to spy The judgement of classification is made in the distribution levied to text.Great majority are all to return to carry out full supervised learning, root using SVM and logistic It is different according to training data.Using to feature be also not quite similar.In addition to unitary word feature, also used based on dictionary information The series of features such as word class, sentiment dictionary, interjection, punctuation mark, user profile, but effect is unsatisfactory.

In recent years, people carry out sentiment analysis using the method for deep learning one after another.There is scholar to propose using recurrence certainly Coding method is predicted solving the problems, such as complex sentiment analysis in combination with the parsing tree of Stanford Parser The structure of sentence and judge the Sentiment orientation of sentence.A kind of improvement RNN models are proposed in academia in 2012：Matrix- Vextor RNN models, the model increased the vocabulary that a matrix comes record modification and center word combination compared with RAE models Show, this enables MV-RNN models to learn the implication of operator in some logical propositions and natural language.But the model parameter Quantity is very big, and this means that and is difficult to use in industrial quarters, because amount of calculation is difficult to meet.In order to improve the model, 2013 Year when again it is proposed that RNTN models, the model introduces tensor and carries out coordinate transform, formulates dependency rule by its middle part Component is divided to make linear transformation so that the shortcoming that the number of parameters of previous model village presence is excessive is addressed.Also scholar adopts The emotion tendency of sentence is judged with the LSTM of different length time step, so that important feature can be traveled to as soon as possible On decision-point.

The content of the invention

The invention aims to solve the shallow-layer sorting technique such as existing SVM, dimension disaster and Sparse can be brought The problems such as, and propose a kind of text sentiment classification method based on combined deep learning model.

A kind of text sentiment classification method based on combined deep learning model is comprised the following steps：

Step one：Each word in text data is processed, using word2vec instruments to the textual data after process Each word is trained according in, obtains term vector dictionary, and each of which word all correspond to a term vector；It is described Word2vec instruments are Google's term vector instrument；

Step 2：It is originally inputted as word sequence (w₁,….,w_s-1,w_s,…,w_max), obtain the matrix M of each sentence, M= (x₁,…,x_s-1,x_s,…,x_max)∈R^d×max；Matrix M is trained and is converted into the vectorial right of fixed dimension by LSTM layers (embedding) input layer is improved, and generates h term vector with context semantic relation of d dimensions；It is wherein described w₁,….,w_s-1,w_s,…,w_maxWord label corresponding to each word in each sentence, x₁,…,x_s-1,x_s,…,x_maxBe with w₁,….,w_s-1,w_s,…,w_maxOne-to-one term vector, LSTM is shot and long term memory models；

Step 3：The d generated in step 2 is tieed up the h term vector with context semantic relation as the input of CNN, Tieed up in the h term vector with context semantic relation from d as a trainable property detector using CNN and extract spy Levy, the CNN is convolutional neural networks；

Step 4：The feature extracted in step 3 is sequentially connected, as softmax's after openness activation primitive Input, output obtains the probability of each classification, all categories probability and for 1, and the maximum classification of probable value is predicted Classification, result of this classification as combined deep learning model.

Beneficial effects of the present invention are：

The inventive method plays an important role to text emotion analysis in the research of emotional semantic classification.Present invention employs one kind Combined deep learning model has carried out the research of sentiment analysis for product review.Deep learning method is compared to traditional SVM Deng shallow-layer sorting technique, the problems such as dimension disaster and Sparse will not be brought, lift system entirety classifying quality obvious. On identical data set, compared with common machines learning method, F1-score values improve 3% to the present invention.By corpus data The term vector that training is obtained is proved to containing specific structurally and semantically information, is a kind of good semantic meaning representation form.This Bright contribution is to be merged on convolutional neural networks model (CNN) and shot and long term memory models (LSTM), and wherein LSTM is used To be modeled to context, by the characteristic vector that context is obtained by word iteration, CNN models are used for from term vector sequence Automatically feature is found, and from local extraction feature, then is integrated into global characteristics to improve classifying quality by local feature.

The present invention proposes a kind of text sentiment classification method based on combined deep learning model.The method merges shot and long term Memory models (LSTM) carries out feeling polarities differentiation to comment data with convolutional neural networks (CNN), and the method adopts LSTM moulds Type is modeled to context, by the characteristic vector that context is obtained by word iteration, using CNN models from term vector sequence Automatically feature is found, and is integrated into global characteristics to improve classifying quality by local feature from after the extraction feature of local.

The present invention uses LSTM networks, is initialized in LSTM networks by training the term vector dictionary for obtaining Embedding layer parameters, while in the training process term vector carries out joint training, finally with the output of LSTM layers as CNN The input of layer.

Illustrate the flow process and effect of the invention.Such as sentence, " this photo-effect on mobile phone is very good, but price is expensive.” The sentence is represented with term vector, through the training of LSTM layers joint term vector is obtained, through CNN layer extraction features, finally Softmax layers export the probability of three classes, and the probable value for obtaining is [0.1,0.3,0.6], this positive class of three classes difference acute pyogenic infection of finger tip, bear class, Pass judgement on mixing class.The probable value that 3rd class is obtained is maximum, then the sentence is judged as passing judgement on mixing class.

Description of the drawings

Fig. 1 is the flow chart that the text emotion based on combined deep learning model is classified；

Fig. 2 is the combined deep learning model network structure that specific embodiment is proposed；

Fig. 3 is conventional network structure figure；

Fig. 4 be word2vec network structures, w in figure_iIt is a word in text, sequence<w_i-c..., w_i-1, w_i+1..., w_i+c>Represent word w_iContext, the logistic regression parameter required for Huffman tree interior joint is designated as

Specific embodiment

Specific embodiment one：As depicted in figs. 1 and 2, a kind of text emotion based on combined deep learning model is classified Method is comprised the following steps：

Step one：Each word in text data is processed, using word2vec instruments to the textual data after process Each word is trained according in, obtains term vector dictionary, and each of which word all correspond to a term vector；It is described Word2vec instruments are Google's term vector instrument；

Step 2：It is originally inputted as word sequence (w₁,….,w_s-1,w_s,…,w_max), obtain the matrix M of each sentence, M= (x₁,…,x_s-1,x_s,…,x_max)∈R^d×max；The matrix M of fixed length is trained and is converted into the vectorial right of fixed dimension by LSTM layers (embedding) input layer is improved, and generates term vectors of the d dimension h with context semantic relation (word in the training process Vector can change with training process)；Wherein described w₁,….,w_s-1,w_s,…,w_maxCorresponding to each word in each sentence Word label, x₁,…,x_s-1,x_s,…,x_maxIt is and w₁,….,w_s-1,w_s,…,w_maxOne-to-one term vector, d is that outside sets Fixed term vector dimension, LSTM is shot and long term memory models；

Step 3：The d generated in step 2 is tieed up the h term vector with context semantic relation as the input of CNN, Tieed up in the h term vector with context semantic relation from d as a trainable property detector using CNN and extract spy Levy, the CNN is convolutional neural networks；

Step 4：The feature extracted in step 3 is sequentially connected, the conduct after one layer of openness activation primitive The input of softmax, output obtains the probability of each classification, all categories probability and for 1, and the maximum classification of probable value is The classification predicted, result (output) of this classification as combined deep learning model.

If without activation primitive, no matter final how many layer of neutral net, it is different linear that output is all simply entered Combination, it is suitable with effect during only two-layer.And openness activation primitive is used, can preferably from the dimension of effective data On, automation dissociation effect is played in study to relatively sparse feature.

Embedding layers are embeding layer；LSTM is shot and long term memory models；CNN is convolutional neural networks, model training When word2vec can carry out joint training.Such as Fig. 3 and Fig. 4；Fig. 3 is conventional network structure figure, and Fig. 4 is word2vec networks knot Composition；

Art methods have all only used a kind of deep learning model mostly, obtain better effects using LSTM at present Long text is in the majority, and the short text for obtaining better effects using CNN is then more, therefore, the present invention is in sentiment analysis problem Long sentence and short sentence carry out modeling features different during emotional semantic classification, propose a kind of sorting technique based on combined deep learning model Emotional semantic classification is carried out to comment data.

Specific embodiment two：Present embodiment from unlike specific embodiment one：It is characterized in that：The step It is to the detailed process that each word in text data is processed in one：

Participle is carried out to sentence in text data using participle program (stammerer participle), spcial character is removed, retain the Chinese Word, English, punctuate and emoticon (spcial character such as phonetic symbol, additional character etc.).

Other steps and parameter are identical with specific embodiment one.

Specific embodiment three：Present embodiment from unlike specific embodiment one or two：The step 2 Central Plains Input begin for word sequence (w₁,….,w_s-1,w_s,…,w_max) be specially：

All words in text data are constituted into a word dictionary, gives each word one id label, by all of text data Word is replaced with id labels, obtains word sequence (w₁,….,w_s-1,w_s,…,w_max), as the input of combined deep learning model.

Other steps and parameter are identical with specific embodiment one or two.

Specific embodiment four：Unlike one of present embodiment and specific embodiment one to three：The step 2 In obtain each sentence the detailed process of matrix M be：

It is originally inputted as word sequence (w₁,….,w_s-1,w_s,…,w_max), wherein most long text length is taken as sentence length, 0 is mended before remaining sentence, the sentence length mended after 0 is identical with most long text length；Inquire about and obtain every from term vector dictionary The term vector of individual word, obtains the matrix M of each sentence, M=(x₁,…,x_s-1,x_s,…,x_max)∈R^d×max, in term vector dictionary Non-existent word uses 0^dReplace.

Other steps and parameter are identical with one of specific embodiment one to three.

Specific embodiment five：Unlike one of present embodiment and specific embodiment one to four：The step 2 Matrix M is trained and is converted into the vector of fixed dimension input layer is improved by middle LSTM layers, generate d dimensions h have it is upper Hereafter the detailed process of the term vector of semantic relation is：

The LSTM layers are made up of LSTM units, each sequential one LSTM unit of correspondence；LSTM units are in each sequential A term vector is input in order, and the artificial term vector value for arranging dimension of LSTM units output, the output valve of each sequential is passed through Concatenation generates the term vector with context semantic relation；

The output of LSTM units artificially arranges the process of the term vector value of dimension：

(1) i is made_t,f_t,o_t,c_t,s_t,h_tRepresent the input gate in moment t respectively, forget door, out gate, mnemon, The state of memory state and hidden neuron, is the vector of H dimensions, and wherein H is the number of hidden neuron in hidden layer；

(2) value i of input gate is calculated_t,

i_t=σ (W_xix_t+W_hih_t-1+W_cic_t-1+b_i)

Wherein, W_xiFor LSTM unit current time input datas x_tWeighting parameter, W_hiIt is defeated for upper moment LSTM unit Go out data h_t-1Weighting parameter, W_ciFor upper moment candidate's mnemon value c_t-1Weighting parameter, b_iFor offset parameter；σ is Activation primitive；

(3) value f for forgeing door is calculated_t,

f_t=σ (W_xfx_t+W_hfh_t-1+W_cfc_t-1+b_f)

Wherein, W_xfFor LSTM unit current time input datas x_tWeighting parameter, W_hfIt is defeated for upper moment LSTM unit Go out data h_t-1Weighting parameter, W_cfFor upper moment candidate's mnemon value c_t-1Weighting parameter, b_fFor offset parameter；

(4) current time mnemon value c is calculated_t,

Wherein, ⊙ represents pointwise product；

(5) out gate o is calculated_t,

o_t=σ (W_xox_t+W_hoh_t-1+W_coc_t-1+b_o)

Wherein, W_xoFor LSTM unit current time input datas x_tWeighting parameter, W_hoIt is defeated for upper moment LSTM unit Go out data h_t-1Weighting parameter, W_coFor upper moment candidate's mnemon value c_t-1Weighting parameter, b_oFor offset parameter；Input Data x_tExactly pre-process after each word table look-up after term vector

(6) LSTM units are output as

h_t=o_t⊙tanh(c_t)

Tanh is an activation primitive.

Using the LSTM layers context term vector that obtains of output as the input of CNN layers, convolutional layer is initially entered, it is and traditional Only using a kind of nuclear phase ratio of length, having used the core of different lengths carries out feature extraction, and multiple wave filters act on term vector Layer, different wave filters generate different feature map, obtain the feature graphic sequence that several columns are 1；

Next step carries out down-sampling, is operated using k-max.K-max operations are the extensions to traditional max operations, k here =2.It selects k maximum value, k-max to operate and cause a convolution nuclear energy to retain more features from each sequence.

Other steps and parameter are identical with one of specific embodiment one to four.

Specific embodiment six：Unlike one of present embodiment and specific embodiment one to five：The step 3 Middle employing CNN is tieed up in the h term vector with context semantic relation from d as a trainable property detector and is extracted spy The detailed process levied is：

The CNN layers are operated comprising convolution operation and down-sampling：

One sentence represents that every a line of matrix represents the term vector of a word, uses with the matrix that a d ties up h rows The convolution kernel of different length carries out computing, and the size for arranging convolution kernel is n kinds, and the scope of wherein n >=2, and size is upper and lower region Between s word, wherein 1≤s≤h；Every kind of size respectively has 100 convolution kernels, and each convolution kernel does convolution algorithm, obtains to matrix 100 × n columns is 1 characteristics dictionary；

Step 3 one：Assume that convolutional layer is using l-th characteristic pattern during size iJ-th convolution results of correspondenceConvolution kernel beConvolution operation enables model from extraction feature in the multiple words in local,WithConvolution As a result it is also vector, its p-th element be,

Step 3 two：It is down-sampling operation to follow closely after convolution operation, and down-sampling retains maximum in each convolution results Value, so as to neglect smaller noise.Operated using k-max, i.e., k maximum is retained to each vector, for example：

Step 3 three：Final down-sampling result again can be used as the input of softmax after an activation primitive.

Other steps and parameter are identical with one of specific embodiment one to five.

Specific embodiment seven：Unlike one of present embodiment and specific embodiment one to six：The step 4 In after activation primitive as the input of softmax, the detailed process for exporting the probability for obtaining each classification is：

The value to the output of CNN layers obtains a characteristic vector through concatenation, sends into Dropout layers, with The softmax layers connection of Dropout layers, using softmax multivariate classification is carried out, and softmax layers are exported between 0 to 1 Numerical value, the Dropout layers are loss property output layer, and it is logistic regression pushing away in many classification problems that softmax is returned Extensively；Softmax layers output valve be as the output detailed process of model：

The k dimensional vectors for assuming function output represent this k probable value for obtaining, then h_θ(x⁽ⁱ⁾) be：

Wherein x is the characteristic vector of Dropout networks output, and y is tag along sort, h_θ(x⁽ⁱ⁾) calculate is in input Characteristic vector is the probable value that y is respectively this three classes label under conditions of x；Based on combined deep learning text emotional semantic classification In framework.

Other steps and parameter are identical with one of specific embodiment one to six.

Embodiment one：

(1) sample extraction.2700 films of bean cotyledon obtained in data hall and scoring and the comment data of TV play (http://www.datatang.com/data/16607), to make experiment more representative, TV play comment therein is removed, After total data is integrated, randomly select and amounted to 26000 comments, the comment passed judgment on star, sky comment will be directed to Remove with non-Chinese comment, it is remaining 22530.

(2) data mark.The data scoring of acquisition is mostly commendation for the comment of 4 stars and 5 stars, and the comment of 1 star and 2 stars Mostly derogatory sense, and it is all kinds of in 3 stars have, the comment of 4 stars and 5 stars is defined as commendation, and the comment of 1 star and 2 stars is defined as derogatory sense, from Choose in 3 stars and pass judgement on hybrid category, finally give training data, comprising positive class 8976 are commented on, bear class and comment on 9528, pass judgement on Mixing class comments on 896.

(3) training sample of all categories is sent into into combined deep learning model framework proposed by the present invention, after training Obtain one three classification sentiment classification model.

(4) new test sample (also from said method and corpus) is sent into into model, is tied according to the prediction of model Fruit and the legitimate reading of test sample, obtain test data.

Test result is as follows：

(5) by COAE2016 (whole nation Chinese tendentiousness evaluation and test in 2016) text emotion classification task (subtask 2) Test data sends into model, predicting the outcome and evaluate and test the data of unit feedback according to model, it was demonstrated that of the invention in the subtask In achieve evaluation and test best result.

Feedback result is as follows：

The present invention can also have other various enforcements, in the case of without departing substantially from spirit of the invention and its essence, this area skill Art personnel work as can make various corresponding changes and deformation according to the present invention, but these corresponding changes and deformation should all belong to this The appended scope of the claims of invention.

Claims (7)

1. a kind of text sentiment classification method based on combined deep learning model, it is characterised in that：It is described based on combined depth The text sentiment classification method of learning model is comprised the following steps：

Step one：Each word in text data is processed, using word2vec instruments in the text data after process Each word is trained, and obtains term vector dictionary, and each of which word all correspond to a term vector；The word2vec works Have for Google's term vector instrument；

Step 2：It is originally inputted as word sequence (w₁,....,w_s-1,w_s,…,w_max), obtain the matrix M of each sentence, M= (x₁,…,x_s-1,x_s,…,x_max)∈R^d×max；Matrix M is trained the vector for being converted into fixed dimension to input layer by LSTM layers It is improved, generates h term vector with context semantic relation of d dimensions；Wherein described w₁,....,w_s-1,w_s,…,w_maxFor Word label in each sentence corresponding to each word, x₁,…,x_s-1,x_s,…,x_maxIt is and w₁,....,w_s-1,w_s,…,w_maxOne One corresponding term vector, LSTM is shot and long term memory models；

Step 3：The d generated in step 2 is tieed up the h term vector with context semantic relation as the input of CNN, is adopted CNN ties up extraction feature in the h term vector with context semantic relation, institute as a trainable property detector from d CNN is stated for convolutional neural networks；

Step 4：The feature extracted in step 3 is sequentially connected, as the defeated of softmax after openness activation primitive Enter, output obtains the probability of each classification, all categories probability and for 1, the maximum classification of probable value is predicted class Not, result of this classification as combined deep learning model.

2. a kind of text sentiment classification method based on combined deep learning model according to claim 1, its feature exists In：It is to the detailed process that each word in text data is processed in the step one：

Participle is carried out to sentence in text data using participle program, spcial character is removed, retain Chinese character, English, punctuate and Emoticon.

3. a kind of text sentiment classification method based on combined deep learning model according to claim 1 and 2, its feature It is：It is originally inputted in the step 2 as word sequence (w₁,....,w_s-1,w_s,…,w_max) be specially：

All words in text data are constituted into a word dictionary, gives each word one id label, all words of text data are used Id labels are replaced, and obtain word sequence (w₁,....,w_s-1,w_s,…,w_max), as the input of combined deep learning model.

4. a kind of text sentiment classification method based on combined deep learning model according to claim 3, its feature exists In：The detailed process that the matrix M of each sentence is obtained in the step 2 is：

It is originally inputted as word sequence (w₁,....,w_s-1,w_s,…,w_max), wherein most long text length is taken as sentence length, its 0 is mended before remaining sentence, the sentence length mended after 0 is identical with most long text length；Inquire about from term vector dictionary and obtain each The term vector of word, obtains the matrix M of each sentence, M=(x₁,…,x_s-1,x_s,…,x_max)∈R^d×max, in term vector dictionary not The word of presence uses 0^dReplace.

5. a kind of text sentiment classification method based on combined deep learning model according to claim 1,2 or 4, it is special Levy and be：Matrix M is trained and is converted into the vector of fixed dimension input layer is improved by LSTM layers in the step 2, Generation d ties up the detailed process of the h term vector with context semantic relation：

The LSTM layers are made up of LSTM units, each sequential one LSTM unit of correspondence；LSTM units are in each sequential by suitable Sequence is input into a term vector, and the artificial term vector value for arranging dimension of LSTM units output, the output valve of each sequential passes through splicing Operation generates the term vector with context semantic relation；

The output of LSTM units artificially arranges the process of the term vector value of dimension：

(1) i is made_t,f_t,o_t,c_t,s_t,h_tThe input gate in moment t is represented respectively, forgets door, out gate, mnemon, memory The state of state and hidden neuron, is the vector of H dimensions, and wherein H is the number of hidden neuron in hidden layer；

(2) value i of input gate is calculated_t,

i_t=σ (W_xix_t+W_hih_t-1+W_cic_t-1+b_i)

Wherein, W_xiFor LSTM unit current time input datas x_tWeighting parameter, W_hiNumber is exported for upper moment LSTM unit According to h_t-1Weighting parameter, W_ciFor upper moment candidate's mnemon value c_t-1Weighting parameter, b_iFor offset parameter；σ is activation Function；

(3) value f for forgeing door is calculated_t,

f_t=σ (W_xfx_t+W_hfh_t-1+W_cfc_t-1+b_f)

Wherein, W_xfFor LSTM unit current time input datas x_tWeighting parameter, W_hfNumber is exported for upper moment LSTM unit According to h_t-1Weighting parameter, W_cfFor upper moment candidate's mnemon value c_t-1Weighting parameter, b_fFor offset parameter；

(4) current time mnemon value c is calculated_t,

Wherein, ⊙ represents pointwise product；

(5) out gate o is calculated_t,

o_t=σ (W_xox_t+W_hoh_t-1+W_coc_t-1+b_o)

Wherein, W_xoFor LSTM unit current time input datas x_tWeighting parameter, W_hoNumber is exported for upper moment LSTM unit According to h_t-1Weighting parameter, W_coFor upper moment candidate's mnemon value c_t-1Weighting parameter, b_oFor offset parameter；

(6) LSTM units are output as

h_t=o_t⊙tanh(c_t)。

6. a kind of text sentiment classification method based on combined deep learning model according to claim 5, its feature exists In：H being tieed up in the step 3 from d as a trainable property detector using CNN, there is context semantic relation The detailed process of extraction feature is in term vector：

The CNN layers are operated comprising convolution operation and down-sampling：

The matrix for tieing up h rows with a d represents that every a line of matrix represents the term vector of a word, using the volume of different length Product core carries out computing, and the size for arranging convolution kernel is n kinds, and the scope of wherein n >=2, and size is the s word in upper and lower interval, its In 1≤s≤h；Every kind of size respectively has 100 convolution kernels, and each convolution kernel does convolution algorithm to matrix, obtains 100 × n columns For 1 characteristics dictionary；

Step 3 one：Assume that convolutional layer is using l-th characteristic pattern during size iJ-th convolution results of correspondenceConvolution Core isConvolution operation enables model from extraction feature in the multiple words in local,WithConvolution results be also to Amount, its p-th element be,

${\&lsqb;m_{j,i}^l*F_i^l\&rsqb;}_p=\underset{k=p}{\overset{p+M-1}{\&Sigma;}}{\&lsqb;F_i^l\&rsqb;}_k{\&lsqb;m_{j,i}^l\&rsqb;}_{k-p+1}$

Step 3 two：It is down-sampling operation to follow closely after convolution operation, and down-sampling retains value maximum in each convolution results, So as to neglect smaller noise；Operated using k-max, i.e., k maximum is retained to each vector；

Step 3 three：Final down-sampling result again can be used as the input of softmax after an activation primitive.

7. a kind of text sentiment classification method based on combined deep learning model according to claim 6, its feature exists In：Input in the step 4 after activation primitive as softmax, output obtains the concrete mistake of the probability of each classification Cheng Wei：

The value to the output of CNN layers obtains a characteristic vector through concatenation, Dropout layers is sent into, with Dropout layers The connection of softmax layers, carry out multivariate classification using softmax, softmax layers export the numerical value between 0 to 1, described Dropout layers are loss property output layer, and softmax layers output valve is as the output detailed process of model：

The k dimensional vectors for assuming function output represent this k probable value for obtaining, then h_θ(x⁽ⁱ⁾) be：

$h_{\mathrm{\&theta;}}\left(x^{\left(i\right)}\right)=\left[\begin{array}{c}p\left(y^{\left(i\right)}=1|x^{\left(i\right)};\mathrm{\&theta;}\right)\\ p\left(y^{\left(i\right)}=2|x^{\left(i\right)};\mathrm{\&theta;}\right)\\ p\left(y^{\left(i\right)}=3|x^{\left(i\right)};\mathrm{\&theta;}\right)\\ \mathrm{......}\\ p\left(y^{\left(i\right)}=k-1|x^{\left(i\right)};\mathrm{\&theta;}\right)\\ p\left(y^{\left(i\right)}=k|x^{\left(i\right)};\mathrm{\&theta;}\right)\end{array}\right]=\frac{1}{\underset{j=1}{\overset{k}{\&Sigma;}}{e}^{{\mathrm{\&theta;}}_j^T{x}^{\left(i\right)}}}\left[\begin{array}{c}{e}^{{\mathrm{\&theta;}}_1^T{x}^{\left(i\right)}}\\ e^{{\mathrm{\&theta;}}_2^T{x}^{\left(i\right)}}\\ \mathrm{...}\\ e^{{\mathrm{\&theta;}}_{k-1}^T{x}^{\left(i\right)}}\\ e^{{\mathrm{\&theta;}}_k^T{x}^{\left(i\right)}}\end{array}\right]$

Wherein x is the characteristic vector of Dropout networks output, and y is tag along sort, h_θ(x⁽ⁱ⁾) calculate be input feature to Amount is the probable value that y is respectively this three classes label under conditions of x.