CN110059191A - A kind of text sentiment classification method and device - Google Patents
A kind of text sentiment classification method and device Download PDFInfo
- Publication number
- CN110059191A CN110059191A CN201910375929.5A CN201910375929A CN110059191A CN 110059191 A CN110059191 A CN 110059191A CN 201910375929 A CN201910375929 A CN 201910375929A CN 110059191 A CN110059191 A CN 110059191A
- Authority
- CN
- China
- Prior art keywords
- text
- convolutional neural
- neural networks
- networks model
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The present disclosure discloses a kind of text sentiment classification method and devices, this method comprises: receiving text data, construct distributed term vector, obtain eigenmatrix；Eigenmatrix is inputted into convolutional neural networks model, convolution algorithm is carried out to eigenmatrix by the convolutional layer of the multi-slide-windows mouth of the convolutional neural networks model；By the k-max and avg-pooling of the convolutional neural networks model, double pond layers carry out pondization operation parallel；By the global characteristics vector of the concatenation layer splicing building text level of the convolutional neural networks model, Naive Bayes Classifier is used to obtain text emotion classification results to after its dimensionality reduction；The convolutional neural networks model parameter is optimized using improved gradient descent algorithm.
Description
Technical field
The disclosure belongs to the technical field of natural language processing and deep learning, be related to a kind of text sentiment classification method and
Device.
Background technique
Only there is provided background technical informations relevant to the disclosure for the statement of this part, it is not necessary to so constitute first skill
Art.
Along with being obviously improved for computer communication level, the role of Internet user is also quiet by the receiver of script
So it is changed into the creator of information.At the same time, social media has also stepped into the fast traffic lane of development, and user is promoted to form with short
Expression way based on text.Therefore, the sentiment analysis of short text is provided with more importantly application value.Text emotion analysis
It is with sentiment dictionary, data mining, the technologies such as machine learning are support, through the actual content of text, handle and obtain author
To the animus of content of text, mood and viewpoint.To preferably react on production application and social practice.Network short essay
Originally user's feelings information abundant is contained, there is research significance abundant.
Mainly include two major classes based on traditional short text sentiment analysis: method based on sentiment dictionary and being based on engineering
The method of habit.Feeling polarities are mainly determined by emotion word based on the method for sentiment dictionary, but sentiment dictionary needs people
Work building, this method judge the Sentiment orientation of sentence entirety using the emotional color of emotion word, do not account for context pass
System is a kind of sensibility classification method of opposite bottom.The latter is mainly using the method for the machine learning for having supervision, with training
Classifier to text carry out emotional semantic classification, although it has compared with traditional sentiment dictionary method in the judgement of feeling polarities
Apparent raising, it is contemplated that context semantic information, but there is also sparse feature representation is limited to, each is special
Sign is all indicated with the sparse vector of a higher-dimension, it is difficult to differentiate semantic similar feature, it is special to frequently rely on artificial extraction
The problems such as sign.
Since in recent years, as the high quality of depth learning technology develops, in the intelligent Understanding of large scale text data
On show unique advantage, existing such as LSTM, the neural network models such as RNN, CNN are largely applied
Text emotion is analyzed and achieves good experimental result, wherein Kim etc. is on multiple emotional semantic classification data sets by a variety of depths
Degree learning model is compared, as a result, it has been found that convolutional neural networks are in text emotion analysis, especially short text sentiment analysis
In, there is better experiment effect.However, inventor has found in R&D process, convolutional neural networks side of classification the emotion more
The problems such as context is semantic weak is extracted there is also accuracy is not high in face.Although changing for convolutional neural networks in recent years
Into emerging one after another, but it still can not preferably solve the problems, such as the random and semantic profound level of network short text creation.
Summary of the invention
For the deficiencies in the prior art, one or more other embodiments of the present disclosure provide a kind of text emotion point
Class method and device realizes text after improving in conjunction with improvement gradient descent algorithm (PSGD) and to convolutional neural networks structure
Emotional semantic classification, effectively makes up existing convolutional neural networks file classification method there is accuracy not high, and it is semantic to extract context
It is weak, can not effectively solve the problems such as pond layer dimensionality reduction loses semantic information and parameter more new algorithm is unstable.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of text sentiment classification method is provided.
A kind of text sentiment classification method, this method comprises:
Text data is received, distributed term vector is constructed, obtains eigenmatrix；
Eigenmatrix is inputted into convolutional neural networks model, passes through the multi-slide-windows mouth of the convolutional neural networks model
Convolutional layer carries out convolution algorithm to eigenmatrix；
By the k-max and avg-pooling of the convolutional neural networks model, double pond layers carry out pondization operation parallel；
By the global characteristics vector of the concatenation layer splicing building text level of the convolutional neural networks model, it is dropped
Text emotion classification results are obtained with Naive Bayes Classifier after dimension；
The convolutional neural networks model parameter is optimized using improved gradient descent algorithm.
Further, in the method, data prediction is carried out to received text data, by the text after data prediction
Notebook data passes through word2vec tools build distribution term vector；
The data prediction includes data cleansing and participle.
Further, in the method, the convolutional layer includes the different convolution kernel of multiple window sizes, passes through the volume
The convolution kernel that multiple window sizes of product neural network model are different carries out convolution algorithm to the eigenmatrix, extracts different window
The text part semantic vector of mouth size；
Each convolutional layer includes the convolution unit of several concurrent operations.
Further, in the method, the convolutional neural networks model further includes filter unit, by the convolutional layer
The text part semantic vector that convolution algorithm obtains completes feature extraction by the filter unit.
Further, in the method, double pond layers include parallel avg- to the k-max and avg-pooling parallel
The pond pooling layer and the pond k-max layer.
Further, in the method, the pond k-max layer determines characteristic pattern down-sampling according to convolution kernel height
Number, the convolution kernel height are inversely proportional with characteristic pattern down-sampling number.
Further, in the method, dimensionality reduction is carried out to global characteristics vector using PCA.
Further, in the method, described that the convolutional neural networks mould is optimized using improved gradient descent algorithm
Shape parameter is the batch data training set for forming convolutional neural networks model by choosing the higher sample of data dependence.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of computer readable storage medium is provided.
A kind of computer readable storage medium, wherein being stored with a plurality of instruction, described instruction is suitable for by terminal device
Reason device loads and executes a kind of text sentiment classification method.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of terminal device is provided.
A kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；Meter
Calculation machine readable storage medium storing program for executing is suitable for being loaded by processor and being executed a kind of text for storing a plurality of instruction, described instruction
Sensibility classification method.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of text emotion sorter is provided.
A kind of text emotion sorter, based on a kind of text sentiment classification method, comprising:
Data acquisition module is configured as receiving text data, constructs distributed term vector, obtains eigenmatrix；
Convolutional neural networks module is configured as eigenmatrix inputting convolutional neural networks model, passes through the convolution
The convolutional layer of the multi-slide-windows mouth of neural network model carries out convolution algorithm to eigenmatrix；Pass through the convolutional neural networks mould
Double pond layers carry out pondization operation to the k-max and avg-pooling of type parallel；Pass through the concatenation of the convolutional neural networks model
The global characteristics vector of layer splicing building text level, uses Naive Bayes Classifier to obtain text emotion point to after its dimensionality reduction
Class result；
Parameter optimization module is configured to optimize the convolutional neural networks model ginseng using improved gradient descent algorithm
Number.
The disclosure the utility model has the advantages that
(1) a kind of text sentiment classification method and device that the disclosure provides, mainly for network short text information on line
The random convolution algorithm layer that more size multi-slide-windows mouths are devised with the characteristics of semantic profound level of creation, it is hidden that depth excavates its
Containing information；The problem of expense is big, and training pattern can not support huge data volume is constructed to term vector, disclosure combination PCA is utilized
The thought of dimensionality reduction carries out data normalization to high-dimensional data, then acquires the covariance matrix and its correspondence of objective matrix
Feature vector, initial data are transformed into a kind of expression of any dimension linear independence finally by linear transformation, thus
Multi objective (high-dimensional) is converted into a few main characteristic component, to reduce expense；
(2) a kind of text sentiment classification method and device that the disclosure provides, for current convolutional neural networks text point
Class model can not effectively solve the problems, such as that pond layer dimensionality reduction loses semantic information, merge the pond k-max method Dynamic Extraction
Speciality and the pond avg-pooling method propose in conjunction with k-max and avg- the average semantic contribution ability of short text
Parallel double pond layer structures of pooling method carry out pondization operation, retain deeper as far as possible while reducing expense
Secondary semantic information；
(3) a kind of text sentiment classification method and device that the disclosure provides, assume only with Naive Bayes Classifier
Vertical, less sensitive to missing data, also fairly simple feature replaces softmax classifier to improve the standard of classification to algorithm
True rate；
(4) a kind of text sentiment classification method and device, improved gradient descent algorithm that the disclosure provides solve BGD
When number of samples is very big, each iteration requires to calculate all samples algorithm, and training time expense is big；SGD algorithm is frequent
Falling into locally optimal solution restrain model can not；And the experience value etc. that Mini-BGD extremely relies on batch_size is asked
Topic；The disclosure, which mentions improved gradient descent algorithm, can guarantee the stability of model, improve training speed, shorten model convergence
Time.
Detailed description of the invention
The Figure of description for constituting a part of this disclosure is used to provide further understanding of the disclosure, and the disclosure is shown
Meaning property embodiment and its explanation do not constitute the improper restriction to the disclosure for explaining the disclosure.
Fig. 1 is a kind of text sentiment classification method flow chart according to one or more embodiments；
Fig. 2 is the CNNpbc file classification method internal structure chart according to one or more embodiments；
Fig. 3 is the convolutional layer schematic diagram according to the multi-slide-windows mouth size of one or more embodiments；
Fig. 4 is parallel double pond layer structure charts according to the k-max+avg-pooling of one or more embodiments；
Fig. 5 is the pond the k-max method structure chart according to one or more embodiments；
Fig. 6 is the stability experiment result line chart according to the verifying PSGD algorithm of one or more embodiments；
Fig. 7 is the training speed experimental result line chart according to the verifying PSGD algorithm of one or more embodiments.
Specific embodiment:
Below in conjunction with the attached drawing in one or more other embodiments of the present disclosure, to one or more other embodiments of the present disclosure
In technical solution be clearly and completely described, it is clear that described embodiment is only disclosure a part of the embodiment,
Instead of all the embodiments.Based on one or more other embodiments of the present disclosure, those of ordinary skill in the art are not being made
Every other embodiment obtained under the premise of creative work belongs to the range of disclosure protection.
It is noted that following detailed description is all illustrative, it is intended to provide further instruction to the disclosure.Unless another
It indicates, all technical and scientific terms that the present embodiment uses have and disclosure person of an ordinary skill in the technical field
Normally understood identical meanings.
It should be noted that term used herein above is merely to describe specific embodiment, and be not intended to restricted root
According to the illustrative embodiments of the disclosure.As used herein, unless the context clearly indicates otherwise, otherwise singular
Also it is intended to include plural form, additionally, it should be understood that, when in the present specification using term "comprising" and/or " packet
Include " when, indicate existing characteristics, step, operation, device, component and/or their combination.
It should be noted that flowcharts and block diagrams in the drawings show according to various embodiments of the present disclosure method and
The architecture, function and operation in the cards of system.It should be noted that each box in flowchart or block diagram can represent
A part of one module, program segment or code, a part of the module, program segment or code may include one or more
A executable instruction for realizing the logic function of defined in each embodiment.It should also be noted that some alternately
Realization in, function marked in the box can also occur according to the sequence that is marked in attached drawing is different from.For example, two connect
The box even indicated can actually be basically executed in parallel or they can also be executed in a reverse order sometimes,
This depends on related function.It should also be noted that each box and flow chart in flowchart and or block diagram
And/or the combination of the box in block diagram, the dedicated hardware based system that functions or operations as defined in executing can be used are come
It realizes, or the combination of specialized hardware and computer instruction can be used to realize.
In the absence of conflict, the feature in the embodiment and embodiment in the disclosure can be combined with each other, and tie below
It closes attached drawing and embodiment is described further the disclosure.
Embodiment one
In order to realize full online data stream cluster, the purpose of efficiency of algorithm is improved, in fact according to the one or more of the disclosure
The one aspect for applying example provides a kind of text sentiment classification method.
As shown in Figure 1, a kind of text sentiment classification method, this method comprises:
Step S1: receiving text data, constructs distributed term vector, obtains eigenmatrix；
Step S2: inputting convolutional neural networks model for eigenmatrix, passes through the mostly sliding of the convolutional neural networks model
The convolutional layer of dynamic window carries out convolution algorithm to eigenmatrix；
Step S3: by the k-max and avg-pooling of the convolutional neural networks model, double pond layers carry out pond parallel
Change operation；
Step S4: by the convolutional neural networks model concatenation layer splicing building text level global characteristics to
Amount uses Naive Bayes Classifier to obtain text emotion classification results to after its dimensionality reduction；
Step S5: the convolutional neural networks model parameter is optimized using improved gradient descent algorithm.
It is not high that there is accuracy in order to make up existing convolutional neural networks file classification method, and it is thin to extract context semanteme
It is weak, can not effectively solve pond layer dimensionality reduction loss semantic information and parameter the problems such as more new algorithm is unstable, be embodied in base
It is lower in the presence of classification accuracy in traditional sentiment dictionary and the file classification method and convolutional neural networks method of machine learning,
The problems such as part of speech is excessively single, and feature is sparse.
The present embodiment proposes in conjunction with the CNNpbc file classification method for improving gradient descent algorithm and handles above-mentioned presence
The problem of, the optimization of system has been carried out to convolutional neural networks model internal structure, it is distributed in building by CNNpbc method
It is constructed when term vector with word2vec method；Convolution algorithm layer using the convolutional layer of multi-slide-windows mouth come
Extract the semantic information of different depth；In pond, layer carries out pond using parallel double pond structures of k-max+avg-pooling
Operation, this method has not only merged the speciality of the pond k-max layer Dynamic Extraction, and combines avg-pooling to short essay
This average semantic contribution ability, to reduce the loss of semantic information after dimensionality reduction；After PCA technology dimensionality reduction, exporting
Layer carries out text emotion classification using Naive Bayes Classifier.In conjunction with PCA dimensionality reduction and Naive Bayes Classifier to training set
Quantity demand is few, and model restrains fast feature.It is excessively high to solve feature vector dimension, model training expense is greatly and traditional convolution is refreshing
The problem of ignoring word context semanteme and syntactic information through network model；It is finally right using improved gradient descent algorithm (PSGD)
Model parameter optimizes, and improves the convergence rate of model.
The present embodiment verifies the classifying quality of institute's climbing form type using Stamford SST data set and Connell MRD data set, real
Test the result shows that, the present embodiment propose CNNpbc (Convolutional Neural Network plus Bayes
Classifier) file classification method can preferably retain text semantic feature, guarantee the stability of training process and raising
The convergence rate of model, improves the precision of classification.
In the step S1 of the present embodiment, distributed term vector is constructed, the disclosure first carries out content of text necessary
Data cleansing and participle, later by word2vec tools build distribution term vector, by word be expressed as one it is dense and low
The vector of dimension.
Short text first passes around data prediction on untreated line, removes noise and is divided using jieba tool
Word；Then by word2vec tools build distribution term vector, word is expressed as dense and low-dimensional a vector.By one
A short text T:{ Wd being made of n word1, Wd2, Wd3...WdnEach of word WdiBe converted to R1*vIn space
Vector Wdi:
wdi=(mi1, mi2..., miv) i ∈ [1, n]
In this way, short text T is converted into Rn×vThe matrix of feature space:
We are using matrix T as the input of the mentioned CNNpbc file classification method convolutional layer of the disclosure.
In the step S2 of the present embodiment, using the different convolution kernel of multiple window sizes to the eigenmatrix T of input
Convolution algorithm is carried out, different convolutional layers has the size of different sliding windows, and has multiple filters under each convolutional layer
Wave device unit, in this way can be with deeper time, the excavation text intension of different emphasis.
As shown in Fig. 2, CNNpbc file classification method uses the different convolution kernel of multiple window sizes to the spy of input
It levies matrix T and carries out convolution algorithm, the context part for extracting different windows size is semantic.Wherein, grey square frame, which represents, passes through
Avg-pooling layers Chi Huahou's as a result, sliding window convolutional layer of the same size we select similar color to replace.Output
Avg-pooling result carry out assembly in the dynamic result that concatenation layer and k-max exports and combined, composition part semantic feature
Vector finally concatenates the pond of variant sliding window as a result, constituting final global semantic feature vector.As shown in figure 3,
CNNpbc file classification method can provide the different convolution kernel of n window size when carrying out convolution operation, use { k respectively1,k2,
k3...knIndicate, each convolutional layer is made of mup convolution unit, and after the operation of volume machine starts, all convolution units will be simultaneously
Row operation, and result is sent to new pond layer and carries out the extensive processing of feature.When convolutional layer carries out operation, kjT in window
The calculation formula of a convolution unit is as shown in formula 1, wherein 0≤t≤mup-1
Wherein, n is the length of text sequence T, Dj,tIt is with KjFor sliding window, to the term vector feature being made of text
Matrix, by n-kjAfter the operation of+1 secondary volume machine, the text part semantic vector of generation.Dj,tIt is n-k that vector, which will fall in dimension,j+ 1
In real number space.Wj,tIt is kjThe weight matrix of t convolution unit of window,Equally, bj,tIt is kjT in window
The biasing of a convolution unit, i.e. bj,l∈R.Matrix Zj,iOne is represented with KjIt is combined for the term vector of window.With text feature square
I-th of term vector wd in battle array TiCentered on, matrix Zj,iIt is generated by each kj/2 term vector concatenation in front and back, formula 2 are as follows:
In the step S3 of the present embodiment, using the parallel double of combination k-max and the avg-pooling method proposed
Pond layer structure carries out pondization operation, retains deeper semantic information as far as possible while reducing expense.
As shown in figure 4, using combining parallel double pond layers of k-max and avg-pooling method to carry out pondization operation, text
Eigen vector has been input into two not after the different convolutional layer of window size and its lower filter unit feature extraction
Dimensionality reduction --- the pond avg layer and the pond k-max layer (blue box replacement) sufficiently combine k-max for same parallel pond layer progress
The speciality and avg-pooling of pond layer Dynamic Extraction effectively reduce dimensionality reduction damage to the average semantic contribution ability of short text
The case where losing semantic information；Necessary semantic splicing finally is carried out in concatenation layer, constitutes global Text eigenvector；Wherein,
K-max method is as shown in figure 5, k-max method considers the influence to the height of convolution kernel sliding window to characteristic pattern is generated.
I.e. using convolution kernel height as the important evidence of characteristic pattern down-sampling number m, convolution kernel is higher, and down-sampling number is just few, conversely,
Convolution kernel height is lower, and down-sampling number is more.The value formula of down-sampling number m is as follows:
Wherein, h represents the convolution kernel i.e. height of sliding window, and (control is in 30 words for the length of behalf short text sentence
Within symbol).Compared to maximum value pondization strategy, k-max method can dynamically be mentioned according to the characteristic of multi-slide-windows mouth convolutional layer
Take multiple important semantic combination features, the relative ranks relationship between keeping characteristics.In Fig. 4, the short text length of input
It is 6, we carry out dynamic pondization strategy by the convolution kernel of selection h=2 and h=3, and the part with shade indicates to extract in figure
The more important feature come.
In the step S4 of the present embodiment, global Text eigenvector is obtained through beading process and carries out dimensionality reduction with PCA
After processing, we are independently assumed using Naive Bayes Classifier, expense is small, and the feature less sensitive to missing data replaces
Common softmax classifier carries out text feature classification work.
Dimensionality reduction is carried out to global characteristics vector with PCA technology, and serves as text feelings with Naive Bayes Classifier
Feel feature classifiers.PCA utilizes the thought of dimensionality reduction, carries out data normalization to high-dimensional data first, then acquires target
Initial data are transformed into one kind arbitrarily finally by linear transformation by the covariance matrix of matrix feature vector corresponding with its
The expression of dimension linear independence, so that multi objective (high-dimensional) is converted into a few main characteristic component, principle are as follows:
If original data set is arranged as Am×nEach row element of matrix is carried out zero averaging, calculation expression by matrix
As shown in formula 3, wherein aijRepresenting matrix Am×nThe i-th row j column element,Representing matrix Am×nThe average value of i-th row, SiTable
Show matrix Am×nThe standard deviation of i-th row:
After zero averaging, the covariance matrix S of matrix X is acquired, formula 4 is as follows:
N indicates the number of sample in above formula, after acquiring covariance matrix S, accordingly obtains its eigenvalue λ1≥λ2≥…≥
λnAnd feature vector d1,d2,…dn；, feature vector is obtained into matrix P according to the descending arrangement of characteristic value, is obtained by formula 5
Data Y after to dimensionality reduction finally calculates the contribution rate V of each characteristic root according to formula 6i
Y=PA (5)
After PCA carries out at dimensionality reduction, we are independent it is assumed that less quick to missing data using Naive Bayes Classifier
The characteristics of sense, replaces common softmax classifier, we set xi∈ X (1 < i < n) represents any of characteristic attribute set X
Characteristic attribute, n represent the characteristic attribute number of matrix X, first with Bayes formula simplicity principle, it is assumed that each condition is only
Vertical to occur, wherein d is the element number of classification results option C.Then there is Cj, j ∈ d meets formula 7:
Then it brings Bayes' theorem into and obtains classification results CjPosterior probability under characteristic attribute X generation, such as formula 8:
The sentence is finally attributed to by the sentence classification with maximum a posteriori probability, sorting criterion according to maximum a posteriori probability
It is as shown in formula 9:
hnbRepresent one by navie bayesian (nb) algorithm train Lai hypoth (assuming that), codomain output is
Under factor of the Bayes classifier for giving X, the c of most probable appearancej.C is its value set.
In the step S5 of the present embodiment, a kind of formed by the selection higher sample of data dependence in batches is devised
Gradient descent algorithm PSGD (the Partial Sampling Gradient Descent) of the data training set algorithm can be protected
The stability of model of a syndrome improves training speed, shortens model convergence time.
Gradient descent method as neural network update weight core optimization algorithm, in recent years develop since mainly by with
Lower three kinds of forms: stochastic gradient descent (SGD), batch gradient decline (BGD) and small lot gradient descent algorithm (Mini-
BGD).But they are there is also various problem, for example, BGD algorithm is when number of samples is very big, each iteration is required pair
All samples calculate, and training time expense is very big；Although SGD algorithm is not due to being the building loss letter on whole training datasets
Number, so that renewal speed is very fast, however accuracy rate declines, and the problem of falling into local optimum restrain model can not；
And Mini-BGD extremely relies on the experience value of batch_size；It is devised based on this disclosure a kind of by choosing data
The higher sample of correlation forms gradient descent algorithm (the Partial Sampling Gradient of batch data training set
Descent) algorithm can guarantee the stability of model, training speed improved, model convergence time is shortened.Its parameter updates
It is as shown in formula 10:
Wherein, θ is Optimal Parameters, and η is learning rateFor parameter gradients.Loss function this algorithm of loss using
Cross entropy.Specific calculation is as follows:
As Figure 6-Figure 7, it tests to verify the stability and training speed of PSGD algorithm, has chosen BGD respectively,
Tri- algorithms of SGD, Mini-batch as a control group, have carried out two groups of check experiments, and data set has chosen in section experiment and uses
MDR data set, the configuration of all hyper parameters is as follows, and wherein parameter k represents the value of convolution kernel window size, and mup is represented
The convolution unit quantity that each convolutional calculation layer is possessed, dwrd represent term vector dimension, are based on CNN short essay herein by reference to other
This disaggregated model retains the hidden unit of input layer and full articulamentum using the probability of Dropout=0.5 at random, and Fhn represents complete
Hidden layer nerve quantity in articulamentum.
Fig. 6 embodies experiment nicety of grading and changes with time rate；Fig. 7 demonstrates PSGD algorithm with the relatively low time
Complexity.Experimental result can be seen that the increase with the number of iterations, more steady using BGD algorithm optimization model training process
Fixed, nicety of grading is also relatively high, but due to the mode of its full sample training, the training time is more long；Although SGD algorithm is instructed
It is shorter to practice the time, but there are biggish noise, stability is insufficient；Mini-BGD algorithm only needs part Sample Refreshment mould every time
Compare the golden mean of the Confucian school in terms of shape parameter, accuracy and stability, without clear superiority；And PSGD algorithm is in this experimentation training time
Shorter training process is also more stable, and precision is also higher, to demonstrate the validity of proposed algorithm.
Embodiment two
According to the one aspect of one or more other embodiments of the present disclosure, a kind of computer readable storage medium is provided.
A kind of computer readable storage medium, wherein being stored with a plurality of instruction, described instruction is suitable for by terminal device
Reason device loads and executes a kind of text sentiment classification method.
Embodiment three
According to the one aspect of one or more other embodiments of the present disclosure, a kind of terminal device is provided.
A kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；Meter
Calculation machine readable storage medium storing program for executing is suitable for being loaded by processor and being executed a kind of text for storing a plurality of instruction, described instruction
Sensibility classification method.
These computer executable instructions execute the equipment according to each reality in the disclosure
Apply method or process described in example.
In the present embodiment, computer program product may include computer readable storage medium, containing for holding
The computer-readable program instructions of row various aspects of the disclosure.Computer readable storage medium, which can be, can keep and store
By the tangible device for the instruction that instruction execution equipment uses.Computer readable storage medium for example can be-- but it is unlimited
In-- storage device electric, magnetic storage apparatus, light storage device, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned
Any appropriate combination.The more specific example (non exhaustive list) of computer readable storage medium includes: portable computing
Machine disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or
Flash memory), static random access memory (SRAM), Portable compressed disk read-only memory (CD-ROM), digital versatile disc
(DVD), memory stick, floppy disk, mechanical coding equipment, the punch card for being for example stored thereon with instruction or groove internal projection structure, with
And above-mentioned any appropriate combination.Computer readable storage medium used herein above is not interpreted instantaneous signal itself,
The electromagnetic wave of such as radio wave or other Free propagations, the electromagnetic wave (example propagated by waveguide or other transmission mediums
Such as, pass through the light pulse of fiber optic cables) or pass through electric wire transmit electric signal.
Computer-readable program instructions described herein can be downloaded to from computer readable storage medium it is each calculate/
Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network
Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway
Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted
Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment
In calculation machine readable storage medium storing program for executing.
Computer program instructions for executing present disclosure operation can be assembly instruction, instruction set architecture (ISA)
Instruction, machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programmings
The source code or object code that any combination of language is write, the programming language include the programming language-of object-oriented such as
C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer-readable program refers to
Order can be executed fully on the user computer, partly be executed on the user computer, as an independent software package
Execute, part on the user computer part on the remote computer execute or completely on a remote computer or server
It executes.In situations involving remote computers, remote computer can include local area network by the network-of any kind
(LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize internet
Service provider is connected by internet).In some embodiments, by being believed using the state of computer-readable program instructions
Breath comes personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or programmable logic
Array (PLA), the electronic circuit can execute computer-readable program instructions, to realize the various aspects of present disclosure.
Example IV
According to the one aspect of one or more other embodiments of the present disclosure, a kind of text emotion sorter is provided.
A kind of text emotion sorter, based on a kind of text sentiment classification method, comprising:
Data acquisition module is configured as receiving text data, constructs distributed term vector, obtains eigenmatrix；
Convolutional neural networks module is configured as eigenmatrix inputting convolutional neural networks model, passes through the convolution
The convolutional layer of the multi-slide-windows mouth of neural network model carries out convolution algorithm to eigenmatrix；Pass through the convolutional neural networks mould
Double pond layers carry out pondization operation to the k-max and avg-pooling of type parallel；Pass through the concatenation of the convolutional neural networks model
The global characteristics vector of layer splicing building text level, uses Naive Bayes Classifier to obtain text emotion point to after its dimensionality reduction
Class result；
Parameter optimization module is configured to optimize the convolutional neural networks model ginseng using improved gradient descent algorithm
Number.
It should be noted that although being referred to several modules or submodule of equipment in the detailed description above, it is this
Division is only exemplary rather than enforceable.In fact, in accordance with an embodiment of the present disclosure, two or more above-described moulds
The feature and function of block can embody in a module.Conversely, the feature and function of an above-described module can be with
Further division is to be embodied by multiple modules.
The disclosure the utility model has the advantages that
(1) a kind of text sentiment classification method and device that the disclosure provides, mainly for network short text information on line
The random convolution algorithm layer that more size multi-slide-windows mouths are devised with the characteristics of semantic profound level of creation, it is hidden that depth excavates its
Containing information；The problem of expense is big, and training pattern can not support huge data volume is constructed to term vector, disclosure combination PCA is utilized
The thought of dimensionality reduction carries out data normalization to high-dimensional data, then acquires the covariance matrix and its correspondence of objective matrix
Feature vector, initial data are transformed into a kind of expression of any dimension linear independence finally by linear transformation, thus
Multi objective (high-dimensional) is converted into a few main characteristic component, to reduce expense；
(2) a kind of text sentiment classification method and device that the disclosure provides, for current convolutional neural networks text point
Class model can not effectively solve the problems, such as that pond layer dimensionality reduction loses semantic information, merge the pond k-max method Dynamic Extraction
Speciality and the pond avg-pooling method propose in conjunction with k-max and avg- the average semantic contribution ability of short text
Parallel double pond layer structures of pooling method carry out pondization operation, retain deeper as far as possible while reducing expense
Secondary semantic information；
(3) a kind of text sentiment classification method and device that the disclosure provides, assume only with Naive Bayes Classifier
Vertical, less sensitive to missing data, also fairly simple feature replaces softmax classifier to improve the standard of classification to algorithm
True rate；
(4) a kind of text sentiment classification method and device, improved gradient descent algorithm that the disclosure provides solve BGD
When number of samples is very big, each iteration requires to calculate all samples algorithm, and training time expense is big；SGD algorithm is frequent
Falling into locally optimal solution restrain model can not；And the experience value etc. that Mini-BGD extremely relies on batch_size is asked
Topic；The disclosure, which mentions improved gradient descent algorithm, can guarantee the stability of model, improve training speed, shorten model convergence
Time.
The foregoing is merely preferred embodiment of the present disclosure, are not limited to the disclosure, for the skill of this field
For art personnel, the disclosure can have various modifications and variations.It is all within the spirit and principle of the disclosure, it is made any to repair
Change, equivalent replacement, improvement etc., should be included within the protection scope of the disclosure.Therefore, the disclosure is not intended to be limited to this
These embodiments shown in text, and it is to fit to the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. a kind of text sentiment classification method, which is characterized in that this method comprises:
Text data is received, distributed term vector is constructed, obtains eigenmatrix；
Eigenmatrix is inputted into convolutional neural networks model, passes through the convolution of the multi-slide-windows mouth of the convolutional neural networks model
Layer carries out convolution algorithm to eigenmatrix；
By the k-max and avg-pooling of the convolutional neural networks model, double pond layers carry out pondization operation parallel；
Through the global characteristics vector of the concatenation layer splicing building text level of the convolutional neural networks model, after its dimensionality reduction
Text emotion classification results are obtained with Naive Bayes Classifier；
The convolutional neural networks model parameter is optimized using improved gradient descent algorithm.
2. a kind of text sentiment classification method as described in claim 1, which is characterized in that in the method, to received text
Notebook data carry out data prediction, by the text data after data prediction by word2vec tools build distribution word to
Amount；
The data prediction includes data cleansing and participle.
3. a kind of text sentiment classification method as described in claim 1, which is characterized in that in the method, the convolutional layer
Including the different convolution kernel of multiple window sizes, pass through the different convolution of multiple window sizes of the convolutional neural networks model
It checks the eigenmatrix and carries out convolution algorithm, extract the text part semantic vector of different windows size；
Each convolutional layer includes the convolution unit of several concurrent operations.
4. a kind of text sentiment classification method as claimed in claim 3, which is characterized in that in the method, the convolution mind
Further include filter unit through network model, passes through by the text part semantic vector that the convolutional layer convolution algorithm obtains described
Filter unit completes feature extraction.
5. a kind of text sentiment classification method as described in claim 1, which is characterized in that in the method, the k-max and
Double pond layers include the parallel pond avg-pooling layer and the pond k-max layer to avg-pooling parallel；The pond k-max
Layer determines that characteristic pattern down-sampling number, the convolution kernel height are inversely proportional with characteristic pattern down-sampling number according to convolution kernel height.
6. a kind of text sentiment classification method as described in claim 1, which is characterized in that in the method, using PCA to complete
Office's feature vector carries out dimensionality reduction.
7. a kind of text sentiment classification method as described in claim 1, which is characterized in that in the method, described use changes
Into gradient descent algorithm optimize the convolutional neural networks model parameter be by choose the higher sample of data dependence come
Form the batch data training set of convolutional neural networks model.
8. a kind of computer readable storage medium, wherein being stored with a plurality of instruction, which is characterized in that described instruction is suitable for by terminal
The processor of equipment is loaded and is executed such as a kind of described in any item text sentiment classification methods of claim 1-7.
9. a kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；It calculates
Machine readable storage medium storing program for executing is for storing a plurality of instruction, which is characterized in that described instruction is suitable for being loaded by processor and being executed such as power
Benefit requires a kind of described in any item text sentiment classification methods of 1-7.
10. a kind of text emotion sorter, which is characterized in that based on such as a kind of described in any item texts of claim 1-7
Sensibility classification method, comprising:
Data acquisition module is configured as receiving text data, constructs distributed term vector, obtains eigenmatrix；
Convolutional neural networks module is configured as eigenmatrix inputting convolutional neural networks model, passes through the convolutional Neural
The convolutional layer of the multi-slide-windows mouth of network model carries out convolution algorithm to eigenmatrix；Pass through the convolutional neural networks model
Double pond layers carry out pondization operation to k-max and avg-pooling parallel；It is spelled by the concatenation layer of the convolutional neural networks model
The global characteristics vector for connecing building text level is tied to using Naive Bayes Classifier to obtain text emotion classification after its dimensionality reduction
Fruit；
Parameter optimization module is configured to optimize the convolutional neural networks model parameter using improved gradient descent algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910375929.5A CN110059191A (en) | 2019-05-07 | 2019-05-07 | A kind of text sentiment classification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910375929.5A CN110059191A (en) | 2019-05-07 | 2019-05-07 | A kind of text sentiment classification method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110059191A true CN110059191A (en) | 2019-07-26 |
Family
ID=67322515
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910375929.5A Pending CN110059191A (en) | 2019-05-07 | 2019-05-07 | A kind of text sentiment classification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110059191A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705268A (en) * | 2019-09-02 | 2020-01-17 | 平安科技（深圳）有限公司 | Article subject extraction method and device based on artificial intelligence and computer-readable storage medium |
| CN110766439A (en) * | 2019-08-30 | 2020-02-07 | 昆山市量子昆慈量子科技有限责任公司 | Hotel network public praise evaluation method and system and electronic equipment |
| CN111178507A (en) * | 2019-12-26 | 2020-05-19 | 集奥聚合(北京)人工智能科技有限公司 | Atlas convolution neural network data processing method and device |
| CN112085837A (en) * | 2020-09-10 | 2020-12-15 | 哈尔滨理工大学 | Three-dimensional model classification method based on geometric shape and LSTM neural network |
| CN112988921A (en) * | 2019-12-13 | 2021-06-18 | 北京四维图新科技股份有限公司 | Method and device for identifying map information change |
| CN113627176A (en) * | 2021-08-17 | 2021-11-09 | 北京计算机技术及应用研究所 | Method for calculating Chinese word vector by using principal component analysis |
| CN113656560B (en) * | 2021-10-19 | 2022-02-22 | 腾讯科技（深圳）有限公司 | Emotion category prediction method and device, storage medium and electronic equipment |
| CN114664290A (en) * | 2022-05-17 | 2022-06-24 | 深圳比特微电子科技有限公司 | Sound event detection method and device and readable storage medium |
| WO2023015631A1 (en) * | 2021-08-13 | 2023-02-16 | 广东技术师范大学 | Missing data-based classification model generation method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107341145A (en) * | 2017-06-21 | 2017-11-10 | 华中科技大学 | A kind of user feeling analysis method based on deep learning |
| CN108038107A (en) * | 2017-12-22 | 2018-05-15 | 东软集团股份有限公司 | Sentence sensibility classification method, device and its equipment based on convolutional neural networks |
| US20180268298A1 (en) * | 2017-03-15 | 2018-09-20 | Salesforce.Com, Inc. | Deep Neural Network-Based Decision Network |
-
2019
- 2019-05-07 CN CN201910375929.5A patent/CN110059191A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180268298A1 (en) * | 2017-03-15 | 2018-09-20 | Salesforce.Com, Inc. | Deep Neural Network-Based Decision Network |
| CN107341145A (en) * | 2017-06-21 | 2017-11-10 | 华中科技大学 | A kind of user feeling analysis method based on deep learning |
| CN108038107A (en) * | 2017-12-22 | 2018-05-15 | 东软集团股份有限公司 | Sentence sensibility classification method, device and its equipment based on convolutional neural networks |
Non-Patent Citations (2)
| Title |
|---|
| 周锦峰: "卷积神经网络在短文本情感多分类标注应用", 《计算机工程与应用》 * |
| 邱宁佳: "结合改进主动学习的SVD-CNN弹幕文本分类算法", 《计算机应用》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110766439A (en) * | 2019-08-30 | 2020-02-07 | 昆山市量子昆慈量子科技有限责任公司 | Hotel network public praise evaluation method and system and electronic equipment |
| CN110705268A (en) * | 2019-09-02 | 2020-01-17 | 平安科技（深圳）有限公司 | Article subject extraction method and device based on artificial intelligence and computer-readable storage medium |
| CN112988921A (en) * | 2019-12-13 | 2021-06-18 | 北京四维图新科技股份有限公司 | Method and device for identifying map information change |
| CN111178507A (en) * | 2019-12-26 | 2020-05-19 | 集奥聚合(北京)人工智能科技有限公司 | Atlas convolution neural network data processing method and device |
| CN112085837A (en) * | 2020-09-10 | 2020-12-15 | 哈尔滨理工大学 | Three-dimensional model classification method based on geometric shape and LSTM neural network |
| CN112085837B (en) * | 2020-09-10 | 2022-04-26 | 哈尔滨理工大学 | Three-dimensional model classification method based on geometric shape and LSTM neural network |
| WO2023015631A1 (en) * | 2021-08-13 | 2023-02-16 | 广东技术师范大学 | Missing data-based classification model generation method |
| CN113627176A (en) * | 2021-08-17 | 2021-11-09 | 北京计算机技术及应用研究所 | Method for calculating Chinese word vector by using principal component analysis |
| CN113627176B (en) * | 2021-08-17 | 2024-04-19 | 北京计算机技术及应用研究所 | Method for calculating Chinese word vector by principal component analysis |
| CN113656560B (en) * | 2021-10-19 | 2022-02-22 | 腾讯科技（深圳）有限公司 | Emotion category prediction method and device, storage medium and electronic equipment |
| CN114664290A (en) * | 2022-05-17 | 2022-06-24 | 深圳比特微电子科技有限公司 | Sound event detection method and device and readable storage medium |
| CN114664290B (en) * | 2022-05-17 | 2022-08-19 | 深圳比特微电子科技有限公司 | Sound event detection method and device and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110059191A (en) | A kind of text sentiment classification method and device | |
| CN109558487A (en) | Document Classification Method based on the more attention networks of hierarchy | |
| CN110134771A (en) | A kind of implementation method based on more attention mechanism converged network question answering systems | |
| CN109241255A (en) | A kind of intension recognizing method based on deep learning | |
| CN109885670A (en) | A kind of interaction attention coding sentiment analysis method towards topic text | |
| CN107818164A (en) | A kind of intelligent answer method and its system | |
| CN109284506A (en) | A kind of user comment sentiment analysis system and method based on attention convolutional neural networks | |
| CN109325112B (en) | A kind of across language sentiment analysis method and apparatus based on emoji | |
| CN109460737A (en) | A kind of multi-modal speech-emotion recognition method based on enhanced residual error neural network | |
| CN109492099A (en) | It is a kind of based on field to the cross-domain texts sensibility classification method of anti-adaptive | |
| CN108763326A (en) | A kind of sentiment analysis model building method of the diversified convolutional neural networks of feature based | |
| CN109948158A (en) | Emotional orientation analytical method based on environment member insertion and deep learning | |
| CN112990296B (en) | Image-text matching model compression and acceleration method and system based on orthogonal similarity distillation | |
| CN111858932A (en) | Multiple-feature Chinese and English emotion classification method and system based on Transformer | |
| CN108197294A (en) | A kind of text automatic generation method based on deep learning | |
| CN107688576B (en) | Construction and tendency classification method of CNN-SVM model | |
| CN107092642A (en) | A kind of information search method, equipment, client device and server | |
| CN107766320A (en) | A kind of Chinese pronoun resolution method for establishing model and device | |
| CN109948149A (en) | A kind of file classification method and device | |
| CN110825850B (en) | Natural language theme classification method and device | |
| CN110472244B (en) | Short text sentiment classification method based on Tree-LSTM and sentiment information | |
| CN112307153A (en) | Automatic construction method and device of industrial knowledge base and storage medium | |
| CN106980620A (en) | A kind of method and device matched to Chinese character string | |
| CN109101490A (en) | The fact that one kind is based on the fusion feature expression implicit emotion identification method of type and system | |
| CN108875034A (en) | A kind of Chinese Text Categorization based on stratification shot and long term memory network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190726 |
|
| RJ01 | Rejection of invention patent application after publication |