CN109213868A - Entity level sensibility classification method based on convolution attention mechanism network - Google Patents
Entity level sensibility classification method based on convolution attention mechanism network Download PDFInfo
- Publication number
- CN109213868A CN109213868A CN201811394014.0A CN201811394014A CN109213868A CN 109213868 A CN109213868 A CN 109213868A CN 201811394014 A CN201811394014 A CN 201811394014A CN 109213868 A CN109213868 A CN 109213868A
- Authority
- CN
- China
- Prior art keywords
- text
- vector
- convolution
- matrix
- attention mechanism
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000007246 mechanism Effects 0.000 title claims abstract description 50
- 239000013598 vector Substances 0.000 claims abstract description 98
- 239000011159 matrix material Substances 0.000 claims abstract description 64
- 230000008451 emotion Effects 0.000 claims abstract description 24
- 230000002996 emotional effect Effects 0.000 claims abstract description 20
- 230000004913 activation Effects 0.000 claims abstract description 13
- 230000009466 transformation Effects 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 6
- 238000012549 training Methods 0.000 claims description 13
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 description 16
- 238000013528 artificial neural network Methods 0.000 description 8
- 230000000306 recurrent effect Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 239000012141 concentrate Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005498 polishing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention proposes a kind of entity level sensibility classification method based on convolution attention mechanism network, and the method obtains text matrix and target entity vector by carrying out processing to target text；Text matrix and target entity vector are subjected to pretreatment and obtain Text eigenvector；Text eigenvector and text matrix are subjected to pretreatment and obtain new Text eigenvector；M step S30 is repeated, M Text eigenvector is obtained；Linear transformation will be carried out after the splicing of all Text eigenvectors, inputs activation primitive, obtain the probability that text belongs to each emotion classification.Word around word is also accounted for range when obtaining attention weight by method of the invention, so that the text representation finally obtained be made to have stronger emotional expression ability.Finally realize more accurate entity level emotional semantic classification task.
Description
Technical field
The present invention relates to natural language analysis technical field more particularly to a kind of realities based on convolution attention mechanism network
Body rank sensibility classification method.
Background technique
In recent years, universal with Internet technology rapid development and shopping at network, there is a large amount of right on internet
In the comment of commodity.Using these comments, the Sentiment orientation of these texts is judged, thus to customer for the need of product
It asks and analyze and then improve product, be research hotspot very concerned in recent years.And the sentiment analysis technology of entity level, then
More refinement ground obtains commentator to the Sentiment orientation of a certain specific object of commodity.The previous emotion based on artificial design features
Classification method is gradually unable to satisfy the demand used, and method neural network based has become the master of present sentiment analysis
Flow direction.Sensibility classification method neural network based has been used for the applied fields such as the analysis of public opinion, commodity user feedback analysis
Jing Zhong.
The sensibility classification method of existing entity level is based primarily upon Recognition with Recurrent Neural Network (RNN) and attention mechanism.
Recognition with Recurrent Neural Network is commonly used for carrying out the amount with sequential relationship due to having the ability that can retain past information
It indicates.And also remain fixed order just in the text, between word and word, so Recognition with Recurrent Neural Network is just suitble to use
In indicating the context relation between this word and word, so as to obtain the vector expression for including text semantic content.
The main function of attention mechanism is to assign a weight to each of sentence word, this weight represents this
Significance level of a word in this sentence, so that contributing vector corresponding to bigger word can the semantic of sentence
Obtain bigger weight.In emotional semantic classification, by using attention mechanism can by some words for being used to indicate emotion with
And degree adverb assigns bigger weight, so that obtaining vector relevant to the emotion of sentence indicates.And the emotion of entity level
Classification is the emotional semantic classification task more refined, it needs to obtain passage for the emotional category of a certain given entity, and
It can include multiple entities, and this section of text is also different for the expressed emotion out of different entities in one section of text.This
When, the effect of attention mechanism is other than the vocabulary for emphasizing to show emotion, it is also necessary to which distinguishing the word that these show emotion is
No is to be directed to target entity, and only assign the weight bigger for the emotion vocabulary of target word, and assign and be directed to other realities
The emotion vocabulary of body assigns smaller weight.Emotional category of the text vector finally obtained in this way aiming at target entity
's.
But this based on the method for Recognition with Recurrent Neural Network and attention mechanism tool, there are two problems.One problem is, by
Have the characteristics that retain past information in Recognition with Recurrent Neural Network, it must successively be calculated when calculating according to order, and be protected
Stay past all information.And such characteristic, so that Recognition with Recurrent Neural Network can not carry out parallel computation when running on GPU,
So that the efficiency calculated can not improve.Second Problem is under normal circumstances, to calculate a word using attention mechanism
When the weight of language, the correlation degree between this word and target entity is only considered.But sometimes one word is for entire sentence
Semanteme contribution is not only related with itself, also related with the word around this word because the degree adverb of one word of modification or
Negative word can significantly change effect of this word in sentence, but current attention mechanism does not consider this
Kind situation.
In view of the foregoing, it is necessary to propose a kind of entity level emotional semantic classification side based on convolution attention mechanism network
The method of method.
Summary of the invention
In order to solve the above-mentioned technical problem, in order to which the technology for solving how to improve entity level emotional semantic classification accuracy is asked
Topic, the invention proposes a kind of entity level sensibility classification method based on convolution attention mechanism network, the method is also wrapped
It includes:
Step S10 carries out processing to target text and obtains text matrix and target entity vector；
Text matrix and target entity vector are carried out pretreatment and obtain Text eigenvector by step S20；
Text eigenvector and text matrix are carried out pretreatment and obtain new Text eigenvector by step S30；
Step S30 is repeated M times, is often repeated once Text eigenvector obtained by as input next time, most
M Text eigenvector is obtained eventually；
Step S40 will carry out linear transformation after the splicing of all Text eigenvectors, input activation primitive, obtain text category
In the probability of each emotion classification；
Step S50 is optimized and then right using error function of the back-propagation algorithm to convolution attention mechanism network
All parameters are updated in network.
Preferably, the pretreatment of the step S20 includes:
Text matrix obtained and target entity vector are inputted convolution attention mechanism unit, obtained by step S21
The attention weight of each word in text；
The term vector that text matrix is included is weighted summation using attention weight, it is special to obtain text by step S22
Levy vector.
Preferably, the term vector that text matrix is included is weighted using attention weight in the step S22 and is asked
With specifically:
Step S221, by term vector corresponding to word each in text multiplied by corresponding attention weight；
Step S222 sums the vector set that step S211 is obtained, and obtains Text eigenvector.
Preferably, the pretreatment of the step S30 includes:
The feature vector of text obtained is inputted convolution attention mechanism unit by step S31；
Text matrix is inputted convolution attention mechanism unit by step S32；
Step S33 repeats the operation for obtaining attention weight and weighted sum operation, obtain new text feature to
Amount.
Preferably, the step S10 includes:
Step S11 carries out participle to target text using open source segmentation methods and obtains orderly set of words；
Step S12, using a large amount of text pre-training term vectors obtained from internet, for each word in text with
And target entity, word is indicated using the vector row that the dimension that pre-training obtains is D；
The term vector of each word in text is arranged and is combined by word order by step S13, obtains text matrix.
Preferably, the convolution attention mechanism in the method includes: known keyword vector and content matrix；
Crucial term vector and term vector each in content matrix are spliced, " keyword-content " matrix is obtained；
Convolution operation is carried out to " keyword-entity " matrix using the convolution kernel of several specified sizes, obtains convolution feature
Matrix；
Convolution eigenmatrix is added with bias vector；
Use hyperbolic tangent function as the activation primitive of convolution operation, formula indicates are as follows:
Wherein x indicates the value of each element in eigenmatrix；
For feature vector corresponding to each word in convolution eigenmatrix, carries out maximizing pondization operation, obtain
Obtain attention weight feature；
Gained attention weight feature vector is normalized using softmax function；Wherein, softmax function
Formula indicates are as follows:
Wherein xjIndicate that j-th of element in feature vector, x indicate current element value.
Preferably, the maximization pondization operation specifically includes:
Only the maximum value in keeping characteristics vector all elements is as attention weight characteristic value corresponding to word；
Obtain dimension and the consistent attention weight feature vector of textual words number.
Preferably, the step S40 includes:
Step S41 splices resulting Text eigenvector；
Vector after splicing is multiplied by step S42 with linear transformation weight matrix, and adds bias vector；
Feature vector after linear transformation is sent into nonlinear activation function by step S43, is obtained text and is belonged to each emotion class
Other probability.
Preferably, the error function in the step S50 includes:
Intersection entropy loss letter between the emotion probability distribution and training sample emotion probability distribution of acquired probability vector
The sum of several and regular terms of network parameter.
Preferably, the cross entropy loss function are as follows:
Wherein n indicates the number of training sample, and x and y respectively indicate the obtained emotion probability distribution of network and training
The practical emotion probability distribution of sample.
Preferably, the regular terms of the network parameter includes:
The regular terms of network parameter refers to the sum of 2 norms of all weight matrix and bias vector being mentioned in network.
The present invention is based on the entity level sensibility classification methods of convolution attention mechanism network, by carrying out to target text
Processing obtains text matrix and target entity vector；Text matrix and target entity vector are subjected to pretreatment and obtain text
Feature vector；Text eigenvector and text matrix are subjected to pretreatment and obtain new Text eigenvector；Repeat M step
S30 obtains M Text eigenvector；Linear transformation will be carried out after the splicing of all Text eigenvectors, inputs activation primitive, obtain
Obtain the probability that text belongs to each emotion classification.Method of the invention also receives the word around word when obtaining attention weight
Enter limit of consideration, to make the text representation finally obtained that there is stronger emotional expression ability.It finally realizes more accurate
Entity level emotional semantic classification task.
Detailed description of the invention
Attached drawing is as a part of the invention, and for providing further understanding of the invention, of the invention is schematic
Examples and descriptions thereof are used to explain the present invention, but does not constitute an undue limitation on the present invention.Obviously, the accompanying drawings in the following description
Only some embodiments to those skilled in the art without creative efforts, can be with
Other accompanying drawings can also be obtained according to these attached drawings.In the accompanying drawings:
Fig. 1 is the entity level sensibility classification method stream based on convolution attention mechanism network of an embodiment of the present invention
Journey schematic diagram；
Fig. 2 is to show the process that text matrix and target entity vector are located in advance in a kind of embodiment of the method for the present invention
It is intended to；
Fig. 3 in a kind of embodiment of the method for the present invention using the term vector that text matrix is included by attention weight by being carried out
The flow diagram of weighted sum；
Fig. 4 is that Text eigenvector and text matrix are carried out pretreated process in a kind of embodiment of the method for the present invention
Schematic diagram；
Fig. 5 is to carry out processing to target text in a kind of embodiment of the method for the present invention to obtain text matrix and target entity
The flow diagram of vector.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
The invention proposes a kind of entity level sensibility classification methods based on convolution attention mechanism network, it is intended to solve
The technical issues of how improving entity level emotional semantic classification accuracy.
In an embodiment of the present invention, referring to Fig.1~Fig. 5, described method includes following steps:
Step S10 carries out processing to target text and obtains text matrix and target entity vector；
Specifically, the step S10 is as shown in Figure 5, comprising:
Step S11 carries out participle to target text using open source segmentation methods and obtains orderly set of words；
Wherein, the embodiment of the present invention can segment Chinese text using stammerer participle library to text, and for
English text since English is inherently made of word, therefore does not need to be segmented.
As an example, for text, " screen of this mobile phone is very big, but battery durable ability is not very strong.", participle
Result afterwards be " this portion mobile phone screen very it is big, but battery continuation of the journey ability be not very it is strong."
Step S12, using a large amount of text pre-training term vectors obtained from internet, for each word in text with
And target entity, word is indicated using the vector row that the dimension that pre-training obtains is D；
In above-mentioned steps, term vector training is carried out using Glove algorithm, term vector obtained has the expression semanteme of word
Ability.
The term vector of each word in text is arranged and is combined by word order by step S13, obtains text matrix.
Specifically, for the text comprising n word, the term vector dimension of each word is D, by word be expressed as word to
After amount, the ordered set for the term vector that n dimension is D is obtained, the text matrix having a size of n × D is obtained after combining according to order,
Entity is then expressed as the vector that dimension is D.
Text matrix and target entity vector are carried out pretreatment and obtain Text eigenvector by step S20；
In this step, it is pre-processed by text matrix and target entity vector to obtain Text eigenvector.
Specific steps are as shown in Figure 2, comprising:
Text matrix obtained and target entity vector are inputted convolution attention mechanism unit, obtained by step S21
The attention weight of each word in text；
Wherein, by inputting text matrix and entity vector matrix in convolution attention mechanism unit, to obtain text
Attention weight of each word includes step S21a to S21f in this.
Step S21a: target entity vector is regarded as the input of convolution attention mechanism unit key term vector, by text square
Battle array is regarded as content matrix.
Step S21b: crucial term vector and term vector each in content matrix are spliced, and are obtained " keyword-content "
Matrix.
Specifically, for the crucial term vector of content matrix and dimension for D having a size of n × D, spliced " keyword-
The size of entity " matrix is n × 2D.
Step S21c: convolution operation is carried out to " keyword-entity " matrix using the convolution kernel of several specified sizes, is obtained
Convolution eigenmatrix.
Specifically, for " keyword-entity " matrix having a size of n × 2D, the k convolution kernels having a size of w × 2D are used
Convolution operation is carried out to it, in order to keep the first dimension size constancy of convolution operation front and back matrix, is needed before operation to " crucial
Word-entity " matrix carries out polishing, i.e., ties up full null matrix of the head and the tail splicing having a size of (w-1)/2 × 2D in matrix first.Finally obtain
Obtain the convolution eigenmatrix having a size of n × k.
Step S21d: convolution eigenmatrix is added with bias vector.
Step S21e: by convolution eigenmatrix unbalanced input activation primitive.
In above-mentioned steps, activation primitive using hyperbolic tangent function as convolution operation, formula is indicated are as follows:
Step S21f: it for k dimensional feature corresponding to each word in convolution eigenmatrix, carries out maximizing pond
Operation obtains attention weight feature vector.
Wherein, maximum pondization operation only retains the maximum value in k dimensional feature as the spy of attention weight corresponding to word
Value indicative obtains dimension and the consistent attention weight feature vector of textual words number.
Step S21h: gained attention weight feature vector is normalized using softmax function.
Specifically, for certain one-dimensional element x in weight feature vectori, normalization after value obtained by following formula
:
It is attention weight corresponding to i-th of word in text by the value that above formula obtains.
The term vector that text matrix is included is weighted summation using attention weight, it is special to obtain text by step S22
Levy vector.
The term vector that text matrix is included is weighted summation, such as Fig. 3 using attention weight in the step S22
It is shown, specifically:
Step S221, by term vector corresponding to word each in text multiplied by corresponding attention weight；
Step S222 sums the vector set that step S221 is obtained, and obtains Text eigenvector.
Text eigenvector and text matrix are carried out pretreatment and obtain new Text eigenvector by step S30.Such as figure
Shown in 4, specific steps include:
The feature vector of text obtained is inputted convolution attention mechanism unit by step S31；
Text matrix is inputted convolution attention mechanism unit by step S32；
Step S33 repeats the operation for obtaining attention weight and weighted sum operation, obtain new text feature to
Amount.
Step S30 is repeated M times, is often repeated once Text eigenvector obtained by as input next time, most
M Text eigenvector is obtained eventually；
Step S30 calculates attention power with by target entity vector and text Input matrix convolution attention mechanism unit
Weight, and obtain that method used by Text eigenvector is identical, and details are not described herein.
It should be noted that step S30 is M times repeatable, a new Text eigenvector will be obtained by being often repeated once.
It is further known that, it is assumed that step S30 is repeated M times, including previous step Text eigenvector obtained, and M+1 can be obtained altogether
Text eigenvector.
Step S40 will carry out linear transformation after the splicing of all Text eigenvectors, input activation primitive, obtain text category
In the probability of each emotion classification；
The step S40 includes:
Step S41 splices resulting Text eigenvector；
Specifically, if one is obtained M+1 Text eigenvector, the dimension of each feature vector is D, vector after splicing
Dimension is then (M+1) × D.
Vector after splicing is multiplied by step S42 with linear transformation weight matrix, and adds bias vector；
Feature vector after linear transformation is sent into nonlinear activation function by step S43, is obtained text and is belonged to each emotion class
Other probability.
Wherein, consistent in function formula and step S21h using softmax function as nonlinear activation function, herein
It repeats no more.
When above-mentioned technical proposal carries out the emotional semantic classification of entity level, convolution operation has been used to realize attention mechanism,
So that above-mentioned algorithm is able to carry out parallel computation when running on GPU, computational efficiency is greatly enhanced.In addition, obtaining
The word around word is also accounted for into range when obtaining attention weight, so that it is stronger to have the text representation finally obtained
Emotional expression ability.Finally realize more accurate entity level emotional semantic classification task.
In an alternative embodiment of the invention, after the step S40 further include:
Step S50 is optimized and then right using error function of the back-propagation algorithm to convolution attention mechanism network
All parameters are updated in network.
Wherein, the embodiment of the present invention can optimize optimization aim using stochastic gradient descent algorithm.
The present invention further provides a kind of entity level emotional semantic classification system based on convolution attention mechanism network, referring to
Figure, the entity level emotional semantic classification system based on convolution attention mechanism network include: storage 101, processor 102 and deposit
Store up the entity grade based on convolution attention mechanism network that can be run on the memory 101 and on the processor 102
Sorrow of separation sense sort program, the entity level emotional semantic classification program based on convolution attention mechanism network is by the processor
102 realize method as described above when executing.
In addition, the embodiment of the present invention also proposes a kind of computer readable storage medium, the computer readable storage medium
On be stored with the entity level emotional semantic classification program based on convolution attention mechanism network, it is described to be based on convolution attention mechanism net
The entity level emotional semantic classification program of network realizes method as described above when being executed by processor.
Although each step is described in the way of above-mentioned precedence in above-described embodiment, this field
Technical staff is appreciated that the effect in order to realize the present embodiment, executes between different steps not necessarily in such order,
It (parallel) execution simultaneously or can be executed with reverse order, these simple variations all protection scope of the present invention it
It is interior.
Technical solution is provided for the embodiments of the invention above to be described in detail.Although applying herein specific
A example the principle of the present invention and embodiment are expounded, still, the explanation of above-described embodiment be only applicable to help manage
Solve the principle of the embodiment of the present invention；Meanwhile to those skilled in the art, according to an embodiment of the present invention, it is being embodied
It can be made a change within mode and application range.
It should be noted that the flowchart or block diagram being referred to herein is not limited solely to form shown in this article,
It can also be divided and/or be combined.
It should be noted that: label and text in attached drawing are intended merely to be illustrated more clearly that the present invention, are not intended as pair
The improper restriction of the scope of the present invention.
Again it should be noted that description and claims of this specification and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, rather than be used to describe a particular order or precedence order.It should be understood that using in this way
Data can be interchanged in appropriate circumstances, so that the embodiment of the present invention described herein can be in addition to illustrating herein
Or the sequence other than those of description is implemented.
Term " includes " or any other like term are intended to cover non-exclusive inclusion, so that including a system
Process, method, article or equipment/device of column element not only includes those elements, but also including being not explicitly listed
Other elements, or further include the intrinsic element of these process, method, article or equipment/devices.
As used herein, term " module " may refer to the software object executed on a computing system or routine.
Disparate modules described herein can be embodied as to the object executed on a computing system or process (for example, as independence
Thread).While it is preferred that realize system and method described herein with software, but with hardware or software and hard
The realization of the combination of part is also possible and can be conceived to.
Each step of the invention can be realized with general computing device, for example, they can concentrate on it is single
On computing device, such as: personal computer, server computer, handheld device or portable device, laptop device or more
Processor device can also be distributed over a network of multiple computing devices, they can be to be different from sequence herein
Shown or described step is executed, perhaps they are fabricated to each integrated circuit modules or will be more in them
A module or step are fabricated to single integrated circuit module to realize.Therefore, the present invention is not limited to any specific hardware and soft
Part or its combination.
Programmable logic device can be used to realize in method provided by the invention, and it is soft also to may be embodied as computer program
Part or program module (it include routines performing specific tasks or implementing specific abstract data types, programs, objects, component or
Data structure etc.), such as embodiment according to the present invention can be a kind of computer program product, run the computer program
Product executes computer for demonstrated method.The computer program product includes computer readable storage medium, should
It include computer program logic or code section on medium, for realizing the method.The computer readable storage medium can
To be the built-in medium being mounted in a computer or the removable medium (example that can be disassembled from basic computer
Such as: using the storage equipment of hot plug technology).The built-in medium includes but is not limited to rewritable nonvolatile memory,
Such as: RAM, ROM, flash memory and hard disk.The removable medium includes but is not limited to: and optical storage media (such as: CD-
ROM and DVD), magnetic-optical storage medium (such as: MO), magnetic storage medium (such as: tape or mobile hard disk), can with built-in
Rewrite the media (such as: storage card) of nonvolatile memory and the media (such as: ROM box) with built-in ROM.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (11)
1. a kind of entity level sensibility classification method based on convolution attention mechanism network, which is characterized in that the method packet
It includes:
Step S10 carries out processing to target text and obtains text matrix and target entity vector；
Text matrix and target entity vector are carried out pretreatment and obtain Text eigenvector by step S20；
Text eigenvector and text matrix are carried out pretreatment and obtain new Text eigenvector by step S30；
Step S30 is repeated M times, Text eigenvector obtained is often repeated once by as input next time, finally obtains
Obtain M Text eigenvector；
Step S40 will carry out linear transformation after the splicing of all Text eigenvectors, input activation primitive, obtain text and belong to respectively
The probability of a emotion classification.
2. the entity level sensibility classification method according to claim 1 based on convolution attention mechanism network, feature
It is, the pretreatment of the step S20 includes:
Text matrix obtained and target entity vector are inputted convolution attention mechanism unit, obtain text by step S21
In each word attention weight；
The term vector that text matrix is included is weighted summation using attention weight by step S22, obtain text feature to
Amount.
3. the entity level sensibility classification method according to claim 2 based on convolution attention mechanism network, feature
It is, the term vector that text matrix is included is weighted summation using attention weight in the step S22 specifically:
Step S221, by term vector corresponding to word each in text multiplied by corresponding attention weight；
Step S222 sums the vector set that step S221 is obtained, and obtains Text eigenvector.
4. the entity level sensibility classification method according to claim 1 based on convolution attention mechanism network, feature
It is, the pretreatment of the step S30 includes:
The feature vector of text obtained is inputted convolution attention mechanism unit by step S31；
Text matrix is inputted convolution attention mechanism unit by step S32；
Step S33 repeats the operation for obtaining attention weight and weighted sum operation, obtains new Text eigenvector.
5. the entity level sensibility classification method according to claim 1 based on convolution attention mechanism network, feature
It is, the step S10 includes:
Step S11 carries out participle to target text using open source segmentation methods and obtains orderly set of words；
Step S12, using a large amount of text pre-training term vectors obtained from internet, for each word and mesh in text
Entity is marked, indicates word using the vector row that the dimension that pre-training obtains is D；
The term vector of each word in text is arranged and is combined by word order by step S13, obtains text matrix.
6. the entity level sensibility classification method according to claim 1 based on convolution attention mechanism network, feature
It is, the convolution attention mechanism in the method includes: known keyword vector and content matrix；
Crucial term vector and term vector each in content matrix are spliced, " keyword-content " matrix is obtained；
Convolution operation is carried out to " keyword-entity " matrix using the convolution kernel of several specified sizes, obtains convolution eigenmatrix；
Convolution eigenmatrix is added with bias vector；
Use hyperbolic tangent function as the activation primitive of convolution operation, formula indicates are as follows:
Wherein x indicates the value of each element in eigenmatrix；
For feature vector corresponding to each word in convolution eigenmatrix, carries out maximizing pondization operation, be infused
Meaning power weight feature；
Gained attention weight feature vector is normalized using softmax function；Wherein, the formula of softmax function
It indicates are as follows:
Wherein xjIndicate that j-th of element in feature vector, x indicate current element value.
7. the entity level sensibility classification method according to claim 6 based on convolution attention mechanism network, feature
It is, the maximization pondization operation specifically includes:
Only the maximum value in keeping characteristics vector all elements is as attention weight characteristic value corresponding to word；
Obtain dimension and the consistent attention weight feature vector of textual words number.
8. the entity level sensibility classification method according to claim 1 based on convolution attention mechanism network, feature
It is, the step S40 includes:
Step S41 splices resulting Text eigenvector；
Vector after splicing is multiplied by step S42 with linear transformation weight matrix, and adds bias vector；
Feature vector after linear transformation is sent into nonlinear activation function by step S43, is obtained text and is belonged to each emotional category
Probability.
9. the entity level sensibility classification method according to claim 1 based on convolution attention mechanism network, feature
It is, the error function in the step S50 includes:
Cross entropy loss function between the emotion probability distribution and training sample emotion probability distribution of acquired probability vector with
And the sum of regular terms of network parameter.
10. the entity level sensibility classification method according to claim 8 based on convolution attention mechanism network, feature
It is, the cross entropy loss function are as follows:
Wherein n indicates the number of training sample, and x and y respectively indicate the obtained emotion probability distribution of network and training sample
Practical emotion probability distribution.
11. the entity level sensibility classification method according to claim 8 based on convolution attention mechanism network, feature
It is, the regular terms of the network parameter includes:
The regular terms of network parameter refers to the sum of 2 norms of all weight matrix and bias vector being mentioned in network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811394014.0A CN109213868A (en) | 2018-11-21 | 2018-11-21 | Entity level sensibility classification method based on convolution attention mechanism network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811394014.0A CN109213868A (en) | 2018-11-21 | 2018-11-21 | Entity level sensibility classification method based on convolution attention mechanism network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109213868A true CN109213868A (en) | 2019-01-15 |
Family
ID=64993568
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811394014.0A Pending CN109213868A (en) | 2018-11-21 | 2018-11-21 | Entity level sensibility classification method based on convolution attention mechanism network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109213868A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919175A (en) * | 2019-01-16 | 2019-06-21 | 浙江大学 | A kind of more classification methods of entity of combination attribute information |
| CN109948165A (en) * | 2019-04-24 | 2019-06-28 | 吉林大学 | Fine granularity feeling polarities prediction technique based on mixing attention network |
| CN110046223A (en) * | 2019-03-13 | 2019-07-23 | 重庆邮电大学 | Film review sentiment analysis method based on modified convolutional neural networks model |
| CN110263162A (en) * | 2019-06-05 | 2019-09-20 | 阿里巴巴集团控股有限公司 | Convolutional neural networks and its method of progress text classification, document sorting apparatus |
| CN110263122A (en) * | 2019-05-08 | 2019-09-20 | 北京奇艺世纪科技有限公司 | A kind of keyword acquisition methods, device and computer readable storage medium |
| CN110377744A (en) * | 2019-07-26 | 2019-10-25 | 北京香侬慧语科技有限责任公司 | A kind of method, apparatus, storage medium and the electronic equipment of public sentiment classification |
| CN110427568A (en) * | 2019-07-25 | 2019-11-08 | 成都品果科技有限公司 | A kind of collaboration attention recommendation system, method and apparatus based on information |
| CN110704619A (en) * | 2019-09-24 | 2020-01-17 | 支付宝(杭州)信息技术有限公司 | Text classification method and device and electronic equipment |
| CN111324739A (en) * | 2020-05-15 | 2020-06-23 | 支付宝(杭州)信息技术有限公司 | Text emotion analysis method and system |
| CN111611789A (en) * | 2019-02-25 | 2020-09-01 | 北京嘀嘀无限科技发展有限公司 | Statement representation method, representation model training method and device |
| CN111832328A (en) * | 2019-04-15 | 2020-10-27 | 北京京东尚科信息技术有限公司 | Bar code detection method, bar code detection device, electronic equipment and medium |
| WO2020224106A1 (en) * | 2019-05-07 | 2020-11-12 | 平安科技（深圳）有限公司 | Text classification method and system based on neural network, and computer device |
| CN113095070A (en) * | 2021-04-06 | 2021-07-09 | 山东省人工智能研究院 | Relation extraction method based on improved word level attention mechanism |
| CN113468876A (en) * | 2020-03-31 | 2021-10-01 | 阿里巴巴集团控股有限公司 | Text processing method, text processing device, electronic device, and computer-readable storage medium |
| CN113642572A (en) * | 2021-07-15 | 2021-11-12 | 上海交通大学 | Image target detection method, system and device based on multi-level attention |
| CN113742482A (en) * | 2021-07-19 | 2021-12-03 | 暨南大学 | Emotion classification method and medium based on multiple word feature fusion |
| CN113762381A (en) * | 2021-09-07 | 2021-12-07 | 上海明略人工智能(集团)有限公司 | Emotion classification method, system, electronic device and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599933A (en) * | 2016-12-26 | 2017-04-26 | 哈尔滨工业大学 | Text emotion classification method based on the joint deep learning model |
| CN107092596A (en) * | 2017-04-24 | 2017-08-25 | 重庆邮电大学 | Text emotion analysis method based on attention CNNs and CCR |
| CN107391483A (en) * | 2017-07-13 | 2017-11-24 | 武汉大学 | A kind of comment on commodity data sensibility classification method based on convolutional neural networks |
| US20180293499A1 (en) * | 2017-04-11 | 2018-10-11 | Sap Se | Unsupervised neural attention model for aspect extraction |
| CN108763326A (en) * | 2018-05-04 | 2018-11-06 | 南京邮电大学 | A kind of sentiment analysis model building method of the diversified convolutional neural networks of feature based |
-
2018
- 2018-11-21 CN CN201811394014.0A patent/CN109213868A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599933A (en) * | 2016-12-26 | 2017-04-26 | 哈尔滨工业大学 | Text emotion classification method based on the joint deep learning model |
| US20180293499A1 (en) * | 2017-04-11 | 2018-10-11 | Sap Se | Unsupervised neural attention model for aspect extraction |
| CN107092596A (en) * | 2017-04-24 | 2017-08-25 | 重庆邮电大学 | Text emotion analysis method based on attention CNNs and CCR |
| CN107391483A (en) * | 2017-07-13 | 2017-11-24 | 武汉大学 | A kind of comment on commodity data sensibility classification method based on convolutional neural networks |
| CN108763326A (en) * | 2018-05-04 | 2018-11-06 | 南京邮电大学 | A kind of sentiment analysis model building method of the diversified convolutional neural networks of feature based |
Non-Patent Citations (1)
| Title |
|---|
| QIAN YI 等: "Aspect-Level Sentiment Classification with Conv-Attention Mechanism", 《ICONIP 2018：NEURAL INFORMATION PROCESSING》 * |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919175A (en) * | 2019-01-16 | 2019-06-21 | 浙江大学 | A kind of more classification methods of entity of combination attribute information |
| CN111611789A (en) * | 2019-02-25 | 2020-09-01 | 北京嘀嘀无限科技发展有限公司 | Statement representation method, representation model training method and device |
| CN111611789B (en) * | 2019-02-25 | 2024-06-07 | 北京嘀嘀无限科技发展有限公司 | Sentence representation method, representation model training method and device |
| CN110046223A (en) * | 2019-03-13 | 2019-07-23 | 重庆邮电大学 | Film review sentiment analysis method based on modified convolutional neural networks model |
| CN110046223B (en) * | 2019-03-13 | 2021-05-18 | 重庆邮电大学 | Film evaluation emotion analysis method based on improved convolutional neural network model |
| CN111832328A (en) * | 2019-04-15 | 2020-10-27 | 北京京东尚科信息技术有限公司 | Bar code detection method, bar code detection device, electronic equipment and medium |
| CN109948165B (en) * | 2019-04-24 | 2023-04-25 | 吉林大学 | Fine granularity emotion polarity prediction method based on mixed attention network |
| CN109948165A (en) * | 2019-04-24 | 2019-06-28 | 吉林大学 | Fine granularity feeling polarities prediction technique based on mixing attention network |
| WO2020224106A1 (en) * | 2019-05-07 | 2020-11-12 | 平安科技（深圳）有限公司 | Text classification method and system based on neural network, and computer device |
| CN110263122A (en) * | 2019-05-08 | 2019-09-20 | 北京奇艺世纪科技有限公司 | A kind of keyword acquisition methods, device and computer readable storage medium |
| CN110263162B (en) * | 2019-06-05 | 2023-05-26 | 创新先进技术有限公司 | Convolutional neural network, text classification method thereof and text classification device |
| CN110263162A (en) * | 2019-06-05 | 2019-09-20 | 阿里巴巴集团控股有限公司 | Convolutional neural networks and its method of progress text classification, document sorting apparatus |
| CN110427568A (en) * | 2019-07-25 | 2019-11-08 | 成都品果科技有限公司 | A kind of collaboration attention recommendation system, method and apparatus based on information |
| CN110377744A (en) * | 2019-07-26 | 2019-10-25 | 北京香侬慧语科技有限责任公司 | A kind of method, apparatus, storage medium and the electronic equipment of public sentiment classification |
| CN110704619A (en) * | 2019-09-24 | 2020-01-17 | 支付宝(杭州)信息技术有限公司 | Text classification method and device and electronic equipment |
| CN113468876A (en) * | 2020-03-31 | 2021-10-01 | 阿里巴巴集团控股有限公司 | Text processing method, text processing device, electronic device, and computer-readable storage medium |
| CN111324739A (en) * | 2020-05-15 | 2020-06-23 | 支付宝(杭州)信息技术有限公司 | Text emotion analysis method and system |
| CN113095070A (en) * | 2021-04-06 | 2021-07-09 | 山东省人工智能研究院 | Relation extraction method based on improved word level attention mechanism |
| CN113642572B (en) * | 2021-07-15 | 2023-10-27 | 上海交通大学 | Image target detection method, system and device based on multi-level attention |
| CN113642572A (en) * | 2021-07-15 | 2021-11-12 | 上海交通大学 | Image target detection method, system and device based on multi-level attention |
| CN113742482A (en) * | 2021-07-19 | 2021-12-03 | 暨南大学 | Emotion classification method and medium based on multiple word feature fusion |
| CN113742482B (en) * | 2021-07-19 | 2024-05-31 | 暨南大学 | Emotion classification method and medium based on multiple word feature fusion |
| CN113762381A (en) * | 2021-09-07 | 2021-12-07 | 上海明略人工智能(集团)有限公司 | Emotion classification method, system, electronic device and medium |
| CN113762381B (en) * | 2021-09-07 | 2023-12-19 | 上海明略人工智能(集团)有限公司 | Emotion classification method, system, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109213868A (en) | Entity level sensibility classification method based on convolution attention mechanism network | |
| CN111177569B (en) | Recommendation processing method, device and equipment based on artificial intelligence | |
| CN116194912A (en) | Method and system for aspect-level emotion classification using graph diffusion transducers | |
| US9720907B2 (en) | System and method for learning latent representations for natural language tasks | |
| US20210042664A1 (en) | Model training and service recommendation | |
| CN110209805B (en) | Text classification method, apparatus, storage medium and computer device | |
| CN108446271B (en) | Text emotion analysis method of convolutional neural network based on Chinese character component characteristics | |
| CN108984530A (en) | A kind of detection method and detection system of network sensitive content | |
| CN110263821B (en) | Training of transaction feature generation model, and method and device for generating transaction features | |
| CN109446430A (en) | Method, apparatus, computer equipment and the readable storage medium storing program for executing of Products Show | |
| CN110619044B (en) | Emotion analysis method, system, storage medium and equipment | |
| CN109684627A (en) | A kind of file classification method and device | |
| CN109766557A (en) | A kind of sentiment analysis method, apparatus, storage medium and terminal device | |
| CN108959305A (en) | A kind of event extraction method and system based on internet big data | |
| CN111950279B (en) | Entity relationship processing method, device, equipment and computer readable storage medium | |
| CN109766437A (en) | A kind of Text Clustering Method, text cluster device and terminal device | |
| US11210673B2 (en) | Transaction feature generation | |
| CN110232123A (en) | The sentiment analysis method and device thereof of text calculate equipment and readable medium | |
| CN105809090A (en) | Method and system for face sex characteristic extraction | |
| Lian et al. | Unsupervised representation learning with future observation prediction for speech emotion recognition | |
| Liu et al. | Saliency as evidence: Event detection with trigger saliency attribution | |
| CN108009248A (en) | A kind of data classification method and system | |
| Peng et al. | Leaf disease image retrieval with object detection and deep metric learning | |
| CN109271624A (en) | A kind of target word determines method, apparatus and storage medium | |
| Gavval et al. | CUDA-Self-Organizing feature map based visual sentiment analysis of bank customer complaints for Analytical CRM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190115 |