CN109635109A - Sentence classification method based on LSTM and combination part of speech and more attention mechanism - Google Patents

Sentence classification method based on LSTM and combination part of speech and more attention mechanism Download PDF

Info

Publication number
CN109635109A
CN109635109A CN201811430542.7A CN201811430542A CN109635109A CN 109635109 A CN109635109 A CN 109635109A CN 201811430542 A CN201811430542 A CN 201811430542A CN 109635109 A CN109635109 A CN 109635109A
Authority
CN
China
Prior art keywords
speech
sentence
layer
attention
lstm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811430542.7A
Other languages
Chinese (zh)
Other versions
CN109635109B (en
Inventor
苏锦钿
周炀
朱展东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN201811430542.7A priority Critical patent/CN109635109B/en
Publication of CN109635109A publication Critical patent/CN109635109A/en
Application granted granted Critical
Publication of CN109635109B publication Critical patent/CN109635109B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds
    • G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/253—Grammatical analysis; Style critique
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Abstract

The invention discloses a kind of based on LSTM and combines the sentence classification method of part of speech and more attention mechanism, comprising steps of each sentence is converted to two based on continuous and dense semantic term vector matrix and part of speech term vector matrix in input layer；Learn word or the contextual information of part of speech in sentence respectively in shared two-way LSTM layers, and is exported after the learning outcome of each step is connected；It is using from attention mechanism and dot product function from attention layer respectively from the important local feature in semantic term vector sequence and part of speech term vector Sequence Learning sentence on each position, it obtains corresponding semantic attention vector sum part of speech and pays attention to force vector, and they are constrained by KL distance；Notice that force vector is weighted summation to two-way LSTM layers of output sequence using obtained semantic attention vector sum part of speech in merging layer, obtains the characterizing semantics and part of speech characterization of sentence, and obtain final sentence semantics to indicate；It is predicted finally by MLP output layer and output of classifying.

Description

Sentence classification method based on LSTM and combination part of speech and more attention mechanism
Technical field
The present invention relates to natural language processing fields, and in particular to one kind is based on LSTM and combines part of speech and more attention machines The sentence classification method of system.
Background technique
Sentence classification is the field natural language processing (Natural Language Processing, NLP) all the time One research hotspot.In recent years, the extensive use with deep learning in NLP, many scholars propose various based on length successively The sentence classification classification method of short-term memory model (Long Short-Term Model, LSTM), and in many sentences classification language Expect library such as Stanford Twitter Sentiment (STS), Stanford Sentiment Treebank binary classification (SSTb2) and on five yuan of classification (SSTb5), TREC, IMDB etc. the effect better than conventional machines learning method is achieved.Relatively For convolutional neural networks CNN, LSTM can preferably portray the contextual information of text sequence data and long-term rely on is closed System, and the gradient for efficiently avoiding the appearance of traditional RNN (Recurrent Neural Network) model disappears or ladder Explosion issues are spent, therefore are widely used in sentence classification task.
At present in the sentence disaggregated model various based on LSTM, mainly using resulting based on large-scale corpus training Word in sentence is converted into the mode of distributed expression by term vector.Existing research has been proved based on large-scale corpus training institute The term vector obtained contains more fully syntactic and semantic information, can greatly improve the effect of sentence classification.It is common at present Pre-training term vector be mainly to utilize CBOW the or Skip-gram model, GloVe algorithm or FastText algorithm of word2vec Deng training gained.These models or algorithm are based primarily upon in certain window the word co-occurrence letter of (or global) in training term vector Breath does not include the part-of-speech information of word itself.Therefore the information that the term vector come contains only content level is trained, is not had Embody the part-of-speech information of word.In general text categorization task (such as newsletter archive classification), knot of the Feature Words for classification Fruit has important indicative function, and these Feature Words are mainly based on noun or verb.For example, " typhoon will enter China east South is coastal " or " China will continue to reduce tax to medium-sized and small enterprises ".And in text emotion classification task, be used to indicate front or The viewpoint word or emotion word of negative emotion tendency are then even more important, these words are mainly based on verb or adjective.For example, " I Like this part film " or " this film is very good to be seen ".Correlative study also indicates that adjective is that viewpoint and the main of emotion are held Carry word.Therefore, the character representation of sentence can preferably be enriched by introducing part-of-speech information, to help to promote the effect that sentence is classified Fruit.In recent years, attention (Attention) mechanism in graph image is introduced into NLP by some scholars, and is appointed in many sons A series of state-of-the-art effects are obtained in business, such as machine translation, text snippet, Relation extraction, read understanding and text Originally contain.Attention mechanism enables model preferably to comprehensively consider in input source each element to the different shadows of objective result Power is rung, and is reduced since the detailed information occurred when sentence is longer loses problem.Some scholars also proposed from attention (Self-attention) mechanism, also referred to as interior attention (Intra-attention), main thought are to utilize each member in sentence The corresponding attention vector sum of the positional information calculation of element characterizes sentence.At present by LSTM and attention (or from attention) mechanism Combine the core for having become many models.But these researchs are simultaneously primarily directed to the attention in content level, equally The part-of-speech information of word is not accounted for.
Summary of the invention
It is a kind of based on LSTM and combination part of speech and more the purpose of the present invention is in view of the above shortcomings of the prior art, providing The sentence classification method of attention mechanism, the method can either make full use of Large Scale Corpus to can provide more accurate grammer And the advantages of semantic information, and the part-of-speech information that can introduce sentence further compensates for pre-training term vector and lacks part-of-speech information Deficiency, to preferably portray feature of the sentence in terms of syntax and semantics.
The purpose of the present invention can be achieved through the following technical solutions:
A kind of sentence classification method based on LSTM and combination part of speech and more attention mechanism, the method are based on following five Layer neural network model, first layer to layer 5 be respectively input layer, shared two-way LSTM layer, from attention layer, merging layer With MLP output layer, specifically includes the following steps:
After being pre-processed in input layer to sentence, be utilized respectively pre-training term vector table and based on it is equally distributed with The matrix that machine initialization generates provides each word and its mathematical notation of part of speech in sentence, so that each sentence is converted to semanteme Term vector matrix and part of speech term vector matrix；
The LSTM layer for passing through two opposite directions in shared two-way LSTM layer learns word in sentence or part of speech respectively Contextual information, and will be exported after the series connection of the learning outcome of each step；
It is using from attention mechanism and dot product function from attention layer respectively from semantic term vector sequence and part of speech word Sequence vector learns the important local feature in sentence on each position, obtains corresponding semantic attention vector sum part of speech and pays attention to Force vector, and they being constrained by KL distance, it is therefore an objective to guarantee that their distributions in sentence on each position to the greatest extent may be used It can be consistent；
Pay attention to force vector to two-way using from the obtained semantic attention vector sum part of speech of attention layer in merging layer LSTM layers of output sequence is weighted summation, obtains the characterizing semantics and part of speech characterization of sentence, then flat by comparing weighting , it connects, sum, maximizing various ways obtain final sentence semantics expression；
Finally by comprising connecting hidden layer entirely and the MLP output layer of softmax layer that connects is predicted and classified defeated entirely Out.
Further, it is described in input layer to sentence carry out pretreatment include sentence is segmented, forbidden character mistake Filter and the operation of length polishing.
Further, the neuronal quantity of hidden layer is connected in MLP output layer entirely according to input layer number, MLP output layer Obtained by the product of number of nodes extracts square root, the neuronal quantity of the softmax layer connected entirely is then the classification number of corresponding classification system Amount.
Further, in the training process of five layers of neural network model, semantic term vector is remained unchanged, and part of speech Term vector is adjusted using Back Propagation Algorithm.
Further, to guarantee that apart from as small as possible, KL distance is added and as nerve in the KL in loss function One of the target of network model optimization.
Compared with the prior art, the invention has the following advantages and beneficial effects:
Sentence classification method provided by the invention based on LSTM and combination part of speech and more attention mechanism, can either be abundant The advantages of can provide more accurate syntactic and semantic information using Large Scale Corpus, but can introduce the part-of-speech information of sentence into One step makes up the deficiency that pre-training term vector lacks part-of-speech information, to preferably portray spy of the sentence in terms of syntax and semantics Sign.The method also fully utilizes advantage of the LSTM in study sentence in terms of word and the contextual information of part of speech, and note Advantage of the power mechanism of anticipating in terms of learn the important local feature of sentence, the disaggregated model of proposition is with accuracy rate height and versatile The advantages that, in some famous open corpus, including 20Newsgroup corpus, IMDB corpus, Movie Review, TREC and Stanford Sentiment Treebank (SSTb) etc., achieves good effect.
Detailed description of the invention
Fig. 1 is the overall construction drawing of five layers of neural network model in the embodiment of the present invention.
Specific embodiment
Present invention will now be described in further detail with reference to the embodiments and the accompanying drawings, but embodiments of the present invention are unlimited In this.
Embodiment:
A kind of sentence classification method based on LSTM and combination part of speech and more attention mechanism is present embodiments provided, mainly Thinking is on the one hand to be indicated using the semantic term vector that pre-training term vector provides word in sentence, on the one hand utilizes part-of-speech tagging work Tool is labeled the word in sentence, and combining simplified part of speech tally set (mainly includes noun, verb, adjective, pair Word, closing tag UNK etc.) by Speech conversion at the form of serial number, then mapped and learnt by embeding layer；Then, sharp Learn the contextual information of semantic term vector and part of speech term vector respectively with a shared two-way LSTM, and by each time The forward direction and reversion choice result of step export after carrying out series connection merging, to respectively obtain the context relation of word and part of speech；? On the basis of this, it is directed to the semantic term vector sequence and part of speech term vector sequence of LSTM layers of output respectively from attention layer using one Learn the location information in sentence, and construct corresponding attention force vector, while attention force vector is carried out about using KL distance Beam, it is therefore an objective to guarantee when the attention weight of term vector semantic on some position is high, the attention weight of part of speech term vector Also high, to preferably capture the semanteme for classifying useful for sentence and part of speech feature；Then, a customized merging is utilized Layer will be used as together with the output of LSTM from the obtained two attentions force vector of attention layer and input, and be weighted and averaged respectively Summation afterwards obtains characterization of the sentence in terms of semantic and part of speech, and result is merged (be respectively adopted weighting stabilize, connect, summing, The multitude of different ways such as maximizing) obtain the final characterizing semantics of sentence；Finally, by one comprising full connection hidden layer and The multilayer perceptron MLP of softmax output layer is predicted and is classified output.In the learning process of model, for pre-training Term vector remains unchanged, and part of speech term vector is then adjusted during model training using Back Propagation Algorithm.
The method is based on following five layers of neural network model, and structure is as shown in Figure 1, first layer is respectively defeated to layer 5 Enter layer, shared two-way LSTM layers, from attention layer, merge layer and MLP output layer, the Partial key parameter such as table 1 in model It is shown:
Table 1
Model first layer first pre-processes sentence, mainly includes punctuation mark filtering, abbreviation polishing, deletes space Deng determining the length threshold of sentence then in conjunction with sentence length distribution and mean square deviation, and carry out length polishing；Then, on the one hand It is indicated, is on the other hand provided using NLTK each in sentence using the semantic vector that pre-training term vector table provides each word in sentence The part-of-speech tagging of a word is then combined with and simplifies the part of speech of same type and be converted into the form of serial number, followed by section [- 0.25,0.25] it is uniformly distributed the term vector that part of speech is initially to specified dimension at random on, and by embeding layer in model training Learnt in the process and is adjusted.For each sentence, finally by the available corresponding semantic term vector matrix of input layer With part of speech term vector matrix.During model training, semantic term vector is remained unchanged, and part of speech term vector is then learnt.
The second layer of model contains a shared two-way LSTM network.The semanteme of sentence obtained for input layer Term vector matrix and part of speech term vector matrix, each two-way LSTM learn it using a forward direction and a reverse LSTM Information above and below, and the learning outcome of each step is subjected to series connection output, finally respectively obtains one and include semanteme And the vector sum of contextual information one vector comprising part of speech and contextual information.
The third layer of model includes one from attention layer, using from attention mechanism and dot product function respectively from semantic word Important local feature in sequence vector and part of speech term vector sequence degree sentence on each position obtains corresponding semantic attention Force vector and part of speech pay attention to force vector, and are constrained by KL distance them.In order to guarantee KL apart from as small as possible, I Joined in loss function KL distance and one of the target as model optimization.
The 4th layer of model includes a customized merging layer, main to pay attention to using from the obtained semanteme of attention layer Force vector and part of speech notice that force vector is weighted summation to LSTM layers of output sequence, obtain the characterizing semantics and part of speech of sentence Then characterization merges to obtain final sentence semantics expression；We are flat by the way that weighting has been comprehensively compared during the experiment , a variety of merging modes such as series connection, summation, maximizing, and its result is analyzed, finally discovery weighted average and series connection Mode of the mode than summing merely or being maximized effect it is more preferable.
The layer 5 of model is a full connection hidden layer and a softmax layer returned for more sorted logics, is used Polynary cross entropy and rmsprop classifier based on stochastic gradient descent are predicted and are exported to the classification of sentence.Entire In the training process of model, the part of speech term vector in input layer is adjusted in combination with back-propagating, and optimize damage simultaneously Lose function and KL distance.
The above, only the invention patent preferred embodiment, but the scope of protection of the patent of the present invention is not limited to This, anyone skilled in the art is in the range disclosed in the invention patent, according to the present invention the skill of patent Art scheme and its patent of invention design are subject to equivalent substitution or change, belong to the scope of protection of the patent of the present invention.

Claims (5)

1. a kind of sentence classification method based on LSTM and combination part of speech and more attention mechanism, which is characterized in that the method Based on following five layers of neural network model, first layer to layer 5 be respectively input layer, shared two-way LSTM layers, from attention Layer merges layer and MLP output layer, specifically includes the following steps:
After pre-processing in input layer to sentence, it is utilized respectively pre-training term vector table and based on equally distributed random first Beginning metaplasia at matrix provide each word and its mathematical notation of part of speech in sentence, thus by each sentence be converted to semantic word to Moment matrix and part of speech term vector matrix；
LSTM layer in shared two-way LSTM layer through two opposite directions learns the upper and lower of word in sentence or part of speech respectively Literary information, and will be exported after the series connection of the learning outcome of each step；
It is using from attention mechanism and dot product function from attention layer respectively from semantic term vector sequence and part of speech term vector Important local feature in Sequence Learning sentence on each position, obtain corresponding semantic attention vector sum part of speech attention to Amount, and they are constrained by KL distance, it is therefore an objective to guarantee their distributions as far as possible one in sentence on each position It causes；
Pay attention to force vector to two-way LSTM using from the obtained semantic attention vector sum part of speech of attention layer in merging layer The output sequence of layer is weighted summation, the characterizing semantics and part of speech characterization of sentence is obtained, then by comparing weighted average, string Connection, summation, maximizing various ways obtain final sentence semantics expression；
Finally by comprising connecting hidden layer entirely and output is predicted and classified to the MLP output layer of softmax layer that connects entirely.
2. a kind of sentence classification method based on LSTM and combination part of speech and more attention mechanism according to claim 1, It is characterized by: it is described in input layer to sentence carry out pretreatment include sentence is segmented, forbidden character filters and long Spend polishing operation.
3. a kind of sentence classification method based on LSTM and combination part of speech and more attention mechanism according to claim 1, It is characterized by: the neuronal quantity of hidden layer is connected in MLP output layer entirely according to input layer number, MLP output layer number of nodes Product extract square root gained, the neuronal quantity of the softmax layer connected entirely is then the categorical measure of corresponding classification system.
4. a kind of sentence classification method based on LSTM and combination part of speech and more attention mechanism according to claim 1, It is characterized by: semantic term vector remains unchanged, and part of speech term vector in the training process of five layers of neural network model It is adjusted using Back Propagation Algorithm.
5. a kind of sentence classification method based on LSTM and combination part of speech and more attention mechanism according to claim 1, It is characterized by: to guarantee that apart from as small as possible, KL distance is added and as neural network mould in the KL in loss function One of the target of type optimization.
CN201811430542.7A 2018-11-28 2018-11-28 Sentence classification method based on LSTM and combined with part-of-speech and multi-attention mechanism Active CN109635109B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811430542.7A CN109635109B (en) 2018-11-28 2018-11-28 Sentence classification method based on LSTM and combined with part-of-speech and multi-attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811430542.7A CN109635109B (en) 2018-11-28 2018-11-28 Sentence classification method based on LSTM and combined with part-of-speech and multi-attention mechanism

Publications (2)

Publication Number Publication Date
CN109635109A true CN109635109A (en) 2019-04-16
CN109635109B CN109635109B (en) 2022-12-16

Family

ID=66069692

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811430542.7A Active CN109635109B (en) 2018-11-28 2018-11-28 Sentence classification method based on LSTM and combined with part-of-speech and multi-attention mechanism

Country Status (1)

Country Link
CN (1) CN109635109B (en)

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110147452A (en) * 2019-05-17 2019-08-20 北京理工大学 A kind of coarseness sentiment analysis method based on level BERT neural network
CN110347831A (en) * 2019-06-28 2019-10-18 西安理工大学 Based on the sensibility classification method from attention mechanism
CN110427627A (en) * 2019-08-02 2019-11-08 北京百度网讯科技有限公司 Task processing method and device based on semantic expressiveness model
CN110457682A (en) * 2019-07-11 2019-11-15 新华三大数据技术有限公司 Electronic health record part-of-speech tagging method, model training method and relevant apparatus
CN110532378A (en) * 2019-05-13 2019-12-03 南京大学 A kind of short text aspect extracting method based on topic model
CN110569499A (en) * 2019-07-18 2019-12-13 中国科学院信息工程研究所 Generating type dialog system coding method and coder based on multi-mode word vectors
CN110781306A (en) * 2019-10-31 2020-02-11 山东师范大学 English text aspect layer emotion classification method and system
CN110795563A (en) * 2019-10-31 2020-02-14 支付宝(杭州)信息技术有限公司 Text classification model training method, event detection method and corresponding devices
CN110929033A (en) * 2019-11-26 2020-03-27 深圳市信联征信有限公司 Long text classification method and device, computer equipment and storage medium
CN110941700A (en) * 2019-11-22 2020-03-31 福州大学 Multi-task joint learning-based argument mining system and working method thereof
CN111339772A (en) * 2020-03-16 2020-06-26 大连外国语大学 Russian text emotion analysis method, electronic device and storage medium
CN111581351A (en) * 2020-04-30 2020-08-25 识因智能科技(北京)有限公司 Dynamic element embedding method based on multi-head self-attention mechanism
CN111709230A (en) * 2020-04-30 2020-09-25 昆明理工大学 Short text automatic summarization method based on part-of-speech soft template attention mechanism
CN111737467A (en) * 2020-06-22 2020-10-02 华南师范大学 Object-level emotion classification method based on segmented convolutional neural network
CN111914085A (en) * 2020-06-18 2020-11-10 华南理工大学 Text fine-grained emotion classification method, system, device and storage medium
CN112084336A (en) * 2020-09-09 2020-12-15 浙江综合交通大数据中心有限公司 Entity extraction and event classification method and device for expressway emergency
CN112163429A (en) * 2020-09-27 2021-01-01 华南理工大学 Sentence relevancy obtaining method, system and medium combining cycle network and BERT
CN112287689A (en) * 2020-10-27 2021-01-29 山东省计算中心（国家超级计算济南中心） Judicial second-examination case situation auxiliary analysis method and system
CN112417890A (en) * 2020-11-29 2021-02-26 中国科学院电子学研究所苏州研究院 Fine-grained entity classification method based on diversified semantic attention model
CN112487796A (en) * 2020-11-27 2021-03-12 北京智源人工智能研究院 Method and device for sequence labeling and electronic equipment
CN112651225A (en) * 2020-12-29 2021-04-13 昆明理工大学 Multi-item selection machine reading understanding method based on multi-stage maximum attention
CN113268565A (en) * 2021-04-27 2021-08-17 山东大学 Method and device for quickly generating word vector based on concept text
CN113535948A (en) * 2021-06-02 2021-10-22 中国人民解放军海军工程大学 LSTM-Attention text classification method introducing essential point information
US20220019741A1 (en) * 2020-07-16 2022-01-20 Optum Technology, Inc. An unsupervised approach to assignment of pre-defined labels to text documents
CN114048319A (en) * 2021-11-29 2022-02-15 中国平安人寿保险股份有限公司 Attention mechanism-based humor text classification method, device, equipment and medium
CN114492420A (en) * 2022-04-02 2022-05-13 北京中科闻歌科技股份有限公司 Text classification method, device and equipment and computer readable storage medium
CN114547287A (en) * 2021-11-18 2022-05-27 电子科技大学 Generation type text abstract method
CN114579707A (en) * 2022-03-07 2022-06-03 桂林旅游学院 BERT neural network and multi-semantic learning-based aspect-level emotion analysis method
CN115906863A (en) * 2022-10-25 2023-04-04 华南师范大学 Emotion analysis method, device and equipment based on comparative learning and storage medium
US11941357B2 (en) 2021-06-23 2024-03-26 Optum Technology, Inc. Machine learning techniques for word-based text similarity determinations

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107590138A (en) * 2017-08-18 2018-01-16 浙江大学 A kind of neural machine translation method based on part of speech notice mechanism
US20180121787A1 (en) * 2016-11-03 2018-05-03 Salesforce.Com, Inc. Joint Many-Task Neural Network Model for Multiple Natural Language Processing (NLP) Tasks
US20180165554A1 (en) * 2016-12-09 2018-06-14 The Research Foundation For The State University Of New York Semisupervised autoencoder for sentiment analysis
CN108446275A (en) * 2018-03-21 2018-08-24 北京理工大学 Long text emotional orientation analytical method based on attention bilayer LSTM
CN108549658A (en) * 2018-03-12 2018-09-18 浙江大学 A kind of deep learning video answering method and system based on the upper attention mechanism of syntactic analysis tree
US20180329883A1 (en) * 2017-05-15 2018-11-15 Thomson Reuters Global Resources Unlimited Company Neural paraphrase generator

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180121787A1 (en) * 2016-11-03 2018-05-03 Salesforce.Com, Inc. Joint Many-Task Neural Network Model for Multiple Natural Language Processing (NLP) Tasks
US20180165554A1 (en) * 2016-12-09 2018-06-14 The Research Foundation For The State University Of New York Semisupervised autoencoder for sentiment analysis
US20180329883A1 (en) * 2017-05-15 2018-11-15 Thomson Reuters Global Resources Unlimited Company Neural paraphrase generator
CN107590138A (en) * 2017-08-18 2018-01-16 浙江大学 A kind of neural machine translation method based on part of speech notice mechanism
CN108549658A (en) * 2018-03-12 2018-09-18 浙江大学 A kind of deep learning video answering method and system based on the upper attention mechanism of syntactic analysis tree
CN108446275A (en) * 2018-03-21 2018-08-24 北京理工大学 Long text emotional orientation analytical method based on attention bilayer LSTM

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHOUHAN LIN ET AL.: "A STRUCTURED SELF-ATTENTIVE", 《ARXIV》 *

Cited By (51)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110532378A (en) * 2019-05-13 2019-12-03 南京大学 A kind of short text aspect extracting method based on topic model
CN110532378B (en) * 2019-05-13 2021-10-26 南京大学 Short text aspect extraction method based on topic model
CN110147452A (en) * 2019-05-17 2019-08-20 北京理工大学 A kind of coarseness sentiment analysis method based on level BERT neural network
CN110147452B (en) * 2019-05-17 2022-03-01 北京理工大学 Coarse grain emotion analysis method based on hierarchy BERT neural network
CN110347831A (en) * 2019-06-28 2019-10-18 西安理工大学 Based on the sensibility classification method from attention mechanism
CN110457682B (en) * 2019-07-11 2022-08-09 新华三大数据技术有限公司 Part-of-speech tagging method for electronic medical record, model training method and related device
CN110457682A (en) * 2019-07-11 2019-11-15 新华三大数据技术有限公司 Electronic health record part-of-speech tagging method, model training method and relevant apparatus
CN110569499A (en) * 2019-07-18 2019-12-13 中国科学院信息工程研究所 Generating type dialog system coding method and coder based on multi-mode word vectors
CN110569499B (en) * 2019-07-18 2021-10-08 中国科学院信息工程研究所 Generating type dialog system coding method and coder based on multi-mode word vectors
CN110427627A (en) * 2019-08-02 2019-11-08 北京百度网讯科技有限公司 Task processing method and device based on semantic expressiveness model
CN110427627B (en) * 2019-08-02 2023-04-28 北京百度网讯科技有限公司 Task processing method and device based on semantic representation model
CN110781306A (en) * 2019-10-31 2020-02-11 山东师范大学 English text aspect layer emotion classification method and system
CN110795563A (en) * 2019-10-31 2020-02-14 支付宝(杭州)信息技术有限公司 Text classification model training method, event detection method and corresponding devices
CN110781306B (en) * 2019-10-31 2022-06-28 山东师范大学 English text aspect layer emotion classification method and system
CN110941700B (en) * 2019-11-22 2022-08-09 福州大学 Multi-task joint learning-based argument mining system and working method thereof
CN110941700A (en) * 2019-11-22 2020-03-31 福州大学 Multi-task joint learning-based argument mining system and working method thereof
CN110929033A (en) * 2019-11-26 2020-03-27 深圳市信联征信有限公司 Long text classification method and device, computer equipment and storage medium
CN111339772B (en) * 2020-03-16 2023-11-14 大连外国语大学 Russian text emotion analysis method, electronic device and storage medium
CN111339772A (en) * 2020-03-16 2020-06-26 大连外国语大学 Russian text emotion analysis method, electronic device and storage medium
CN111709230A (en) * 2020-04-30 2020-09-25 昆明理工大学 Short text automatic summarization method based on part-of-speech soft template attention mechanism
CN111581351A (en) * 2020-04-30 2020-08-25 识因智能科技(北京)有限公司 Dynamic element embedding method based on multi-head self-attention mechanism
CN111581351B (en) * 2020-04-30 2023-05-02 识因智能科技(北京)有限公司 Dynamic element embedding method based on multi-head self-attention mechanism
CN111914085B (en) * 2020-06-18 2024-04-23 华南理工大学 Text fine granularity emotion classification method, system, device and storage medium
CN111914085A (en) * 2020-06-18 2020-11-10 华南理工大学 Text fine-grained emotion classification method, system, device and storage medium
CN111737467B (en) * 2020-06-22 2023-05-23 华南师范大学 Object-level emotion classification method based on segmented convolutional neural network
CN111737467A (en) * 2020-06-22 2020-10-02 华南师范大学 Object-level emotion classification method based on segmented convolutional neural network
US20220019741A1 (en) * 2020-07-16 2022-01-20 Optum Technology, Inc. An unsupervised approach to assignment of pre-defined labels to text documents
CN112084336A (en) * 2020-09-09 2020-12-15 浙江综合交通大数据中心有限公司 Entity extraction and event classification method and device for expressway emergency
CN112163429A (en) * 2020-09-27 2021-01-01 华南理工大学 Sentence relevancy obtaining method, system and medium combining cycle network and BERT
CN112163429B (en) * 2020-09-27 2023-08-29 华南理工大学 Sentence correlation obtaining method, system and medium combining cyclic network and BERT
CN112287689A (en) * 2020-10-27 2021-01-29 山东省计算中心（国家超级计算济南中心） Judicial second-examination case situation auxiliary analysis method and system
CN112287689B (en) * 2020-10-27 2022-06-24 山东省计算中心（国家超级计算济南中心） Judicial second-examination case situation auxiliary analysis method and system
CN112487796A (en) * 2020-11-27 2021-03-12 北京智源人工智能研究院 Method and device for sequence labeling and electronic equipment
CN112417890B (en) * 2020-11-29 2023-11-24 中国科学院电子学研究所苏州研究院 Fine granularity entity classification method based on diversified semantic attention model
CN112417890A (en) * 2020-11-29 2021-02-26 中国科学院电子学研究所苏州研究院 Fine-grained entity classification method based on diversified semantic attention model
CN112651225A (en) * 2020-12-29 2021-04-13 昆明理工大学 Multi-item selection machine reading understanding method based on multi-stage maximum attention
CN112651225B (en) * 2020-12-29 2022-06-14 昆明理工大学 Multi-item selection machine reading understanding method based on multi-stage maximum attention
CN113268565A (en) * 2021-04-27 2021-08-17 山东大学 Method and device for quickly generating word vector based on concept text
CN113268565B (en) * 2021-04-27 2022-03-25 山东大学 Method and device for quickly generating word vector based on concept text
CN113535948A (en) * 2021-06-02 2021-10-22 中国人民解放军海军工程大学 LSTM-Attention text classification method introducing essential point information
CN113535948B (en) * 2021-06-02 2022-08-16 中国人民解放军海军工程大学 LSTM-Attention text classification method introducing essential point information
US11941357B2 (en) 2021-06-23 2024-03-26 Optum Technology, Inc. Machine learning techniques for word-based text similarity determinations
CN114547287B (en) * 2021-11-18 2023-04-07 电子科技大学 Generation type text abstract method
CN114547287A (en) * 2021-11-18 2022-05-27 电子科技大学 Generation type text abstract method
CN114048319A (en) * 2021-11-29 2022-02-15 中国平安人寿保险股份有限公司 Attention mechanism-based humor text classification method, device, equipment and medium
CN114048319B (en) * 2021-11-29 2024-04-23 中国平安人寿保险股份有限公司 Humor text classification method, device, equipment and medium based on attention mechanism
CN114579707B (en) * 2022-03-07 2023-07-28 桂林旅游学院 Aspect-level emotion analysis method based on BERT neural network and multi-semantic learning
CN114579707A (en) * 2022-03-07 2022-06-03 桂林旅游学院 BERT neural network and multi-semantic learning-based aspect-level emotion analysis method
CN114492420A (en) * 2022-04-02 2022-05-13 北京中科闻歌科技股份有限公司 Text classification method, device and equipment and computer readable storage medium
CN115906863B (en) * 2022-10-25 2023-09-12 华南师范大学 Emotion analysis method, device, equipment and storage medium based on contrast learning
CN115906863A (en) * 2022-10-25 2023-04-04 华南师范大学 Emotion analysis method, device and equipment based on comparative learning and storage medium

Also Published As

Publication number Publication date
CN109635109B (en) 2022-12-16

Similar Documents

Publication Publication Date Title
CN109635109A (en) Sentence classification method based on LSTM and combination part of speech and more attention mechanism
Abdullah et al. SEDAT: sentiment and emotion detection in Arabic text using CNN-LSTM deep learning
CN108984745B (en) Neural network text classification method fusing multiple knowledge maps
US20220147836A1 (en) Method and device for text-enhanced knowledge graph joint representation learning
CN109558487A (en) Document Classification Method based on the more attention networks of hierarchy
CN109325112B (en) A kind of across language sentiment analysis method and apparatus based on emoji
CN108829722A (en) A kind of Dual-Attention relationship classification method and system of remote supervisory
CN107562792A (en) A kind of question and answer matching process based on deep learning
CN107832400A (en) A kind of method that location-based LSTM and CNN conjunctive models carry out relation classification
US11580975B2 (en) Systems and methods for response selection in multi-party conversations with dynamic topic tracking
CN109614487A (en) A method of the emotional semantic classification based on tensor amalgamation mode
CN109598387A (en) Forecasting of Stock Prices method and system based on two-way cross-module state attention network model
CN109710769A (en) A kind of waterborne troops's comment detection system and method based on capsule network
CN110321563A (en) Text emotion analysis method based on mixing monitor model
CN107679225A (en) A kind of reply generation method based on keyword
CN114881042B (en) Chinese emotion analysis method based on graph-convolution network fusion of syntactic dependency and part of speech
Marreddy et al. Clickbait detection in telugu: Overcoming nlp challenges in resource-poor languages using benchmarked techniques
CN109871449A (en) A kind of zero sample learning method end to end based on semantic description
CN111914553B (en) Financial information negative main body judging method based on machine learning
CN113157919A (en) Sentence text aspect level emotion classification method and system
Cheng et al. DOCUMENT CLASSIFICATION BASED ON CONVOLUTIONAL NEURAL NETWORK AND HIERARCHICAL ATTENTION NETWORK.
CN112182227A (en) Text emotion classification system and method based on transD knowledge graph embedding
Wei et al. Named entity recognition method for educational emergency field based on BERT
Zhu et al. A Semantic Similarity Computing Model based on Siamese Network for Duplicate Questions Identification.
Wang et al. Emotion analysis of microblog based on emotion dictionary and Bi-GRU

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant