CN110489750A - Burmese participle and part-of-speech tagging method and device based on two-way LSTM-CRF - Google Patents
Burmese participle and part-of-speech tagging method and device based on two-way LSTM-CRF Download PDFInfo
- Publication number
- CN110489750A CN110489750A CN201910739718.5A CN201910739718A CN110489750A CN 110489750 A CN110489750 A CN 110489750A CN 201910739718 A CN201910739718 A CN 201910739718A CN 110489750 A CN110489750 A CN 110489750A
- Authority
- CN
- China
- Prior art keywords
- burmese
- participle
- speech
- sentence
- word
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012549 training Methods 0.000 claims abstract description 73
- 239000013598 vector Substances 0.000 claims abstract description 57
- 230000006870 function Effects 0.000 claims abstract description 13
- 230000011218 segmentation Effects 0.000 claims description 27
- 230000001419 dependent effect Effects 0.000 claims description 13
- 239000000284 extract Substances 0.000 claims description 11
- 238000012512 characterization method Methods 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000007812 deficiency Effects 0.000 abstract 1
- 238000003058 natural language processing Methods 0.000 abstract 1
- 238000002474 experimental method Methods 0.000 description 21
- 230000000694 effects Effects 0.000 description 17
- 238000012360 testing method Methods 0.000 description 9
- 230000000052 comparative effect Effects 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Machine Translation (AREA)
Abstract
The present invention relates to Burmese participles and part-of-speech tagging method and device based on two-way LSTM-CRF, belong to natural language processing technique field.The present invention generates the term vector based on syllable splitting using BERT and indicates；Burmese is carried out using two-way LSTM-CRF and segments task pre-training, obtains Burmese participle optimal models；Burmese part-of-speech tagging task pre-training is carried out using two-way LSTM-CRF, obtains Burmese part of speech label optimal models；By calling trained participle or part-of-speech tagging model, the Burmese sentence after the vectorization of input is segmented, part of speech label prediction；The participle of the Burmese based on two-way LSTM-CRF and part-of-speech tagging device is made according to above-mentioned steps function modoularization.The present invention has been able to achieve to Burmese sentence while participle and part-of-speech tagging, solves the problems, such as that Burmese causes to segment and mark inaccuracy due to data deficiency.
Description
Technical field
The present invention relates to Burmese participles and part-of-speech tagging method and device based on two-way LSTM-CRF, belong to nature language
Say processing technology field.
Background technique
Burmese participle and part-of-speech tagging are the background tasks in the research of Burmese natural language, traditional Burmese participle
Method with part of speech label is handled based on statistical learning mostly, and the method based on statistical learning needs the Burma largely marked
Pasture language corpus, handmarking are time-consuming and laborious.Currently, by the method for Burmese participle and part of speech label task combination deep learning
Manage effect it is more preferable, but simply by Burmese carry out term vector characterization be it is inadequate, the smallest unit-in-context of Burmese is sound
Section, can be more preferable in conjunction with the effect of syllable characteristic processing Burmese participle and part of speech label task.In addition, most of research work
It is all to separate to handle by Burmese participle and part of speech label task, Burmese part of speech label accuracy relies heavily on
In the order of accuarcy of participle, Burmese participle and part of speech label are separately handled to the mistake for being easy for causing part-of-speech tagging task,
Therefore, set forth herein Burmese of the method for the deep learning based on two-way LSTM-CRF to scarcity of resources type to carry out participle and word
Property mark joint training.
Summary of the invention
The Burmese participle and part-of-speech tagging method that the present invention provides a kind of based on two-way LSTM-CRF, for solving
Burmese is combined as the participle inaccuracy of Burmese caused by corpus that lacks in training, and by Burmese participle and part-of-speech tagging
Can solve the not high problem of Burmese part-of-speech tagging accuracy rate.
The technical scheme is that a kind of Burmese participle and part-of-speech tagging method based on two-way LSTM-CRF, institute
State based on two-way LSTM-CRF Burmese participle and part-of-speech tagging method specific step is as follows:
Step1, Burmese pretreatment carry out term vector characterization: utilizing the syllable in BERT combination Burmese word context
Information, generating the term vector based on syllable splitting indicates；
Step2, coding training is carried out to the Burmese after term vector expression using two-way LSTM, extracts Burmese participle
Feature carries out the mark of word segmentation to global Burmese word using CRF, and training Burmese participle model simultaneously obtains Burmese participle most
Excellent model；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is predicted, obtains the overall situation most
Excellent solution prediction output Burmese word segmentation result；
Step3, coding training is carried out to the Burmese after above-mentioned participle using two-way LSTM, extracts Burmese part-of-speech tagging
Feature, using CRF to global Burmese word carry out part of speech label, training Burmese part of speech markup model simultaneously obtain Burmese
Part of speech marks optimal models；Optimal models are marked using the good part of speech of Burmese pre-training, the Burmese sentence of input is carried out
Prediction obtains globally optimal solution prediction output Burmese part-of-speech tagging result.
Step4, pass through and call trained participle or part-of-speech tagging model, to Burma's sentence after the vectorization of input
Son is segmented, part of speech label is predicted.
As a preferred solution of the present invention, the specific steps of the step Step1 are as follows:
Step1.1, a point Burmese sentence for good word is got by Asian language treebank website；
Step1.2, the word of the Burmese sentence of good word will be divided according to syllable splitting using Burmese syllable splitting tool
Word；
Step1.3, it is indicated using BERT pre-training Burmese based on the term vector of syllable splitting.
The specific steps of the step Step1.1 are as follows: by Asian language treebank website (http: //
Www2.nict.go.jp/astrec-att/member/mutiyama/ALT/ Burma's sentence of 20106 points of good words) is got
Son.For example,
The specific steps of the step Step1.2 are as follows:
Burmese minimum unit is syllable, for example, Chinese semantic meaning is that " my dog is very lovely." Burmese sentenceIt is indicated using the Burmese word after syllable splitting
As a preferred solution of the present invention, the specific steps of the step Step1.3 are as follows:
Step1.3.1,20106 Burmese sentences progress duplicate removal of acquisition is cut into 70002 Burmese words, led to
The multilingual model for calling BERT is crossed, the Burmese term vector for directly generating 768 dimensions with contextual information indicates；
Step1.3.2, again by after syllable splitting Burmese word generate term vector indicate；
Step1.3.3, finally, the two is spliced, obtaining, there is the term vector of Burmese syllable information to characterize.
As a preferred solution of the present invention, the specific steps of the step Step2 are as follows:
Step2.1, Burmese is extracted based on the Burmese participle feature that syllable vector generates using two-way LSTM,
Obtaining complete two-way LSTM indicates；
Step2.2, the mark of word segmentation is carried out to global Burmese word using CRF, training Burmese participle model simultaneously obtains
Burmese segments optimal models；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is carried out pre-
It surveys, obtains globally optimal solution prediction output Burmese word segmentation result.
As a preferred solution of the present invention, the specific steps of the step Step2.1 are as follows:
Step2.1.1, as unit of sentence, a Burmese sentence containing i word, which is carried out vectorization, indicates note
Make: X=(x1,x2,x3,x4,...,)；
The hidden status switch of Step2.1.2, forward direction LSTM outputWith reversed LSTM'sEach
The hidden state of position output carries out opsition dependent and splices to obtain complete hidden status switch: ht=[ht；ht]∈Rn×m, obtain one
The two-way LSTM of a complete Burmese segmentation sequence is indicated.
As a preferred solution of the present invention, the specific steps of the step Step2.2 are as follows:
Step2.2.1、AijIndicate the transfer score of the label from the i-th to j-th；Represent yiThe label of a sentence
tagyiIt is transferred to yi+1The label tagy of a sentencei+1A possibility that marking, length is equal to the label sequence of sentence length
Arrange y=(y1,y2,...,yn), then marking of the participle model for the label of sentence x equal to y is s (X, y)Participle,Table
Show from the participle probability after n word prediction:
Step2.2.2, normalized using Softmax after participle probability P (y | X)Participle:
Wherein,Indicate the process for solving logarithm in participle model to each Burmese word；
By maximizing log-likelihood function when Step2.2.3, the training of Burmese participle model, a training sample is obtained
(x,yx) participle loss function log (p (y | x))ParticipleIt is as follows:
Wherein,Expression is worked asWhen participle logarithm；It indicates to belong to Yx's y-th
Sentence, x indicates that Burmese word, Y indicate that Burmese sentence, Yx indicate that multiple sentences have the collection of Burmese word utterances in Yx
It closes；
Step2.2.4, Burmese participle model use the Viterbi algorithm of Dynamic Programming in prediction process when decoding
To solve optimal participle prediction result y*Participle:
Wherein,Expression is worked asWhen, the maximum value of score.
As a preferred solution of the present invention, the specific steps of the step Step3 are as follows:
Step3.1, Burmese is carried out based on the Burmese part-of-speech tagging feature that syllable vector generates using two-way LSTM
It extracts, obtaining complete two-way LSTM indicates；
Step3.2, part of speech label is carried out to global Burmese word using CRF, training Burmese part of speech markup model is simultaneously
Obtain Burmese part of speech label optimal models；Optimal models are marked using the good part of speech of Burmese pre-training, to the Burma of input
Sentence is predicted, globally optimal solution prediction output Burmese part-of-speech tagging result is obtained.
As a preferred solution of the present invention, the specific steps of the step Step3.1 are as follows:
Step3.1.1, as unit of sentence, the Burmese sentence of a point good word containing i word is subjected to vector
Changing indicates to be denoted as: XBurmese after participle=(x1, x2, x3, x4...)；
The hidden status switch of Step3.1.2, forward direction LSTM outputWith reversed LSTM'sIn
The hidden state of each position output carries out opsition dependent and splices to obtain complete hidden status switch: ht=[ht；ht]∈Rn×m, obtain
Two-way LSTM to a complete Burmese part of speech flag sequence is indicated.
As a preferred solution of the present invention, the specific steps of the step Step3.2 are as follows:
Step3.2.1、AijIndicate the transfer score of the label from the i-th to j-th；Represent yiThe label of a sentence
tagyiIt is transferred to yi+1The label tagy of a sentencei+1A possibility that marking, length is equal to the label sequence of sentence length
Arrange y=(y1,y2,...,yn), then marking of the part of speech markup model for the label of sentence x equal to y is s (X, y)Part of speech label,Indicate the part of speech marking probability after predicting from n word:
Step3.2.2, normalized using Softmax after part of speech marking probability be P (y | X)Part of speech label:
Indicate the process for solving logarithm in part of speech markup model to each Burmese word；
By maximizing log-likelihood function when Step3.2.3, the training of Burmese part of speech markup model, a training is obtained
Sample (x, yx) part of speech label loss function log (p (y | x))Part of speech label:
Wherein,Expression is worked asWhen part of speech label logarithm；It indicates to belong to Yx y-th
Sentence, x indicates that Burmese word, Y indicate that Burmese sentence, Yx indicate that multiple sentences have the collection of Burmese word utterances in Yx
It closes；
Step3.2.4, Burmese part of speech markup model use the Viterbi of Dynamic Programming in prediction process when decoding
Algorithm marks prediction result y* to solve optimal part of speechPart of speech label:
Wherein,Expression is worked asWhen, the maximum value of score.
A kind of Burmese participle and part-of-speech tagging device based on two-way LSTM-CRF, including following module:
Burmese preprocessing module, for generating base using the syllable information in BERT combination Burmese word context
It is indicated in the term vector of syllable characteristic cutting；
Burmese word segmentation module is mentioned for carrying out coding training to the Burmese after term vector expression using two-way LSTM
The feature for taking Burmese to segment carries out the mark of word segmentation to global Burmese word using CRF, and training Burmese participle model simultaneously obtains
Optimal models are segmented to Burmese；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is carried out
Prediction obtains globally optimal solution prediction output Burmese word segmentation result；
Burmese part-of-speech tagging module, for being carried out using two-way LSTM to the Burmese after Burmese word segmentation module participle
Coding training, extracts the feature of Burmese part-of-speech tagging, carries out part of speech label to global Burmese word using CRF, training is remote
Pasture words and phrases markup model simultaneously obtains Burmese part of speech label optimal models；It is marked using the good part of speech of Burmese pre-training optimal
Model predicts the Burmese sentence of input, obtains globally optimal solution prediction output Burmese part-of-speech tagging result；
Burmese sentence participle, part-of-speech tagging module are predicted, for by calling trained participle or part-of-speech tagging
Model segments the Burmese sentence after the vectorization of input, part of speech label prediction.
The beneficial effects of the present invention are:
1, the present invention utilizes BERT models coupling Burmese syllable characteristic information, generates the word with Burmese syllable information
Vector indicates, strengthens the ability of Burmese term vector characterization, to improve the performance of Burmese participle and part of speech label；
2, present invention joint Burmese participle and part of speech mark task, reinforce the characterization ability of word, promote Burmese point
The performance of word and part of speech label；
3, the present invention is generated special based on syllable firstly, using the syllable information in BERT combination Burmese word context
Levy cutting term vector indicate, secondly, encoded using two-way LSTM, respectively extract Burmese participle and part-of-speech tagging spy
Sign, can combine participle and part-of-speech tagging training mission in this way and the feature extracted transmitting is independent of each other.Finally, sharp
The result of globally optimal solution prediction output Burmese participle and part-of-speech tagging is obtained with CRF.
Detailed description of the invention
Fig. 1 is total flow chart in the present invention；
Fig. 2 is the flow diagram of the method for the present invention；
Fig. 3 is Burmese participle and part-of-speech tagging apparatus structure block diagram based on two-way LSTM-CRF.
Specific embodiment
Embodiment 1: as shown in Figure 1-3, a kind of Burmese participle and part-of-speech tagging method based on two-way LSTM-CRF, institute
State based on two-way LSTM-CRF Burmese participle and part-of-speech tagging method specific step is as follows:
Step1, Burmese pretreatment carry out term vector characterization: utilizing the syllable in BERT combination Burmese word context
Information, generating the term vector based on syllable splitting indicates；
Step1.1, pass through Asian language treebank website (http://www2.nict.go.jp/astrec-att/
Member/mutiyama/ALT/ the Burmese sentence of 20106 points of good words) is got.For example,
Table 1 gets the format of Burmese sentence
Step1.2, the word of the Burmese sentence of good word will be divided according to syllable splitting using Burmese syllable splitting tool
Word；
Burmese minimum unit is syllable, for example, Chinese semantic meaning is that " my dog is very lovely." Burmese sentenceIt is indicated using the Burmese word after syllable splittingIt specifically can be as shown in table 2:
Burmese word after 2 syllable splitting of table indicates
Step1.3, it is indicated using BERT pre-training Burmese based on the term vector of syllable splitting.
Step1.3.1,20106 Burmese sentences progress duplicate removal of acquisition is cut into 70002 Burmese words, led to
The multilingual model for calling BERT is crossed, the Burmese term vector for directly generating 768 dimensions with contextual information indicates；
Step1.3.2, again by after syllable splitting Burmese word generate term vector indicate；
Step1.3.3, finally, the two is spliced, obtaining, there is the term vector of Burmese syllable information to characterize.
Step2, coding training is carried out to the Burmese after term vector expression using two-way LSTM, extracts Burmese participle
Feature carries out the mark of word segmentation to global Burmese word using CRF, and training Burmese participle model simultaneously obtains Burmese participle most
Excellent model；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is predicted, obtains the overall situation most
Excellent solution prediction output Burmese word segmentation result；
As a preferred solution of the present invention, the specific steps of the step Step2 are as follows:
Step2.1, Burmese is extracted based on the Burmese participle feature that syllable vector generates using two-way LSTM,
Obtaining complete two-way LSTM indicates；
Step2.2, the mark of word segmentation is carried out to global Burmese word using CRF, training Burmese participle model simultaneously obtains
Burmese segments optimal models；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is carried out pre-
It surveys, obtains globally optimal solution prediction output Burmese word segmentation result.
As a preferred solution of the present invention, the specific steps of the step Step2.1 are as follows:
Step2.1.1, as unit of sentence, a Burmese sentence containing i word, which is carried out vectorization, indicates note
Make: X=(x1,x2,x3,x4,...,)；
Step2.1.2, using the good term vector of pre-training by each Burmese word x in sentencei∈RdIt is mapped to corresponding
Matrix in, d be term vector training dimension.Before inputting next layer, it is arranged dropout layers to alleviate over-fitting.
By the sequence vector (x of each word of a sentence1,x2,x3,x4...) defeated at various locations as two-way LSTM
Hidden state out carries out the input that opsition dependent splices M time step, then the hidden status switch that positive LSTM is exported
With reversed LSTM'sThe hidden state exported at various locations carries out opsition dependent and splices to obtain a complete hidden state
Sequence:
ht=[ht；ht]∈Rn×m
It is indicated by the two-way LSTM of above formula available one complete Burmese segmentation sequence.
After dropout is set, a linear layer is accessed, hidden state vector is mapped to k dimension from m dimension, k is mark collection
Number of tags, so that the feature of the Burmese participle automatically extracted, is denoted as:
P=(p1,p2,...,pn)∈Rn×k
It can be the p in above-mentioned formulai∈RkPer one-dimensional pijIt can be regarded as xiIt is categorized into the marking of j-th of label
Value.Pi,yiBe i-th of Word prediction be yiThe label tagy of a sentenceiA possibility that.
As a preferred solution of the present invention, the specific steps of the step Step2.2 are as follows:
Step2.2.1, CRF layers of parameter is the matrix A of one (k+2) × (k+2), AijIndicate the label from the i-th to j-th
Transfer score；Represent yiThe label tagy of a sentenceiIt is transferred to yi+1The label tagy of a sentencei+1Possibility
Property marking, length is equal to the sequence label y=(y of sentence length1,y2,...,yn), then participle model is for sentence x
Label equal to y marking be s (X, y)Participle,Indicate the participle probability after predicting from n word:
Step2.2.2, normalized using Softmax after participle probability P (y | X)Participle:
Wherein,Indicate the process for solving logarithm in participle model to each Burmese word；
By maximizing log-likelihood function when Step2.2.3, the training of Burmese participle model, a training sample is obtained
(x,yx) participle loss function log (p (y | x))ParticipleIt is as follows:
Wherein,Expression is worked asWhen participle logarithm；It indicates to belong to Yx's y-th
Sentence, x indicates that Burmese word, Y indicate that Burmese sentence, Yx indicate that multiple sentences have the collection of Burmese word utterances in Yx
It closes；
Step2.2.4, Burmese participle model use the Viterbi algorithm of Dynamic Programming in prediction process when decoding
To solve optimal participle prediction result y*Participle:
Wherein,Expression is worked asWhen, the maximum value of score.
Step3, coding training is carried out to the Burmese after above-mentioned participle using two-way LSTM, extracts Burmese part-of-speech tagging
Feature, using CRF to global Burmese word carry out part of speech label, training Burmese part of speech markup model simultaneously obtain Burmese
Part of speech marks optimal models；Optimal models are marked using the good part of speech of Burmese pre-training, the Burmese sentence of input is carried out
Prediction obtains globally optimal solution prediction output Burmese part-of-speech tagging result.
As a preferred solution of the present invention, the specific steps of the step Step3 are as follows:
Step3.1, Burmese is carried out based on the Burmese part-of-speech tagging feature that syllable vector generates using two-way LSTM
It extracts, obtaining complete two-way LSTM indicates；
Step3.2, part of speech label is carried out to global Burmese word using CRF, training Burmese part of speech markup model is simultaneously
Obtain Burmese part of speech label optimal models；Optimal models are marked using the good part of speech of Burmese pre-training, to the Burma of input
Sentence is predicted, globally optimal solution prediction output Burmese part-of-speech tagging result is obtained.
As a preferred solution of the present invention, the specific steps of the step Step3.1 are as follows:
Step3.1.1, as unit of sentence, the Burmese sentence of a point good word containing i word is subjected to vector
Changing indicates to be denoted as: XBurmese after participle=(x1, x2, x3, x4...)；
Step3.1.2, using the good term vector of pre-training by each Burmese word x in sentencei∈RdIt is mapped to corresponding
Matrix in, d be term vector training dimension.Before inputting next layer, it is arranged dropout layers to alleviate over-fitting.
By the Burmese sentence sequence vector X of point good word of each word of a sentenceBurmese after participle=(x1, x2, x3,
x4...) the hidden state that is exported at various locations as two-way LSTM carries out the input that opsition dependent splices M time step, then general
The hidden status switch of positive LSTM outputWith reversed LSTM'sThe hidden state exported at various locations into
Row opsition dependent splices to obtain a complete hidden status switch:
ht=[ht；ht]∈Rn×m
It is indicated by the two-way LSTM of above formula available one complete Burmese part of speech flag sequence.
After dropout is set, a linear layer is accessed, hidden state vector is mapped to k dimension from m dimension, k is mark collection
Number of tags, so that the feature of the Burmese part-of-speech tagging automatically extracted, is denoted as:
P=(p1,p2,...,pn)∈Rn×k
It can be the p in above-mentioned formulai∈RkPer one-dimensional pijIt can be regarded as xiIt is categorized into the marking of j-th of label
Value.Be i-th of Word prediction be yiThe label tagy of a sentenceiA possibility that.
As a preferred solution of the present invention, the specific steps of the step Step3.2 are as follows:
Step3.2.1, CRF layers of parameter is the matrix A of one (k+2) × (k+2), AijIndicate the label from the i-th to j-th
Transfer score；Represent yiThe label tagy of a sentenceiIt is transferred to yi+1The label tagy of a sentencei+1Possibility
Property marking, length is equal to the sequence label y=(y of sentence length1,y2,...,yn), then part of speech markup model for
Marking of the label of sentence x equal to y is s (X, y)Part of speech label,Indicate the part of speech marking probability after predicting from n word:
Step3.2.2, normalized using Softmax after part of speech marking probability be P (y | X)Part of speech label:
Indicate the process for solving logarithm in part of speech markup model to each Burmese word；
By maximizing log-likelihood function when Step3.2.3, the training of Burmese part of speech markup model, a training is obtained
Sample (x, yx) part of speech label loss function log (p (y | x))Part of speech label:
Wherein,Expression is worked asWhen part of speech label logarithm；It indicates to belong to Yx y-th
Sentence, x indicates that Burmese word, Y indicate that Burmese sentence, Yx indicate that multiple sentences have the collection of Burmese word utterances in Yx
It closes；
Step3.2.4, Burmese part of speech markup model use the Viterbi of Dynamic Programming in prediction process when decoding
Algorithm marks prediction result y* to solve optimal part of speechPart of speech label:
Wherein,Expression is worked asWhen, the maximum value of score.
Step4, pass through and call trained participle or part-of-speech tagging model, to Burma's sentence after the vectorization of input
Son is segmented, part of speech label is predicted.
In order to illustrate effect of the invention, the present invention has carried out following comparative experiments, and used experimental data is from Asia
The Burmese data set of continent low-resource language treebank comprising 20011 points of good words and has the Burmese sentence of part-of-speech tagging, will
After Burmese sentence removes repetition, 75498 Burmese words are generated after carrying out cutting.Experiment Training collection and test set according to
The ratio of 9:1 is divided.
In strict accordance with standard evaluation index accurate rate (Precision), recall rate (Recall) and F value (F1- in experiment
Measure formula) is as follows:
Wherein TP is genuine positive example, and FP is false counter-example, and FP is false positive example, and TN is genuine counter-example.
In order to verify the effect of two-way LSTM-CRF model proposed in this paper, design following comparative testing is analyzed.
It compares with traditional CRF mode and is compared with the model of CNN, the results are shown in Table 3 for specific experiment.
In the case where ensuring its dependent variable all unanimous circumstances, contrast model parameter is based on the former hyper parameter setting provided, the experiment of survey
Shown in P, R, F1 of test data table 3 specific as follows.
The different neural network model comparisons of table 3
After using the Burmese participle of different models and part-of-speech tagging result by comparison, it was therefore concluded that: application is based on sound
The participle of the BERT+ of section two-way LSTM-CRF, part-of-speech tagging effect are an advantage over other three kinds.
Whether can be had an impact to experimental result for verifying using the experiment effect of different term vectors.Contrived experiment compares such as
Under:
The expression that Burmese term vector is generated using the mode of word2vec, in experiment 2, by changing the Burma generated
The dimension size comparative experiments effect of words and phrases vector.In the case where ensuring its dependent variable all unanimous circumstances, the equal base of comparative experiments parameter
In the hyper parameter setting that original provides, P, R, F1 of the test data of the experiment of survey are specifically as shown in table 4.
Table 4 is the comparison different with term vector mode is generated of the of different sizes of term vector dimension
It is concluded that using the Burmese term vector ratio using BERT method based on syllable splitting directly by analysis comparison
It to be got well using the effect of Word2vec.And generating the process of term vector in Word2vec is not that dimension is the bigger the better.
Dropout layers are provided in hyper parameter setting, the main purpose of Droupout layers of setting is can be relatively effective
The generation for alleviating over-fitting, achievees the effect that regularization to a certain extent.Compare examination by the way that different size is arranged in experiment 3
Test effect.In the case where ensuring its dependent variable all unanimous circumstances, comparative experiments parameter is based on the former hyper parameter setting provided, survey
P, R, F1 of the test data of experiment are specifically as shown in table 5.
Table 5 is the comparison of different sizes of the Dropout number of plies
Dropout layers of size, which is arranged, prevents over-fitting, similar to giving up a part of neuron, but can be in experiment 3
Find out not to be that the number of plies is the bigger the better, specific effect is as follows: 0.4 > Dropout layers of 0.8 > Dropout of size of Dropout layers of size
Layer size 0.6 has and selects Dropout layers of size 0.4 most suitable in experimentation known to above-mentioned experimental result.
Burmese minimum unit is syllable, is exactly secondly character, is cut by comparing the participle based on character with based on syllable
The effect of the Burmese participle divided.In the case where ensuring its dependent variable all unanimous circumstances, comparative experiments parameter is based on what original provided
Hyper parameter setting, P, R, F1 of the test data of the experiment of survey are specifically as shown in table 6.
Table 6 is based on character and based on syllable participle comparison
After comparison, it is found that the effect of the Burmese term vector generated based on syllable splitting is better than based on character rank
Generate term vector.The language feature of Burmese itself can be better blended into conjunction with the feature of Burmese syllable, incorporate Burmese
Contextual information, preferably segmented, the effect of part-of-speech tagging.
The size of experimental data set also will affect final participle effect, by changing the size of experimental data set to having a competition
Test result.In the case where ensuring its dependent variable all unanimous circumstances, comparative experiments parameter is based on the former hyper parameter setting provided, survey
P, R, F1 of the test data of experiment are specifically as shown in table 7.
The comparison of different sizes of 7 data set of table
Experiment effect will be will affect using the size of the method training pattern data set of deep learning, pass through 5 comparison of experiment
The effect obtained on the Burmese data set marked known to analysis result at 20000 is best.
Design according to the present invention, the Burmese participle and part of speech that the present invention also provides a kind of based on two-way LSTM-CRF
Annotation equipment, as shown in figure 3, the device includes following module:
Burmese preprocessing module, for generating base using the syllable information in BERT combination Burmese word context
It is indicated in the term vector of syllable characteristic cutting；
Burmese word segmentation module is mentioned for carrying out coding training to the Burmese after term vector expression using two-way LSTM
The feature for taking Burmese to segment carries out the mark of word segmentation to global Burmese word using CRF, and training Burmese participle model simultaneously obtains
Optimal models are segmented to Burmese；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is carried out
Prediction obtains globally optimal solution prediction output Burmese word segmentation result；
Burmese part-of-speech tagging module, for being carried out using two-way LSTM to the Burmese after Burmese word segmentation module participle
Coding training, extracts the feature of Burmese part-of-speech tagging, carries out part of speech label to global Burmese word using CRF, training is remote
Pasture words and phrases markup model simultaneously obtains Burmese part of speech label optimal models；It is marked using the good part of speech of Burmese pre-training optimal
Model predicts the Burmese sentence of input, obtains globally optimal solution prediction output Burmese part-of-speech tagging result；
Burmese sentence participle, part-of-speech tagging module are predicted, for by calling trained participle or part-of-speech tagging
Model segments the Burmese sentence after the vectorization of input, part of speech label prediction.
Above in conjunction with attached drawing, the embodiment of the present invention is explained in detail, but the present invention is not limited to above-mentioned
Embodiment within the knowledge of a person skilled in the art can also be before not departing from present inventive concept
Put that various changes can be made.
Claims (10)
1. Burmese participle and part-of-speech tagging method based on two-way LSTM-CRF, it is characterised in that:
Specific step is as follows for the Burmese participle and part-of-speech tagging method based on two-way LSTM-CRF:
Step1, Burmese pretreatment carry out term vector characterization: being believed using the syllable in BERT combination Burmese word context
Breath, generating the term vector based on syllable splitting indicates；
Step2, coding training is carried out to the Burmese after term vector expression using two-way LSTM, extracts the spy of Burmese participle
It levies, the mark of word segmentation is carried out to global Burmese word using CRF, train Burmese participle model and obtain Burmese participle and is optimal
Model；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is predicted, global optimum is obtained
Solution prediction output Burmese word segmentation result；
Step3, coding training is carried out to the Burmese after above-mentioned participle using two-way LSTM, extracts the spy of Burmese part-of-speech tagging
Sign carries out part of speech label to global Burmese word using CRF, and training Burmese part of speech markup model simultaneously obtains Burmese part of speech
Mark optimal models；Optimal models are marked using the good part of speech of Burmese pre-training, the Burmese sentence of input is predicted,
Obtain globally optimal solution prediction output Burmese part-of-speech tagging result.
Step4, by call it is trained participle or part-of-speech tagging model, to the Burmese sentence after the vectorization of input into
Row participle, part of speech label prediction.
2. the Burmese participle and part-of-speech tagging method, feature according to claim 1 based on two-way LSTM-CRF exists
In: the specific steps of the step Step1 are as follows:
Step1.1, a point Burmese sentence for good word is got by Asian language treebank website；
Step1.2, the word of the Burmese sentence of good word will be divided according to syllable splitting word using Burmese syllable splitting tool；
Step1.3, it is indicated using BERT pre-training Burmese based on the term vector of syllable splitting.
3. the Burmese participle and part-of-speech tagging method, feature according to claim 2 based on two-way LSTM-CRF exists
In: the specific steps of the step Step1.3 are as follows:
Step1.3.1,20106 Burmese sentences progress duplicate removal of acquisition is cut into 70002 Burmese words, passes through tune
With the multilingual model of BERT, the Burmese term vector for directly generating 768 dimensions with contextual information is indicated；
Step1.3.2, again by after syllable splitting Burmese word generate term vector indicate；
Step1.3.3, finally, the two is spliced, obtaining, there is the term vector of Burmese syllable information to characterize.
4. the Burmese participle and part-of-speech tagging method, feature according to claim 1 based on two-way LSTM-CRF exists
In: the specific steps of the step Step2 are as follows:
Step2.1, Burmese is extracted using two-way LSTM based on the Burmese participle feature that syllable vector generates, is obtained
Complete two-way LSTM is indicated；
Step2.2, the mark of word segmentation is carried out to global Burmese word using CRF, training Burmese participle model simultaneously obtains Burma
Language segments optimal models；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is predicted, is obtained
Output Burmese word segmentation result is predicted to globally optimal solution.
5. the Burmese participle and part-of-speech tagging method, feature according to claim 4 based on two-way LSTM-CRF exists
In: the specific steps of the step Step2.1 are as follows:
Step2.1.1, as unit of sentence, a Burmese sentence containing i word is subjected to vectorization expression and is denoted as: X
=(x1,x2,x3,x4,...,)；
The hidden status switch of Step2.1.2, forward direction LSTM outputWith reversed LSTM'sAt various locations
The hidden state of output carries out opsition dependent and splices to obtain complete hidden status switch: ht=[ht；ht]∈Rn×m, obtain one it is complete
The two-way LSTM of whole Burmese segmentation sequence is indicated.
6. the Burmese participle and part-of-speech tagging method, feature according to claim 4 based on two-way LSTM-CRF exists
In: the specific steps of the step Step2.2 are as follows:
Step2.2.1、AijIndicate the transfer score of the label from the i-th to j-th；Represent yiThe label of a sentence
tagyiIt is transferred to yi+1The label tagy of a sentencei+1A possibility that marking, length is equal to the label sequence of sentence length
Arrange y=(y1,y2,...,yn), then marking of the participle model for the label of sentence x equal to y is s (X, y)Participle,Table
Show from the participle probability after n word prediction:
Step2.2.2, normalized using Softmax after participle probability P (y | X)Participle:
Wherein,Indicate the process for solving logarithm in participle model to each Burmese word；
Step2.2.3, Burmese participle model training when by maximize log-likelihood function, obtain a training sample (x,
yx) participle loss function log (p (y | x))ParticipleIt is as follows:
Wherein,Expression is worked asWhen participle logarithm；Indicate y-th of sentence for belonging to Yx,
X indicates that Burmese word, Y indicate that Burmese sentence, Yx indicate that multiple sentences have the set of Burmese word utterances in Yx；
Step2.2.4, Burmese participle model are asked when decoding using the Viterbi algorithm of Dynamic Programming in prediction process
Solve optimal participle prediction result y*Participle:
Wherein,Expression is worked asWhen, the maximum value of score.
7. the Burmese participle and part-of-speech tagging method, feature according to claim 1 based on two-way LSTM-CRF exists
In: the specific steps of the step Step3 are as follows:
Step3.1, Burmese is extracted based on the Burmese part-of-speech tagging feature that syllable vector generates using two-way LSTM,
Obtaining complete two-way LSTM indicates；
Step3.2, part of speech label is carried out to global Burmese word using CRF, training Burmese part of speech markup model simultaneously obtains
Burmese part of speech marks optimal models；Optimal models are marked using the good part of speech of Burmese pre-training, to Burma's sentence of input
Son is predicted, globally optimal solution prediction output Burmese part-of-speech tagging result is obtained.
8. the Burmese participle and part-of-speech tagging method, feature according to claim 7 based on two-way LSTM-CRF exists
In: the specific steps of the step Step3.1 are as follows:
Step3.1.1, as unit of sentence, the Burmese sentence of a point good word containing i word is subjected to vectorization table
Show and be denoted as: XBurmese after participle=(x1, x2, x3, x4...)；
The hidden status switch of Step3.1.2, forward direction LSTM outputWith reversed LSTM'sEach
The hidden state of position output carries out opsition dependent and splices to obtain complete hidden status switch: ht=[ht；ht]∈Rn×m, obtain one
The two-way LSTM of a complete Burmese part of speech flag sequence is indicated.
9. the Burmese participle and part-of-speech tagging method, feature according to claim 7 based on two-way LSTM-CRF exists
In: the specific steps of the step Step3.2 are as follows:
Step3.2.1、AijIndicate the transfer score of the label from the i-th to j-th；Represent yiThe label tagy of a sentencei
It is transferred to yi+1The label tagy of a sentencei+1A possibility that marking, length is equal to the sequence label y=of sentence length
(y1,y2,...,yn), then marking of the part of speech markup model for the label of sentence x equal to y is s (X, y)Part of speech label,Indicate the part of speech marking probability after predicting from n word:
Step3.2.2, normalized using Softmax after part of speech marking probability be P (y | X)Part of speech label:
Indicate the process for solving logarithm in part of speech markup model to each Burmese word；
By maximizing log-likelihood function when Step3.2.3, the training of Burmese part of speech markup model, a training sample is obtained
(x,yx) part of speech label loss function log (p (y | x))Part of speech label:
Wherein,Expression is worked asWhen part of speech label logarithm；Indicate y-th of sentence for belonging to Yx
Son, x indicates that Burmese word, Y indicate that Burmese sentence, Yx indicate that multiple sentences have the set of Burmese word utterances in Yx；
Step3.2.4, Burmese part of speech markup model use the Viterbi algorithm of Dynamic Programming in prediction process when decoding
To solve optimal part of speech label prediction result y*Part of speech label:
Wherein,Expression is worked asWhen, the maximum value of score.
10. Burmese participle and part-of-speech tagging device based on two-way LSTM-CRF, it is characterised in that: including following module:
Burmese preprocessing module, for generating and being based on sound using the syllable information in BERT combination Burmese word context
The term vector for saving feature cutting indicates；
Burmese word segmentation module is extracted remote for carrying out coding training to the Burmese after term vector expression using two-way LSTM
The feature of pasture language participle carries out the mark of word segmentation to global Burmese word using CRF, and training Burmese participle model simultaneously obtains Burma
Pasture language segments optimal models；Using the good participle optimal models of Burmese pre-training, the Burmese sentence of input is predicted,
Obtain globally optimal solution prediction output Burmese word segmentation result；
Burmese part-of-speech tagging module, for being encoded using two-way LSTM to the Burmese after Burmese word segmentation module participle
The feature of Burmese part-of-speech tagging is extracted in training, carries out part of speech label, training Burmese to global Burmese word using CRF
Part of speech markup model simultaneously obtains Burmese part of speech label optimal models；Optimal mould is marked using the good part of speech of Burmese pre-training
Type predicts the Burmese sentence of input, obtains globally optimal solution prediction output Burmese part-of-speech tagging result；
It predicts Burmese sentence participle, part-of-speech tagging module, calls trained participle or part-of-speech tagging model for passing through,
Burmese sentence after the vectorization of input is segmented, part of speech label prediction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910739718.5A CN110489750A (en) | 2019-08-12 | 2019-08-12 | Burmese participle and part-of-speech tagging method and device based on two-way LSTM-CRF |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910739718.5A CN110489750A (en) | 2019-08-12 | 2019-08-12 | Burmese participle and part-of-speech tagging method and device based on two-way LSTM-CRF |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110489750A true CN110489750A (en) | 2019-11-22 |
Family
ID=68550763
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910739718.5A Pending CN110489750A (en) | 2019-08-12 | 2019-08-12 | Burmese participle and part-of-speech tagging method and device based on two-way LSTM-CRF |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110489750A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781254A (en) * | 2020-01-02 | 2020-02-11 | 四川大学 | Automatic case knowledge graph construction method, system, equipment and medium |
| CN111046946A (en) * | 2019-12-10 | 2020-04-21 | 昆明理工大学 | Burma language image text recognition method based on CRNN |
| CN111797356A (en) * | 2020-07-06 | 2020-10-20 | 上海冰鉴信息科技有限公司 | Webpage table information extraction method and device |
| CN112149417A (en) * | 2020-09-16 | 2020-12-29 | 北京小米松果电子有限公司 | Part-of-speech tagging method and device, storage medium and electronic equipment |
| CN112183086A (en) * | 2020-09-23 | 2021-01-05 | 北京先声智能科技有限公司 | English pronunciation continuous reading mark model based on sense group labeling |
| CN112287688A (en) * | 2020-09-17 | 2021-01-29 | 昆明理工大学 | English-Burmese bilingual parallel sentence pair extraction method and device integrating pre-training language model and structural features |
| CN112364654A (en) * | 2020-11-11 | 2021-02-12 | 安徽工业大学 | Education-field-oriented entity and relation combined extraction method |
| CN112634878A (en) * | 2020-12-15 | 2021-04-09 | 深港产学研基地（北京大学香港科技大学深圳研修院） | Speech recognition post-processing method and system and related equipment |
| CN113112007A (en) * | 2021-06-11 | 2021-07-13 | 平安科技（深圳）有限公司 | Method, device and equipment for selecting sequence length in neural network and storage medium |
| WO2021143206A1 (en) * | 2020-07-16 | 2021-07-22 | 平安科技（深圳）有限公司 | Single-statement natural language processing method and apparatus, computer device, and readable storage medium |
| CN113239692A (en) * | 2021-05-12 | 2021-08-10 | 同方知网数字出版技术股份有限公司 | Ancient Chinese-based word segmentation method |
| CN113901210A (en) * | 2021-09-15 | 2022-01-07 | 昆明理工大学 | Method for marking verbosity of Thai and Burma characters by using local multi-head attention to mechanism fused word-syllable pair |
| US11514321B1 (en) * | 2020-06-12 | 2022-11-29 | Amazon Technologies, Inc. | Artificial intelligence system using unsupervised transfer learning for intra-cluster analysis |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016030730A1 (en) * | 2014-08-29 | 2016-03-03 | Yandex Europe Ag | Method for text processing |
| CN108763212A (en) * | 2018-05-23 | 2018-11-06 | 北京神州泰岳软件股份有限公司 | A kind of address information extraction method and device |
| CN109117472A (en) * | 2018-11-12 | 2019-01-01 | 新疆大学 | A kind of Uighur name entity recognition method based on deep learning |
| CN109558569A (en) * | 2018-12-14 | 2019-04-02 | 昆明理工大学 | A kind of Laotian part-of-speech tagging method based on BiLSTM+CRF model |
| CN109960782A (en) * | 2018-12-27 | 2019-07-02 | 同济大学 | A kind of Tibetan language segmenting method and device based on deep neural network |
| CN110032648A (en) * | 2019-03-19 | 2019-07-19 | 微医云(杭州)控股有限公司 | A kind of case history structuring analytic method based on medical domain entity |
-
2019
- 2019-08-12 CN CN201910739718.5A patent/CN110489750A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016030730A1 (en) * | 2014-08-29 | 2016-03-03 | Yandex Europe Ag | Method for text processing |
| CN108763212A (en) * | 2018-05-23 | 2018-11-06 | 北京神州泰岳软件股份有限公司 | A kind of address information extraction method and device |
| CN109117472A (en) * | 2018-11-12 | 2019-01-01 | 新疆大学 | A kind of Uighur name entity recognition method based on deep learning |
| CN109558569A (en) * | 2018-12-14 | 2019-04-02 | 昆明理工大学 | A kind of Laotian part-of-speech tagging method based on BiLSTM+CRF model |
| CN109960782A (en) * | 2018-12-27 | 2019-07-02 | 同济大学 | A kind of Tibetan language segmenting method and device based on deep neural network |
| CN110032648A (en) * | 2019-03-19 | 2019-07-19 | 微医云(杭州)控股有限公司 | A kind of case history structuring analytic method based on medical domain entity |
Non-Patent Citations (1)
| Title |
|---|
| 潘华山等: "基于层叠条件随机场的高棉语分词及词性标注方法", 《中文信息学报》 * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111046946A (en) * | 2019-12-10 | 2020-04-21 | 昆明理工大学 | Burma language image text recognition method based on CRNN |
| CN111046946B (en) * | 2019-12-10 | 2021-03-02 | 昆明理工大学 | Burma language image text recognition method based on CRNN |
| CN110781254A (en) * | 2020-01-02 | 2020-02-11 | 四川大学 | Automatic case knowledge graph construction method, system, equipment and medium |
| US11514321B1 (en) * | 2020-06-12 | 2022-11-29 | Amazon Technologies, Inc. | Artificial intelligence system using unsupervised transfer learning for intra-cluster analysis |
| CN111797356A (en) * | 2020-07-06 | 2020-10-20 | 上海冰鉴信息科技有限公司 | Webpage table information extraction method and device |
| CN111797356B (en) * | 2020-07-06 | 2023-08-08 | 上海冰鉴信息科技有限公司 | Webpage form information extraction method and device |
| WO2021143206A1 (en) * | 2020-07-16 | 2021-07-22 | 平安科技（深圳）有限公司 | Single-statement natural language processing method and apparatus, computer device, and readable storage medium |
| CN112149417A (en) * | 2020-09-16 | 2020-12-29 | 北京小米松果电子有限公司 | Part-of-speech tagging method and device, storage medium and electronic equipment |
| CN112287688B (en) * | 2020-09-17 | 2022-02-11 | 昆明理工大学 | English-Burmese bilingual parallel sentence pair extraction method and device integrating pre-training language model and structural features |
| CN112287688A (en) * | 2020-09-17 | 2021-01-29 | 昆明理工大学 | English-Burmese bilingual parallel sentence pair extraction method and device integrating pre-training language model and structural features |
| CN112183086A (en) * | 2020-09-23 | 2021-01-05 | 北京先声智能科技有限公司 | English pronunciation continuous reading mark model based on sense group labeling |
| CN112364654A (en) * | 2020-11-11 | 2021-02-12 | 安徽工业大学 | Education-field-oriented entity and relation combined extraction method |
| CN112634878A (en) * | 2020-12-15 | 2021-04-09 | 深港产学研基地（北京大学香港科技大学深圳研修院） | Speech recognition post-processing method and system and related equipment |
| CN112634878B (en) * | 2020-12-15 | 2024-05-17 | 深港产学研基地（北京大学香港科技大学深圳研修院） | Speech recognition post-processing method and system and related equipment |
| CN113239692A (en) * | 2021-05-12 | 2021-08-10 | 同方知网数字出版技术股份有限公司 | Ancient Chinese-based word segmentation method |
| CN113112007B (en) * | 2021-06-11 | 2021-10-15 | 平安科技（深圳）有限公司 | Method, device and equipment for selecting sequence length in neural network and storage medium |
| CN113112007A (en) * | 2021-06-11 | 2021-07-13 | 平安科技（深圳）有限公司 | Method, device and equipment for selecting sequence length in neural network and storage medium |
| CN113901210A (en) * | 2021-09-15 | 2022-01-07 | 昆明理工大学 | Method for marking verbosity of Thai and Burma characters by using local multi-head attention to mechanism fused word-syllable pair |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110489750A (en) | Burmese participle and part-of-speech tagging method and device based on two-way LSTM-CRF | |
| CN108363790B (en) | Method, device, equipment and storage medium for evaluating comments | |
| CN110427463B (en) | Search statement response method and device, server and storage medium | |
| CN109344236B (en) | Problem similarity calculation method based on multiple characteristics | |
| CN110704621B (en) | Text processing method and device, storage medium and electronic equipment | |
| CN107330011A (en) | The recognition methods of the name entity of many strategy fusions and device | |
| CN110532557B (en) | Unsupervised text similarity calculation method | |
| CN110929030A (en) | Text abstract and emotion classification combined training method | |
| CN111767741A (en) | Text emotion analysis method based on deep learning and TFIDF algorithm | |
| CN113505200B (en) | Sentence-level Chinese event detection method combined with document key information | |
| CN111291188B (en) | Intelligent information extraction method and system | |
| US20230069935A1 (en) | Dialog system answering method based on sentence paraphrase recognition | |
| CN112883732A (en) | Method and device for identifying Chinese fine-grained named entities based on associative memory network | |
| CN108038099B (en) | Low-frequency keyword identification method based on word clustering | |
| CN110196982A (en) | Hyponymy abstracting method, device and computer equipment | |
| CN111738007A (en) | Chinese named entity identification data enhancement algorithm based on sequence generation countermeasure network | |
| CN110287298A (en) | A kind of automatic question answering answer selection method based on question sentence theme | |
| CN110442880B (en) | Translation method, device and storage medium for machine translation | |
| CN114153971B (en) | Error correction recognition and classification equipment for Chinese text containing errors | |
| CN111666376B (en) | Answer generation method and device based on paragraph boundary scan prediction and word shift distance cluster matching | |
| CN110334186A (en) | Data query method, apparatus, computer equipment and computer readable storage medium | |
| CN114676255A (en) | Text processing method, device, equipment, storage medium and computer program product | |
| CN112699686A (en) | Semantic understanding method, device, equipment and medium based on task type dialog system | |
| CN114818717A (en) | Chinese named entity recognition method and system fusing vocabulary and syntax information | |
| CN112417132A (en) | New intention recognition method for screening negative samples by utilizing predicate guest information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191122 |
|
| RJ01 | Rejection of invention patent application after publication |