CN108717434A - A kind of text sort method of the point-by-point tactful and pairs of strategy of mixing - Google Patents
A kind of text sort method of the point-by-point tactful and pairs of strategy of mixing Download PDFInfo
- Publication number
- CN108717434A CN108717434A CN201810460253.5A CN201810460253A CN108717434A CN 108717434 A CN108717434 A CN 108717434A CN 201810460253 A CN201810460253 A CN 201810460253A CN 108717434 A CN108717434 A CN 108717434A
- Authority
- CN
- China
- Prior art keywords
- sample
- point
- score
- node
- vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000007246 mechanism Effects 0.000 claims abstract description 18
- 239000013598 vector Substances 0.000 claims description 76
- 238000013528 artificial neural network Methods 0.000 claims description 17
- 230000007787 long-term memory Effects 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 9
- 230000000306 recurrent effect Effects 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 3
- 238000011161 development Methods 0.000 claims description 3
- 230000001537 neural effect Effects 0.000 claims 1
- 238000012549 training Methods 0.000 description 12
- 230000009467 reduction Effects 0.000 description 4
- 241000251468 Actinopterygii Species 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 239000013604 expression vector Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000003042 antagnostic effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000015654 memory Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Landscapes
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of text sort methods of the point-by-point tactful and pairs of strategy of mixing, are related to one kind to sentence, and syntax tree equal samples preferably utilize the sort method of the different information between sample during sorting, including：Point-by-point phase sorting, pairs of phase sorting.Point-by-point phase sorting gives a mark to sample and does the first minor sort, and filter out the higher candidate of score, on the basis of these candidates, pairs of phase sorting further is carried out to it, wherein tactful sort uses a kind of encode, with reference to the flow for generating, comparing, re-encode, giving a mark in pairs, a kind of node method of weighting based on span is being devised with reference to during, a kind of method that span and attention mechanism are combined, finally according to the basic score of sample and point-by-point sequence score, pairs of phase sorting score carries out integrated ordered.
Description
Technical field
The invention belongs to sorting by computer learning areas, are related to a kind of text sequence of the point-by-point tactful and pairs of strategy of mixing
Method.
Background technology
Sequence study be in handling ordered problem using using machine learning method come the method for training pattern.Sequence is learned
Habit can be applied in information retrieval, natural language processing, data mining etc..The research of sequence study is always information retrieval
The hot and difficult issue in field.
Strategy during learning tasks are divided into three by sequence study:Point-by-point strategy (pointwise) is tactful in pairs
(pairwise), list policy (listwise).In point-by-point strategy, sequencing problem is converted into classification, recurrence or sequence
Classification problem.Therefore, the structuring of sequence can be ignored in this fashion.In pairs of strategy, sample is formed into sample pair,
Single document is not considered, but considers the relationship between sample pair.Point-by-point method is by each sample inside training set as training
Example, control methods is by sample to as training example.
In machine translation, syntactic analysis task, translation result and syntax tree are the result is that by searching decisions step by step
It obtains, generally use column searching method decodes, and in Candidate Set after search, often there is the prediction result than model
Better result.It is a promising optimization direction that better sample is selected in last Candidate Set, not with Optimized model
It is same, it waits for being many complete samples in Candidate Set, model can preferably arrange sample according to its global information
Sequence, and it is not limited solely to local feature.
Invention content
Goal of the invention：The present invention provides a kind of text sort method of the point-by-point tactful and pairs of strategy of mixing.Point-by-point strategy
Can be given a mark for it according to the structure feature of sample itself, it can model the structure of single sample well, into strategy then more
Difference and contact between two samples are paid attention in, by the advantages of both combining, to improve sorting unit performance.
The invention discloses a kind of text sort methods of the point-by-point tactful and pairs of strategy of mixing, include the following steps：
Step 1 is ranked up sentence or syntax tree, and node wherein included is respectively word or tree node, sample when initial
Originally there is a basic score, this score comes from original model.Sample is ranked up using point-by-point order models, in point-by-point plan
In the result slightly to have sorted, the score of point-by-point phase sorting is obtained, the higher sample of K score is chosen and forms Candidate Set, use
Pairs of order models carry out marking sequence to Candidate Set；
Step 2, strategy sequence in pairs is carried out to the Candidate Set of K sample composition, obtains sample in pairs of phase sorting
Score；
Step 3, the score, the score of pairs of phase sorting of comprehensive point-by-point phase sorting and the basic score of sample are to sample
It is weighted sequence.
Step 1 includes：
Step 1-1 encodes sample using encoder, and the continuous type for obtaining each node indicates, i.e., by sample set
Close x={ x1,…,xi,…,xn, be converted to h={ h1,…,hi,…,hnHidden layer expression, xnIndicate n-th of section in sample set
Point, node are also known as unit, hnIt indicates to obtain x by codingnHidden layer indicate；The encoder is structure recurrent neural network
Recursive Neural Network or tree-like shot and long term memory network Tree-LSTM or two-way shot and long term memory networks Bi-
LSTM。
Hidden layer is indicated that carrying out linear transformation obtains the score of each unit in sample, by obtaining for all units by step 1-2
The score S for dividing summation just to obtain samplepointwise：
Wherein V1For model parameter, for node to be indicated to be converted to by linear transformation in point phase sorting one by one
Point；
The score S of the point-by-point phase sorting of sample is calculated in step 1-31：
S1=α * Spointwise+(1-α)*Sbase,
Wherein, α indicates that hyper parameter, this hyper parameter are obtained by being adjusted in development set data, SbaseFor the basis of sample
Score, basic score is from the baseline system for generating sample；
Step 1-4 is ranked up sample by the obtained scores of step 1-3, chooses the higher sample composition of K score
Candidate Set, K are hyper parameter, are generally set to 8, and K sample is combined into sample pair, altogether K* (K-1)/2 sample pair.
Step 2 includes：
Step 2-1, sample A and sample B in K sample form sample pair<A, B>, using encoder by sample pair<A, B
>In nodes encoding at hidden layer vector indicate；
Step 2-2 is compared to obtain the reference vector of node in sample pair；
Step 2-3 obtains the new vector expression after comparison according to the hidden layer vector sum of node with reference to vector；
Step 2-4 is that the vector that step 2-3 is obtained indicates to be encoded by encoder；It is equivalent to and makees neural network
It is encoded again on the expression vector after two samples for comparing so that network can be captured and be concluded between two samples
Different information.
Step 2-5, obtains sample pair<A, B>Relative score, all samples pair with respect to giving a mark in comprehensive K sample
To sample pairs of phase sorting score.
In step 2-1, the encoder is structure recurrent neural network Recursive Neural Network or tree-like
Shot and long term memory network Tree-LSTM or two-way shot and long term memory networks Bi-LSTM.
Step 2-2 includes：Based on attention mechanism (Attention Mechanism), sample A is calculated by following formula
In node aiReference vector
Wherein, n indicates the number of sample A nodes,
bjIndicate j-th of node in sample B, eijIndicate node aiAnd bjThe degree of correlation.
The node a in sample A can also be calculated in step 2-2 by following formulaiReference vector
Wherein span () function representation obtains the range intervals of the leaf node included by node x, len () function representation
The leaf node number included by node is obtained, b indicates to be included in section span (a in sample Bi) section some node, should
Method be the node method of weighting based on span, suitable for reordering for syntax tree, refer in syntax tree, can be according to each
The span of internal node, that is, the leaf node that includes range, to establish the antagonistic relations between two samples, for sample
This is right<A, B>, A interior joints a's indicates weighting by all nodes in the range of the span of a includes in B with reference to vector and obtains.
In step 2-2, the node a in sample A can also be calculated by following formulaiReference vector
Wherein span () function representation obtains the range intervals of the leaf node included by node, and b indicates section in sample B
span(ai) some node for including, e indicates node aiWith the degree of correlation of b.This method is that span and attention mechanism are combined
Method, it is syntax tree to be equally also only applicable to sample, refers to for sample pair<A, B>, the use of attention mechanism is being A interior joints
aiWhen generation with reference to vector, the range that attention mechanism is paid close attention to is limited to aiSpan section in avoid incoherent letter
Breath.
Step 2-3 includes：New vector is obtained by following formula to indicate
Or new vector is obtained using following formula and is indicated
Wherein g1、g2The output vector for indicating two door units respectively, for controlling information flow, ⊙ indicates two together
The each dimension of vector of dimension is multiplied, wherein W1And b1,W2And b1The parameter of respectively two doors, σ are activation primitive,
All parameters in middle model are all obtained by stochastic gradient descent SGD and training, and training objective usually uses largest interval knot
Structureization predicts training objective (bibliography：Socher,Richard,et al."Parsing with Compositional
Vector Grammars."Meeting of the Association for Computational Linguistics
2013:455-465.)。
Step 2-4 includes：Step 2-3 mono- is obtained n vector and indicates, is denoted as For n-th to
Amount indicates that l values are 1~n, and the vector that step 2-3 is obtained is indicatedIt is encoded again using encoder
Obtain new vector m ..., mi,…,mn, mnIt indicatesNew vector after coding indicates.
Step 2-5 includes：The coding result of step 2-4 is subjected to linear transformation and obtains the score of each node in sample,
The score summation of all nodes has just been obtained to a relative score of sampleWherein V2Join for model
Number obtains node marking, m for doing linear transformation to hidden layer expression in pairs of phase sortingiFor with reference to vectorIt encodes again
The vector obtained afterwards indicates, the summation of the relative score of A and every other sample then can be obtained sample A in sequence rank in pairs
The score of section, the score for the pairs of phase sorting of each sample are denoted as Spairwise。
Step 3 includes：Score, the score of pairs of phase sorting and the basic score of sample of comprehensive point-by-point phase sorting,
Mode is weighted sequence in the following way, and by the final ranking results S of the tree of highest scoring2Output, wherein α, β are super ginseng
Number, by being adjusted on fraction data set and optimal result being taken to obtain：
S2=α * Spointwise+β*Spairwise+(1-α-*β)*Sbase。
The present invention in resetting sequence task by the way that point-by-point tactful and pairs of strategy is combined, using neural network as
Encoder carries out the ability to express that coding improves model to sample with tree-like length memory network in tree construction sample, leads to
Cross first layer sequence reduction Candidate Set number so that pairs of ordering strategy can be used effectively, in second layer sequencer procedure,
In conjunction with attention mechanism so that the contact and difference that model can preferably between modeling sample, this method can effectively improve
Sort performance.
Advantageous effect：The present invention uses point-by-point order models first by combining point-by-point tactful and pairs of strategy
It is given a mark to sample and does the first minor sort, and filter out the higher candidate of score, it is further right on the basis of these candidates
Its pairs of phase sorting of progress is comparing wherein pairs of tactful sequence uses a kind of flow for encoding, comparing, encode, giving a mark
Different control methods, the sequence after the comparison marking summation of output as this stage are chosen according to the characteristics of sample in the process
Score, finally according to the basic score of sample and point-by-point sequence score, pairs of phase sorting score carries out integrated ordered.The method
The advantage for combining point-by-point tactful and pairs of strategy, has preferably modeled the difference between two samples, the spy of sample is utilized
Point improves the performance of sorting unit.
Description of the drawings
The present invention is done with reference to the accompanying drawings and detailed description and is further illustrated, it is of the invention above-mentioned or
Otherwise advantage will become apparent.
Fig. 1 is the flow chart of the present invention.
Fig. 2 is the structural schematic diagram of order models tactful in pairs of the invention
Fig. 3 is the exemplary plot of ranked object syntax tree of the present invention
Fig. 4 is method of the present invention with reference to generation phase combination attention mechanism and node span.
Fig. 5 is to generate then flow chart that Candidate Set is ranked up to sentence I do like eating fish.
Specific implementation mode
In order to enable those skilled in the art to better understand the solution of the present invention, with reference to the accompanying drawings and embodiments to this hair
It is bright to be described further.Obviously, described embodiments are only a part of the embodiments of the present invention, rather than whole implementation
Example.Based on the embodiments of the present invention, obtained by those of ordinary skill in the art without making creative efforts
Every other embodiment, shall fall within the protection scope of the present invention.
The invention discloses a kind of text sort methods of the point-by-point tactful and pairs of strategy of mixing, include the following steps：
Step 1 is ranked up sentence or syntax tree, and node wherein included is respectively word or tree node, sample when initial
Originally there is a basic score, this score comes from original model.Sample is ranked up using point-by-point order models, in point-by-point plan
In the result slightly to have sorted, the score of point-by-point phase sorting is obtained, the higher sample of K score is chosen and forms Candidate Set, use
Pairs of order models carry out marking sequence to Candidate Set；
Step 2, strategy sequence in pairs is carried out to the Candidate Set of K sample composition, obtains sample in pairs of phase sorting
Score；
Step 3, the score, the score of pairs of phase sorting of comprehensive point-by-point phase sorting and the basic score of sample are to sample
It is weighted sequence.
Step 1 includes：
Step 1 is ranked up sentence or syntax tree, and node wherein included is respectively word or tree node, sample when initial
Originally there is a basic score, this score comes from original model.Sample is ranked up using point-by-point order models, in point-by-point plan
In the result slightly to have sorted, the score of point-by-point phase sorting is obtained, the higher sample of K score is chosen and forms Candidate Set, use
Pairs of order models carry out marking sequence to Candidate Set；
Step 2, strategy sequence in pairs is carried out to the Candidate Set of K sample composition, obtains sample in pairs of phase sorting
Score；
Step 3, the score, the score of pairs of phase sorting of comprehensive point-by-point phase sorting and the basic score of sample are to sample
It is weighted sequence.
Step 1 includes：
Step 1-1 encodes sample using encoder, and the continuous type for obtaining each node indicates, i.e., by sample set
Close x={ x1,…,xi,…,xn, be converted to h={ h1,…,hi,…,hnHidden layer expression, xnIndicate n-th of section in sample set
Point, node are also known as unit, hnIt indicates to obtain x by codingnHidden layer indicate；The encoder is structure recurrent neural network
Recursive Neural Network or tree-like shot and long term memory network Tree-LSTM or two-way shot and long term memory networks Bi-
LSTM。
Hidden layer is indicated that carrying out linear transformation obtains the score of each unit in sample, by obtaining for all units by step 1-2
The score S for dividing summation just to obtain samplepointwise：
Wherein V1For model parameter, for node to be indicated to be converted to by linear transformation in point phase sorting one by one
Point；
The score S of the point-by-point phase sorting of sample is calculated in step 1-31：
S1=α * Spointwise+(1-α)*Sbase,
Wherein, α indicates that hyper parameter, this hyper parameter are obtained by being adjusted in development set data, SbaseFor the basis of sample
Score, basic score is from the baseline system for generating sample；
Step 1-4 is ranked up sample by the obtained scores of step 1-3, chooses the higher sample composition of K score
Candidate Set, K are hyper parameter, are generally set to 8, and K sample is combined into sample pair, altogether K* (K-1)/2 sample pair.
Step 2 includes：
Step 2-1, sample A and sample B in K sample form sample pair<A, B>, using encoder by sample pair<A, B
>In nodes encoding at hidden layer vector indicate；
Step 2-2 is compared to obtain the reference vector of node in sample pair；
Step 2-3 obtains the new vector expression after comparison according to the hidden layer vector sum of node with reference to vector；
Step 2-4 is that the vector that step 2-3 is obtained indicates to be encoded by encoder；It is equivalent to and makees neural network
It is encoded again on the expression vector after two samples for comparing so that network can be captured and be concluded between two samples
Different information.
Step 2-5, obtains sample pair<A, B>Relative score, all samples pair with respect to giving a mark in comprehensive K sample
To sample pairs of phase sorting score.
In step 2-1, the encoder is structure recurrent neural network Recursive Neural Network or tree-like
Shot and long term memory network Tree-LSTM or two-way shot and long term memory networks Bi-LSTM.
Step 2-2 includes：Based on attention mechanism (Attention Mechanism), sample A is calculated by following formula
In node aiReference vector
Wherein, n indicates the number of sample A nodes,
bjIndicate j-th of node in sample B, eijIndicate node aiAnd bjThe degree of correlation.
The node a in sample A can also be calculated in step 2-2 by following formulaiReference vector
Wherein span () function representation obtains the range intervals of the leaf node included by node x, len () function representation
The leaf node number included by node is obtained, b indicates to be included in section span (a in sample Bi) section some node, should
Method be the node method of weighting based on span, suitable for reordering for syntax tree, refer in syntax tree, can be according to each
The span of internal node, that is, the leaf node that includes range, to establish the antagonistic relations between two samples, for sample
This is right<A, B>, A interior joints a's indicates weighting by all nodes in the range of the span of a includes in B with reference to vector and obtains.
In step 2-2, the node a in sample A can also be calculated by following formulaiReference vector
Wherein span () function representation obtains the range intervals of the leaf node included by node, and b indicates section in sample B
span(ai) some node for including, e indicates node aiWith the degree of correlation of b.This method is that span and attention mechanism are combined
Method, it is syntax tree to be equally also only applicable to sample, refers to for sample pair<A, B>, the use of attention mechanism is being A interior joints
aiWhen generation with reference to vector, the range that attention mechanism is paid close attention to is limited to aiSpan section in avoid incoherent letter
Breath.
Step 2-3 includes：New vector is obtained by following formula to indicate
Or new vector is obtained using following formula and is indicated
Wherein g1、g2The output vector for indicating two door units respectively, for controlling information flow, ⊙ indicates two together
The each dimension of vector of dimension is multiplied, wherein W1And b1,W2And b1The parameter of respectively two doors, σ are activation primitive,
All parameters in middle model are all obtained by stochastic gradient descent SGD and training, and training objective usually uses largest interval knot
Structureization predicts training objective (bibliography：Socher,Richard,et al."Parsing with Compositional
Vector Grammars."Meeting of the Association for Computational Linguistics
2013:455-465.)。
Step 2-4 includes：Step 2-3 mono- is obtained n vector and indicates, is denoted as For n-th to
Amount indicates that l values are 1~n, and the vector that step 2-3 is obtained is indicatedIt is encoded again using encoder
Obtain new vector m ..., mi,…,mn, mnIt indicatesNew vector after coding indicates.
Step 2-5 includes：The coding result of step 2-4 is subjected to linear transformation and obtains the score of each node in sample,
The score summation of all nodes has just been obtained to a relative score of sampleWherein V2Join for model
Number obtains node marking, m for doing linear transformation to hidden layer expression in pairs of phase sortingiFor with reference to vectorIt compiles again
The vector obtained after code indicates that the relative score summation of A and every other sample then can be obtained sample A is sorting in pairs
The score in stage, the score for the pairs of phase sorting of each sample are denoted as Spairwise。
Step 3 includes：Score, the score of pairs of phase sorting and the basic score of sample of comprehensive point-by-point phase sorting,
Mode is weighted sequence in the following way, and by the final ranking results S of the tree of highest scoring2Output, wherein α, β are super ginseng
Number, by being adjusted on fraction data set and optimal result being taken to obtain：
S2=α * Spointwise+β*Spairwise+(1-α-*β)*Sbase。
Embodiment
The present invention proposes a kind of text sort method of mixing point-by-point strategy and Comparing method (as depicted in figs. 1 and 2),
Sequence is divided into two stages, the first stage carries out preliminary screening by point-by-point strategy, then again in the Candidate Set after reduction
Finer sequence is carried out by sorting in pairs, improves the performance of sorting unit.The point-by-point strategy of a kind of mixing of the present invention
The step of with the method for reordering of pairs of strategy, is as follows：
The object to sort in the present embodiment is if Fig. 3 is to syntax tree sorting, and baseline system uses Stamford syntactic analysis
Device (Stanford Parser), 50 syntax trees of each sentence generation highest scoring, parser are given in gathering for training
Marking with it is known as basic score, and each sentence has a correct syntax tree in training set.
Sample is encoded first, coding implement body uses structure recurrent neural network, calculates and is each saved in tree construction
The hidden layer vector of point indicates.Hidden layer is indicated that carrying out linear transformation obtains the score of each unit in sample, by all units
Score, which is summed, has just obtained the score of sample, i.e.,Wherein VTFor model parameter.
This 50 syntax trees are ranked up by weighting sequence, wherein basic score is Sbase, it is scored at S point by point1：
S1=α * Spointwise+(1-α)*Sbase
According to weighting sequence as a result, and take first 8 to enter in pairs of sequencer procedure, 8 samples are combined into sample
It is right, 28 samples pair altogether.
It in the pairs of sorting coding stage, is encoded using pairs of order models, this process and point-by-point Policy model coding
The process taken is the same, and only coder parameters are different.In sample pair<A, B>In be that each node is calculated with reference to vector table
Show, in as A each node calculated in B with the corresponding information of this node, be used in combination vector to indicate, same method is also made
With on B, the method being combined using span and attention mechanism, wherein aibjThe node in sample A and sample B is indicated respectively,
Calculating aiReference vector when, by go traversal sample B in it is all in aiSpan scope in all nodes, calculate it is similar
Degree, is then based on this similarity and is weighted, you can obtain aiReference vectorWherein span (ai) indicate node aiIt is wrapped
The range of the leaf node included, as shown in Figure 4, the span of grayed-out nodes is (eating fish), i.e. the 4th word in sample A
To the 5th word, so the node being comprised in this section can only be gone in sample B to calculate similarity：
It compares to obtain new vector expression with reference to vectorial by the hidden layer vector sum to node, comparative approach is used and done
Difference, the method for doing point multiplication operation, i.e., make the difference hidden layer vector sum with reference to vector and the operation of dot product and splicing obtain new vector
It indicates and is passed in next layer network structure.
Using encoder again to the vector after comparisonCoding again is carried out, is equivalent in fact and acts on neural network
It is encoded again on expression vector after comparing two samples so that network can capture and conclude the difference between two samples
Different information.
Vector after comparison is input to encoder to encode, and one group of parameter is multiplied by according to coding result and obtains node
Marking, all nodes marking summation obtain the relative score of sample, and each sample obtains B there are two score, that is, A meeting
Divide and B is to the score of A, A is to the score that the sum of relative score of other 7 samples is pairs of phase sorting A.
In the integrated ordered stage, by all steps of front, preceding 8 sample standard deviations have point-by-point order models marking and in pairs row
Sequence model give a mark and basic score, the present embodiment overall flow, as Fig. 5 description shown in, input in baseline system about sentence
50 syntax trees namely 50 samples that sub " I do like eating fish " is obtained, wherein Tree4 are correct sample,
Per one tree, all there are one basic score Sbase, it is seen that the basic score highest of wherein Tree1 is obtained by point-by-point order models
To point-by-point marking Spointwise, both scores are weighted to obtain a ranking results, and take before highest scoring 4, be carried out pairs of
Sequence obtains Spairwisee, finally according to the weighting of these three scores, the highest tree of weight score is selected, is exported as a result, managed
In the case of thinking, the not high correct sample Tree4 of script score highest position can be come.
The present invention in resetting sequence task by the way that point-by-point tactful and pairs of strategy is combined, using neural network as
Encoder carries out the ability to express that coding improves model to sample with tree-like length memory network in tree construction sample, leads to
Cross first layer sequence reduction Candidate Set number so that pairs of ordering strategy can be used effectively, in second layer sequencer procedure,
In conjunction with attention mechanism so that the contact and difference that model can preferably between modeling sample, this method can effectively improve
Sort performance.By introducing the order models of the tree-like point-by-point strategy of shot and long term memory network optimization, height is gone out by this model discrimination
Mass candidates collection reduces the calculation amount of strategy in pairs, in the pairs of tactful stage, is compared two-by-two on the Candidate Set after reduction
Compared with by using the correspondence of the intermediate node of sample so that attention mechanism is calculating and more efficiently, optimizing in accuracy
The result that reorders.
The present invention provides a kind of text sort methods of the point-by-point tactful and pairs of strategy of mixing, implement the technical side
There are many method and approach of case, the above is only a preferred embodiment of the present invention, it is noted that for the art
For those of ordinary skill, various improvements and modifications may be made without departing from the principle of the present invention, these improvement
Protection scope of the present invention is also should be regarded as with retouching.The available prior art of each component part being not known in the present embodiment is subject to
It realizes.
Claims (10)
1. a kind of text sort method of the point-by-point tactful and pairs of strategy of mixing, which is characterized in that include the following steps：
Step 1, sample is ranked up using point-by-point order models, in the result that point-by-point strategy has sorted, is arranged point by point
The score in sequence stage chooses the higher sample of K score and forms Candidate Set, and K is preset hyper parameter, uses pairs of sequence
Model carries out marking sequence to Candidate Set；
Step 2, strategy sequence in pairs is carried out to the Candidate Set of K sample composition, obtain sample pairs of phase sorting score；
Step 3, the score, the score of pairs of phase sorting of comprehensive point-by-point phase sorting and the basic score of sample carry out sample
Weighting sequence.
2. according to the method described in claim 1, it is characterized in that, step 1 includes：
Step 1-1 encodes sample using encoder, and the continuous type for obtaining each node indicates, i.e., by sample x=
{x1,…,xi,…,xn, be converted to h={ h1,…,hi,…,hnHidden layer expression, xiIndicate i-th of node in sample, node is again
Referred to as unit, hiIt indicates to obtain x by codingiHidden layer indicate；The encoder is structure recurrent neural network Recursive
Neural Network or tree-like shot and long term memory network Tree-LSTM or two-way shot and long term memory networks Bi-LSTM；
Hidden layer is indicated that carrying out linear transformation obtains the score of each unit in sample, and the score of all nodes is asked by step 1-2
The score S of sample is just obtainedpointwise, wherein n indicate single sample in node number：
Wherein V1For model parameter, score is converted to by linear transformation for indicating node in point phase sorting one by one；
The weight score S of the point-by-point phase sorting of sample is calculated in step 1-31：
S1=α * Spointwise+(1-α)*Sbase,
Wherein, α indicates that hyper parameter, this hyper parameter are obtained by being adjusted in development set data, SbaseFor the basic score of sample,
Basic score is from the baseline system for generating sample；
Step 1-4 is ranked up sample by the obtained weight scores of step 1-3, chooses the higher sample composition of K score
Candidate Set, K are hyper parameter, and K sample is combined into sample pair, altogether K* (K-1)/2 sample pair.
3. according to the method described in claim 2, it is characterized in that, step 2 includes：
Step 2-1, sample A and sample B in K sample form sample pair<A, B>, using encoder by sample pair<A, B>In
Nodes encoding at hidden layer vector indicate；
Step 2-2 is compared to obtain the reference vector of node in sample pair；
Step 2-3 obtains the new vector expression after comparison according to the hidden layer vector sum of node with reference to vector；
Step 2-4 is that the vector that step 2-3 is obtained indicates to be encoded by encoder；
Step 2-5, obtains sample pair<A, B>Relative score, opposite give a mark of all samples pair obtains sample in comprehensive K sample
Originally in the score of pairs of phase sorting.
4. according to the method described in claim 3, it is characterized in that, in step 2-1, the encoder is structure recurrent neural net
Network Recursive Neural Network or tree-like shot and long term memory network Tree-LSTM or two-way shot and long term memory networks
Bi-LSTM。
5. according to the method described in claim 4, it is characterized in that, step 2-2 includes：Based on attention mechanism (Attention
Mechanism), the node a in sample A is calculated by following formulaiReference vector
Wherein, n indicates the number of sample B node, bjIndicate j-th of node in sample B, eijIndicate node aiAnd bjCorrelation
Degree.
6. according to the method described in claim 4, it is characterized in that, step 2-2 includes：It is calculated in sample A by following formula
Node aiReference vector
Wherein span () function representation obtains the range intervals of the leaf node included by node, and b indicates to be included in area in sample B
Between span (ai) section some node, len () function representation obtain node included by leaf node number.
7. according to the method described in claim 4, it is characterized in that, step 2-2 includes：It is calculated in sample A by following formula
Node aiReference vector
Wherein span () function representation obtains the range intervals of the leaf node included by node, and b indicates section span in sample B
(ai) some node for including, e indicates node aiWith the degree of correlation of b.
8. the method according to any one of claim 5~7, which is characterized in that step 2-3 includes：Pass through following formula
New vector is obtained to indicate
Or new vector is obtained using following formula and is indicated
Wherein g1、g2The output vector for indicating two door units respectively, for controlling information flow, ⊙ indicates two same dimensions
The each dimension of vector be multiplied, wherein W1And b1,W2And b1The parameter of respectively two doors, σ are activation primitive.
9. according to the method described in claim 8, it is characterized in that, step 2-4 includes：N vector table is obtained in step 2-3 mono-
Show, is denoted as It is indicated for n-th of vector, l values are 1~n, and the vector that step 2-3 is obtained is indicatedEncoded again using encoder obtain new vector m ..., mi,…,mn, mnIt indicatesAfter coding
New vector indicates.
10. according to the method described in claim 9, it is characterized in that, step 2-5 includes：The coding result of step 2-4 is carried out
Linear transformation obtains the score of each node in sample, and one that the score summation of all nodes has just been obtained sample obtains relatively
PointWherein V2For model parameter, obtained for doing linear transformation to hidden layer expression in pairs of phase sorting
It gives a mark to node, miFor with reference to vectorThe vector obtained after encoding again indicates, then by the phase of A and every other sample
To score summation can be obtained sample A pairs of phase sorting score, be each sample pairs of phase sorting minute mark
For Spairwise。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810460253.5A CN108717434B (en) | 2018-05-15 | 2018-05-15 | Text ordering method for mixed point-by-point strategy and paired strategy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810460253.5A CN108717434B (en) | 2018-05-15 | 2018-05-15 | Text ordering method for mixed point-by-point strategy and paired strategy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108717434A true CN108717434A (en) | 2018-10-30 |
| CN108717434B CN108717434B (en) | 2020-07-31 |
Family
ID=63899996
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810460253.5A Active CN108717434B (en) | 2018-05-15 | 2018-05-15 | Text ordering method for mixed point-by-point strategy and paired strategy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108717434B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103336793A (en) * | 2013-06-09 | 2013-10-02 | 中国科学院计算技术研究所 | Personalized paper recommendation method and system thereof |
| US20160041982A1 (en) * | 2014-08-05 | 2016-02-11 | Facebook, Inc. | Conditioned Search Ranking Models on Online Social Networks |
| CN106156135A (en) * | 2015-04-10 | 2016-11-23 | 华为技术有限公司 | The method and device of inquiry data |
| US20170193387A1 (en) * | 2015-12-31 | 2017-07-06 | Nuance Communications, Inc. | Probabilistic Ranking for Natural Language Understanding |
| CN107329960A (en) * | 2017-06-29 | 2017-11-07 | 哈尔滨工业大学 | Unregistered word translating equipment and method in a kind of neural network machine translation of context-sensitive |
-
2018
- 2018-05-15 CN CN201810460253.5A patent/CN108717434B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103336793A (en) * | 2013-06-09 | 2013-10-02 | 中国科学院计算技术研究所 | Personalized paper recommendation method and system thereof |
| US20160041982A1 (en) * | 2014-08-05 | 2016-02-11 | Facebook, Inc. | Conditioned Search Ranking Models on Online Social Networks |
| CN106156135A (en) * | 2015-04-10 | 2016-11-23 | 华为技术有限公司 | The method and device of inquiry data |
| US20170193387A1 (en) * | 2015-12-31 | 2017-07-06 | Nuance Communications, Inc. | Probabilistic Ranking for Natural Language Understanding |
| CN107329960A (en) * | 2017-06-29 | 2017-11-07 | 哈尔滨工业大学 | Unregistered word translating equipment and method in a kind of neural network machine translation of context-sensitive |
Non-Patent Citations (3)
| Title |
|---|
| HUI ZHANG ET AL.: "K-Best Combination of Syntactic Parsers", 《PROCEEDINGS OF THE 2009 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING》 * |
| 梁金莲 等: "由粗到精的哈萨克语短语结构句法分析研究", 《中文信息学报》 * |
| 袁里驰: "基于统计的句法分析方法", 《中南大学学报（自然科学版）》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108717434B (en) | 2020-07-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tan et al. | Deep semantic role labeling with self-attention | |
| CN110348016B (en) | Text abstract generation method based on sentence correlation attention mechanism | |
| CN109597891B (en) | Text emotion analysis method based on bidirectional long-and-short-term memory neural network | |
| CN110083705A (en) | A kind of multi-hop attention depth model, method, storage medium and terminal for target emotional semantic classification | |
| CN110083700A (en) | A kind of enterprise's public sentiment sensibility classification method and system based on convolutional neural networks | |
| CN110442684A (en) | A kind of class case recommended method based on content of text | |
| CN110489755A (en) | Document creation method and device | |
| CN108984724A (en) | It indicates to improve particular community emotional semantic classification accuracy rate method using higher-dimension | |
| CN108647226B (en) | Hybrid recommendation method based on variational automatic encoder | |
| CN109933656A (en) | Public sentiment polarity prediction technique, device, computer equipment and storage medium | |
| CN107066446A (en) | A kind of Recognition with Recurrent Neural Network text emotion analysis method of embedded logic rules | |
| KR20190063978A (en) | Automatic classification method of unstructured data | |
| CN113239186A (en) | Graph convolution network relation extraction method based on multi-dependency relation representation mechanism | |
| CN113761893B (en) | Relation extraction method based on mode pre-training | |
| CN110287323A (en) | A kind of object-oriented sensibility classification method | |
| DE112020003538T5 (en) | CROSS-MODAL RECOVERY WITH WORD OVERLAP BASED CLUSTERS | |
| CN112784602A (en) | News emotion entity extraction method based on remote supervision | |
| CN114881042A (en) | Chinese emotion analysis method based on graph convolution network fusion syntax dependence and part of speech | |
| CN116403231A (en) | Multi-hop reading understanding method and system based on double-view contrast learning and graph pruning | |
| CN110019796A (en) | A kind of user version information analysis method and device | |
| Banerjee et al. | Relation extraction using multi-encoder lstm network on a distant supervised dataset | |
| CN111382333B (en) | Case element extraction method in news text sentence based on case correlation joint learning and graph convolution | |
| He et al. | Reinforcement learning with external knowledge and two-stage q-functions for predicting popular reddit threads | |
| Kurup et al. | Evolution of neural text generation: Comparative analysis | |
| CN107329951A (en) | Build name entity mark resources bank method, device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |