CN108228564A - The name entity recognition method of confrontation study is carried out in crowdsourcing data - Google Patents
The name entity recognition method of confrontation study is carried out in crowdsourcing data Download PDFInfo
- Publication number
- CN108228564A CN108228564A CN201810007733.6A CN201810007733A CN108228564A CN 108228564 A CN108228564 A CN 108228564A CN 201810007733 A CN201810007733 A CN 201810007733A CN 108228564 A CN108228564 A CN 108228564A
- Authority
- CN
- China
- Prior art keywords
- result
- sequence
- way
- reverse
- results
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Abstract
The present invention relates to a kind of name entity recognition methods that confrontation study is carried out in crowdsourcing data, in specific field, as long as make entity Marking Guidelines, just can language material be marked with relatively low mark cost rapid build on a large scale with the method that crowdsourcing marks, the predicament for lacking mark language material is alleviated to a certain extent, it is so more preferable that use crowdsourcing data, improve learning quality of the model in crowdsourcing data：Different from data occupation mode before, we enable model independently to learn in language material, and mark is improper as caused by different labeled person, confrontation learning model is added on original model, the mark noise in language material is determined with this, improve the recognition capability of model, preferably tend to the high quality mark strategy of expert, enrich the realistic meaning of neural network model, be allowed to meet specific application.
Description
Technical field
The present invention relates to name Entity recognition tasks, real more particularly to the name that confrontation study is carried out in crowdsourcing data
Body recognition methods.
Background technology
Name Entity recognition task is usually abstracted into sequence labelling problem.Traditional learning method, such as hidden Ma Erke
Husband's model, maximum entropy hidden Markov model and condition random field learning model are all the structures on extensive artificial mark language material
Feature is made, this have the consequence that the performance of model can largely be limited to the quality of feature；In recent years based on nerve
The model of network achieves great achievement by large-scale application to the field.But Entity recognition task is named at present,
Still it is faced with the predicament for lacking extensive artificial mark language material.
(1) data acquisition：Expert marks and crowdsourcing data
1) the most commonly used data capture method is the method for " domain expert's mark ".It engages and is familiar with language material and mark rule
Professional then is labeled, and obtains high quality language material.
2) " crowdsourcing " this concept is by the U.S.《Line》The reporter Jeff of magazine bold and unconstrained (JeffHowe) is in June, 2006
It proposes.It is exactly that a task or work are contracted out to public (layman) to complete in simple terms, in the hope of accelerating to appoint
The speed that business is completed, reduces cost.In natural language processing field, this method is used in language material and obtains：Script is needed
The language material marked by domain expert distributes to the multidigit social tagging person for only carrying out short-term cultivation, thus can be in the short time
It is interior that large-scale mark language material is obtained with relatively low cost.
(2) the sequence labelling model based on neural network
It is widely used in the LSTM-CRF models of sequence labelling problem
In Chinese names Entity recognition task, the input of model is a Chinese sequence, by each Chinese in sequence
Character vectorization represents that can be obtained by a preferable data by LSTM network extraction informations represents, finally will linearly unite
Meter model (conditional random field models) is combined with neural network structure, makes the more preferable dependence that must learn between sequence label of model
Relationship.(it specifically may refer to document Lample G, Ballesteros M, Subramanian S, et al.Neural
Architectures for Named Entity Recognition[J].2016:260-270.)
There are following technical problems for traditional technology：
Data acquisition technology：
1. domain expert's mask method mainly has following shortcoming：
1) number of expert mark person is in general fewer, can be labeled the very few of work, mark speed for a long time
Degree is slow, it is difficult to obtain mark language material on a large scale, can not meet the needs of practical application；
2) engage the expense of expert mark person relatively high, mark cost is big.
The shortcomings that crowdsourcing mask method：
1) crowdsourcing personnel are mostly without what experience, so needing to formulate detailed mark before mark to data fields
Rule will carry out mark personnel short-term special training, time-consuming and laborious.
2) different labeled person may have different understanding and labeled accustomed for rule and language material, cause to deposit in annotation results
Inconsistent and marking error is largely being marked, the quality of data is low.
3) it is difficult to assess the quality of crowdsourcing mark language material.
The shortcomings that sequence labelling model based on neural network：
Neural network is widely used in the multiple tasks of natural language processing in recent years, although all achieving not mostly
Wrong achievement, but the shortcomings that own also clearly：
1) no one can make it very strict and system theoretical proof so far.
2) basis of Establishment of Neural Model is to carry out vectorization expression to language material, but for these middle table registrations
According to (vector), even final result, we can not accurately connect it and things logic specific in reality, can only
These data of the supposition of " obscure " represent certain relationship between language material.
3) neural network places one's entire reliance upon data (data scale, the quality of data), and model is no any logic in itself
Leading ability, it only can merely think that the training corpus of all inputs is all the high quality language material marked by expert, according to
What its training logic finally learnt meets expert's mark language material distribution.But actual conditions are not so optimistic, such as
Fruit language material is of low quality, such as our crowdsourcing data for using, and there are a certain number of marking errors, are marked on the whole with expert
Data have certain gap, because not having judgement to the quality of language material, its final performance can be also greatly affected model.
Invention content
Based on this, it is necessary to for above-mentioned technical problem, provide a kind of name that confrontation study is carried out in crowdsourcing data
Entity recognition method,, just can be with relatively low with the method that crowdsourcing marks as long as making entity Marking Guidelines in specific field
Mark cost rapid build mark language material on a large scale, alleviate the predicament for lacking mark language material to a certain extent, it is more preferable
Using crowdsourcing data, learning quality of the model in crowdsourcing data is improved：Different from data occupation mode before, we make mould
Type can independently learn in language material as caused by different labeled person that mark is improper, and confrontation study mould is added on original model
Type determines the mark noise in language material with this, improves the recognition capability of model, preferably tends to the high quality mark plan of expert
Slightly, the realistic meaning of neural network model is enriched, is allowed to meet specific application.
A kind of name entity recognition method that confrontation study is carried out in crowdsourcing data, including：
Training stage：
In the first input layer, a training sequence is inputted；
In the first mapping layer, each word in the training sequence is mapped to a vector；
It is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
It is LSTM layers two-way second, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
By step " it is LSTM layers two-way first, the training sequence is calculated according to forward and reverse respectively as a result, will
The two results are spliced；" output result and step " it is LSTM layers two-way first, by the training sequence according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the training sequence according to forward direction
It reversely calculates respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, by institute
Training sequence is stated to be calculated respectively according to forward and reverse as a result, the two results are spliced；" output result splicing；" knot
Fruit, output represent the vector of the score value of annotated sequence；
In Tag Estimation layer, " in full articulamentum, input step is " by step " in the first two-way LSTM for input step
Layer, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and
Step " calculates the training sequence according to forward and reverse as a result, by the two results at first two-way LSTM layers respectively
Splicing；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, use condition with
Predict final sequence label in airport；
Calculating step, " in Tag Estimation layer, " in full articulamentum, input step is " by step " first for input step
It is LSTM layers two-way, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" it is defeated
Go out result and step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by this
Two result splicings；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, making
With the sequence label that condition random field prediction is final；" output result and correct answer difference, i.e. the first difference；
In the second input layer, the corresponding sequence label of the training sequence is inputted；
In the second mapping layer, each word in the sequence label is mapped to a vector；
It is LSTM layers two-way second, the sequence label is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
It is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
By step " it is LSTM layers two-way second, the sequence label is calculated according to forward and reverse respectively as a result, will
The two results are spliced；" output result and step " it is LSTM layers two-way in third, by the sequence label according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to positive and
Reversely calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, will described in
Sequence label is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；" knot
Fruit extracts feature, calculates the score of multidigit mark person；
Calculate step " in convolutional layer, input step " by step " it is LSTM layers two-way second, the sequence label is pressed
It is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and step " in the two-way LSTM of third
Layer, the sequence label is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result spell
It connects；" as a result, extraction feature, calculate multidigit mark person score；" output result and correct answer difference, i.e., second
Difference；
Judge whether first difference and second difference meet predetermined condition；
If so, terminate the training stage, into forecast period；Otherwise, first difference and second difference pair are utilized
Systematic parameter reversely updates, and continues back to step and " in the first input layer, inputs a training sequence；" continue to train；
Forecast period：
In the first input layer, a sequence to be predicted is inputted；
In the first mapping layer, each word in the sequence to be predicted is mapped to a vector；
It is LSTM layers two-way first, the sequence to be predicted is calculated according to forward and reverse respectively as a result, by this two
A result splicing；
It is LSTM layers two-way second, the sequence to be predicted is calculated according to forward and reverse respectively as a result, by this two
A result splicing；
By step " it is LSTM layers two-way first, the sequence to be predicted is calculated according to forward and reverse respectively as a result,
The two results are spliced；" output result and step " it is LSTM layers two-way first, by the sequence to be predicted according to forward direction
It reversely calculates respectively as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the sequence to be predicted according to just
To with reversely calculate respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, will
The sequence to be predicted is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；”
As a result, output represent annotated sequence score value vector；
In Tag Estimation layer, the final sequence label of the random field prediction of use condition, retention forecasting result.
The above-mentioned name entity recognition method that confrontation study is carried out in crowdsourcing data, in specific field, as long as system
Entity Marking Guidelines are set, just can language material be marked with relatively low mark cost rapid build on a large scale with the method that crowdsourcing marks,
The predicament for lacking mark language material is alleviated to a certain extent, it is so more preferable that use crowdsourcing data, model is improved in crowdsourcing data
Learning quality：Different from data occupation mode before, we enable model independently to learn in language material by different labeled person
Caused by mark it is improper, confrontation learning model is added on original model, the mark noise in language material is determined with this, is carried
The recognition capability of high model preferably tends to the high quality mark strategy of expert, enriches the realistic meaning of neural network model,
It is allowed to meet specific application.
In other one embodiment, in step, " in convolutional layer, input step is " by step " in the second two-way LSTM
Layer, the sequence label is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and
Step " it is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two results
Splicing；" output result splicing；" as a result, extraction feature, calculate multidigit mark person score；" in, calculate multidigit mark person
The formula of score be：
In other one embodiment, step " judge first difference and second difference whether meet it is pre-
Fixed condition；" specifically include：The absolute value of the difference of first difference and second difference is less than predetermined value.
The training method of model in a kind of name entity recognition method that confrontation study is carried out in crowdsourcing data, including：
In the first input layer, a training sequence is inputted；
In the first mapping layer, each word in the training sequence is mapped to a vector；
It is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
It is LSTM layers two-way second, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
By step " it is LSTM layers two-way first, the training sequence is calculated according to forward and reverse respectively as a result, will
The two results are spliced；" output result and step " it is LSTM layers two-way first, by the training sequence according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the training sequence according to forward direction
It reversely calculates respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, by institute
Training sequence is stated to be calculated respectively according to forward and reverse as a result, the two results are spliced；" output result splicing；" knot
Fruit, output represent the vector of the score value of annotated sequence；
In Tag Estimation layer, " in full articulamentum, input step is " by step " in the first two-way LSTM for input step
Layer, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and
Step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two results
Splicing；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, use condition with
Predict final sequence label in airport；
Calculating step, " in Tag Estimation layer, " in full articulamentum, input step is " by step " first for input step
It is LSTM layers two-way, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" it is defeated
Go out result and step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by this
Two result splicings；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, making
With the sequence label that condition random field prediction is final；" output result and correct answer difference, i.e. the first difference；
In the second input layer, the corresponding sequence label of the training sequence is inputted；
In the second mapping layer, each word in the sequence label is mapped to a vector；
It is LSTM layers two-way second, the sequence label is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
It is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
By step " it is LSTM layers two-way second, the sequence label is calculated according to forward and reverse respectively as a result, will
The two results are spliced；" output result and step " it is LSTM layers two-way in third, by the sequence label according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to positive and
Reversely calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, will described in
Sequence label is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；" knot
Fruit extracts feature, calculates the score of multidigit mark person；
Calculate step " in convolutional layer, input step " by step " it is LSTM layers two-way second, the sequence label is pressed
It is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and step " in the two-way LSTM of third
Layer, the sequence label is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result spell
It connects；" as a result, extraction feature, calculate multidigit mark person score；" output result and correct answer difference, i.e., second
Difference；
Judge whether first difference and second difference meet predetermined condition；
If so, terminate training；Otherwise, systematic parameter is reversely updated using first difference and second difference,
It continues back to step and " in the first input layer, inputs a training sequence；" continue to train.
In other one embodiment, in step, " in convolutional layer, input step is " by step " in the second two-way LSTM
Layer, the sequence label is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and
Step " it is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two results
Splicing；" output result splicing；" as a result, extraction feature, calculate multidigit mark person score；" in, calculate multidigit mark person
The formula of score be：
In other one embodiment, step " judge first difference and second difference whether meet it is pre-
Fixed condition；" specifically include：The absolute value of the difference of first difference and second difference is less than predetermined value.
A kind of computer equipment can be run on a memory and on a processor including memory, processor and storage
The step of computer program, the processor realizes above-mentioned middle any one the method when performing described program.
A kind of computer readable storage medium, is stored thereon with computer program, which realizes when being executed by processor
The step of above-mentioned any one the method.
Description of the drawings
Fig. 1 is a kind of name entity recognition method that confrontation study is carried out in crowdsourcing data provided by the embodiments of the present application
Confrontation neural network model figure.
Fig. 2 is a kind of name entity recognition method that confrontation study is carried out in crowdsourcing data provided by the embodiments of the present application
Flow chart.
Fig. 3 is a kind of name entity recognition method that confrontation study is carried out in crowdsourcing data provided by the embodiments of the present application
Another flow chart.
Specific embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and embodiments, it is right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
A kind of name entity recognition method that confrontation study is carried out in crowdsourcing data, including：
Training stage：
In the first input layer, a training sequence is inputted；
In the first mapping layer, each word in the training sequence is mapped to a vector；
It is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
It is LSTM layers two-way second, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
By step " it is LSTM layers two-way first, the training sequence is calculated according to forward and reverse respectively as a result, will
The two results are spliced；" output result and step " it is LSTM layers two-way first, by the training sequence according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the training sequence according to forward direction
It reversely calculates respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, by institute
Training sequence is stated to be calculated respectively according to forward and reverse as a result, the two results are spliced；" output result splicing；" knot
Fruit, output represent the vector of the score value of annotated sequence；
In Tag Estimation layer, " in full articulamentum, input step is " by step " in the first two-way LSTM for input step
Layer, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and
Step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two results
Splicing；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, use condition with
Predict final sequence label in airport；
Calculating step, " in Tag Estimation layer, " in full articulamentum, input step is " by step " first for input step
It is LSTM layers two-way, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" it is defeated
Go out result and step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by this
Two result splicings；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, making
With the sequence label that condition random field prediction is final；" output result and correct answer difference, i.e. the first difference；
In the second input layer, the corresponding sequence label of the training sequence is inputted；
In the second mapping layer, each word in the sequence label is mapped to a vector；
It is LSTM layers two-way second, the sequence label is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
It is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
By step " it is LSTM layers two-way second, the sequence label is calculated according to forward and reverse respectively as a result, will
The two results are spliced；" output result and step " it is LSTM layers two-way in third, by the sequence label according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to positive and
Reversely calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, will described in
Sequence label is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；" knot
Fruit extracts feature, calculates the score of multidigit mark person；
Calculate step " in convolutional layer, input step " by step " it is LSTM layers two-way second, the sequence label is pressed
It is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and step " in the two-way LSTM of third
Layer, the sequence label is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result spell
It connects；" as a result, extraction feature, calculate multidigit mark person score；" output result and correct answer difference, i.e., second
Difference；
Judge whether first difference and second difference meet predetermined condition；
If so, terminate the training stage, into forecast period；Otherwise, first difference and second difference pair are utilized
Systematic parameter reversely updates, and continues back to step and " in the first input layer, inputs a training sequence；" continue to train；
Forecast period：
In the first input layer, a sequence to be predicted is inputted；
In the first mapping layer, each word in the sequence to be predicted is mapped to a vector；
It is LSTM layers two-way first, the sequence to be predicted is calculated according to forward and reverse respectively as a result, by this two
A result splicing；
It is LSTM layers two-way second, the sequence to be predicted is calculated according to forward and reverse respectively as a result, by this two
A result splicing；
By step " it is LSTM layers two-way first, the sequence to be predicted is calculated according to forward and reverse respectively as a result,
The two results are spliced；" output result and step " it is LSTM layers two-way first, by the sequence to be predicted according to forward direction
It reversely calculates respectively as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the sequence to be predicted according to just
To with reversely calculate respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, will
The sequence to be predicted is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；”
As a result, output represent annotated sequence score value vector；
In Tag Estimation layer, the final sequence label of the random field prediction of use condition, retention forecasting result.
The above-mentioned name entity recognition method that confrontation study is carried out in crowdsourcing data, in specific field, as long as system
Entity Marking Guidelines are set, just can language material be marked with relatively low mark cost rapid build on a large scale with the method that crowdsourcing marks,
The predicament for lacking mark language material is alleviated to a certain extent, it is so more preferable that use crowdsourcing data, model is improved in crowdsourcing data
Learning quality：Different from data occupation mode before, we enable model independently to learn in language material by different labeled person
Caused by mark it is improper, confrontation learning model is added on original model, the mark noise in language material is determined with this, is carried
The recognition capability of high model preferably tends to the high quality mark strategy of expert, enriches the realistic meaning of neural network model,
It is allowed to meet specific application.
In other one embodiment, in step, " in convolutional layer, input step is " by step " in the second two-way LSTM
Layer, the sequence label is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and
Step " it is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two results
Splicing；" output result splicing；" as a result, extraction feature, calculate multidigit mark person score；" in, calculate multidigit mark person
The formula of score be：
In other one embodiment, step " judge first difference and second difference whether meet it is pre-
Fixed condition；" specifically include：The absolute value of the difference of first difference and second difference is less than predetermined value.
The training method of model in a kind of name entity recognition method that confrontation study is carried out in crowdsourcing data, including：
In the first input layer, a training sequence is inputted；
In the first mapping layer, each word in the training sequence is mapped to a vector；
It is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
It is LSTM layers two-way second, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
By step " it is LSTM layers two-way first, the training sequence is calculated according to forward and reverse respectively as a result, will
The two results are spliced；" output result and step " it is LSTM layers two-way first, by the training sequence according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the training sequence according to forward direction
It reversely calculates respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, by institute
Training sequence is stated to be calculated respectively according to forward and reverse as a result, the two results are spliced；" output result splicing；" knot
Fruit, output represent the vector of the score value of annotated sequence；
In Tag Estimation layer, " in full articulamentum, input step is " by step " in the first two-way LSTM for input step
Layer, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and
Step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two results
Splicing；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, use condition with
Predict final sequence label in airport；
Calculating step, " in Tag Estimation layer, " in full articulamentum, input step is " by step " first for input step
It is LSTM layers two-way, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" it is defeated
Go out result and step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by this
Two result splicings；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, making
With the sequence label that condition random field prediction is final；" output result and correct answer difference, i.e. the first difference；
In the second input layer, the corresponding sequence label of the training sequence is inputted；
In the second mapping layer, each word in the sequence label is mapped to a vector；
It is LSTM layers two-way second, the sequence label is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
It is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two
As a result splice；
By step " it is LSTM layers two-way second, the sequence label is calculated according to forward and reverse respectively as a result, will
The two results are spliced；" output result and step " it is LSTM layers two-way in third, by the sequence label according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to positive and
Reversely calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, will described in
Sequence label is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；" knot
Fruit extracts feature, calculates the score of multidigit mark person；
Calculate step " in convolutional layer, input step " by step " it is LSTM layers two-way second, the sequence label is pressed
It is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and step " in the two-way LSTM of third
Layer, the sequence label is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result spell
It connects；" as a result, extraction feature, calculate multidigit mark person score；" output result and correct answer difference, i.e., second
Difference；
Judge whether first difference and second difference meet predetermined condition；
If so, terminate training；Otherwise, systematic parameter is reversely updated using first difference and second difference,
It continues back to step and " in the first input layer, inputs a training sequence；" continue to train.
In other one embodiment, in step, " in convolutional layer, input step is " by step " in the second two-way LSTM
Layer, the sequence label is calculated respectively according to forward and reverse as a result, the two results are spliced；" output result and
Step " it is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two results
Splicing；" output result splicing；" as a result, extraction feature, calculate multidigit mark person score；" in, calculate multidigit mark person
The formula of score be：
In other one embodiment, step " judge first difference and second difference whether meet it is pre-
Fixed condition；" specifically include：The absolute value of the difference of first difference and second difference is less than predetermined value.
A kind of computer equipment can be run on a memory and on a processor including memory, processor and storage
The step of computer program, the processor realizes above-mentioned middle any one the method when performing described program.
A kind of computer readable storage medium, is stored thereon with computer program, which realizes when being executed by processor
The step of above-mentioned any one the method.
The specific application scenarios of the present invention are described below：
The purpose of this patent is summarized, the use quality of crowdsourcing data is exactly improved, enables model " intelligence " that language material must be avoided
In marking error.In order to accomplish this point, then we make firstly the need of a collection of crowdsourcing data of construction on crowdsourcing data set
With the method that " confrontation " learns come the performance of lift scheme.
Crowdsourcing marks language material acquisition process：
1. according to language material feature and practical application request, substantial definition and Marking Guidelines are formulated；
2. recruiting mark personnel, giveed training according to specification.Mark personnel are tested after training, to test knot
Fruit employs the high personnel of mark quality for standard；
3. in every a batch data to be marked, 20 percent is land mine data (data for having answer), in order to
Assess the annotation results of mark person；Every words are randomly assigned to three mark persons and are labeled.
Each language material may possess identical annotation results, it is also possible to which annotation results are different.With " Adidas/
For eight or five folding of Adidas whole audience today " this sentence mark language material, it is desirable that mark out " brand " entity in sentence to come.Some marks
Note person thinks that " Adidas " and " Adidas " should be divided mark into two independent brand entities, also has mark person to think
" Adidas/Adidas " should mark as a whole.
Fig. 1 is a kind of name entity recognition method that confrontation study is carried out in crowdsourcing data provided by the embodiments of the present application
Confrontation neural network model figure.Our model can be divided into two parts, the baseline models on the right side of Fig. 1 (refer to
Document Lample G, Ballesteros M, Subramanian S, et al.Neural Architectures for Named
Entity Recognition[J].2016:260-270.) and the confrontation learning model in left side.
The model structure on right side independently can name entity recognition system (baseline i.e. cited below) as one,
The LSTM-CRF models being widely used at present.Its input is a Chinese sequence x=c1...cn, it is therefore an objective to determine one
Best sequence label y=y1...yn, wherein yiIt is ciLabel, such as " B " " E " " O " in table 1.
| Chars | Yao | It is bright | It is | In | State | People | 。 |
| Labels | B | E | O | O | O | O | O |
Table 1 will name Entity recognition to regard sequence labelling problem as
The learning procedure of baseline models is introduced first：
Each word vectorization in sentence is represented first and is input in LSTM neural network models, each word corresponds to only
One multidimensional floating type vector.In frame portion point red on the right side of Fig. 1, wi is each character (and ci meanings in formula
It is identical)；Xi is the corresponding vector of each character.
To the corresponding vector x t of each word in sequence, LSTM layers can all calculate one and represent vectorTo represent current
The text message on this word left side.While the text message on the right of this current word of study in general,Model can equally be made
Learn to useful information.So the sequence of current sequence can be inverted, a new sequence is obtained, is then input to another
In LSTM layers a.We are referred to as preceding to LSTM layers by the former, and the latter is known as LSTM layers backward.This is two independent networks, phase
It is not interfered between mutually, respectively possesses different parameters.Forward and backward LSTM is combined,Just obtain
One bilayer model.
By output h LSTM layers two-waytIt is input in a full articulamentum, the output vector of this layer represents annotated sequence
Score value, dimension are identical with the number of label.
In order to which model is made preferably to learn to the dependence between sequence label, our use condition random fields
(CRF) final sequence label is predicted：
We define a function first：
X is read statement described above in score formula；Y is the corresponding possible prediction label of sentence X
Sequence；The output of matrix P full articulamentums in being 3, size are n × k, and wherein k is the size of the tally set of setting, and n is works as preamble
The length of row.Matrix A is a transfer matrix, matrix element AI, jIt is the probability that label i is transferred to label j；y0And ynIt is us
The beginning and end label of the sequence of addition, so matrix A is the square formation that a size is k+2.
According to scoring function, we can be normalized on all possible sequence label, calculate each label
The conditional probability of sequence：
In the training process, we maximize the probability of correct sequence label by calculating log-likelihood estimated value：
We judge to obtain correct sequence label of the label output sequence for current sequence of best result：
The details of more than baseline.
Model is divided into two states, training and prediction (prediction is exactly to actually use this model).In the training process, it is
System can calculate corresponding sequence label according to one one trained sentence of input, this sequence label is marked at the beginning and correctly
Label sequence is that difference is bigger certainly, that is to say, that the poor performance of model at the beginning.Then model pre- can be measured with oneself
To result and correct option a difference (loss) is calculated, and reversely update systematic parameter, newer target be exactly to the greatest extent
It can this difference of energy minimization loss.It is trained as a sentence is input into, this model is for the Tag Estimation of sequence
Ability can become better and better, the peak until reaching a performance (this is the process of a loop iteration).
There is so trained model, then we can do practical thing with it：Namely I will need
Go the sequence label result of prediction read statement.With input by sentence model, model can also obtain one according to above step
As a result, it is believed that this result is exactly that the correct sequence label of this word (might have certain mistake, but base certainly
It is correct on this)
Training and the difference predicted are exactly：Training is the performance in order to improve model, and can constantly update mould in the process
Type；Prediction is intended merely to use this model, will not update model detail.The illustraton of model changed can be had a look, training
As being with the input content of prediction.
One two-way LSTM layers in former baseline systems is increased to two, and be respectively designated as common by this patent
And private, our purpose are that common modules is enable to learn to the publicly-owned markup information of multiple mark persons, private moulds
Block meets the privately owned labeled accustomed distribution of multiple mark persons.
It proposes to add in confrontation learning model simultaneously, as shown in fig. 1 on the left-hand side：Classifier modules are added in master mould.Mesh
Be to aid in optimization update common LSTM modules learning process, the information for being allowed to learn tend to expert mark plan
Slightly, so as to which the estimated performance for making the right branch of model gets a promotion.It is with the corresponding sequence label of baseline system list entries
(by some mark person mark)To input, after sequence information vectorization, by the double of label entitled in Fig. 1
Information extraction is carried out to LSTM, after final and common output merges, then further information extraction is carried out and classifies.Classification
Device is to maximize classification error with the training simultaneously of baseline models and update, its optimization aim, illustrates common moulds in this way
Block increasingly meets the mark strategy of expert.
The frame to entire model and training process copy the declarative procedure of baseline to illustrate details below：
Because the input of our model training stage and forecast period is different, following separate introduction：
(1) it is a kind of name entity that confrontation study is carried out in crowdsourcing data provided by the embodiments of the present application refering to Fig. 2
The flow chart of recognition methods, training stage (being included in the modification on former baseline models) details are as follows：As a training
Data, it has all had known answer sequence label.Some word in sequence is inputted and is mapped to its corresponding vector x t
The namely effect of input layer 1 and mapping layer 1, as this with the input of the baseline mentioned above is；In addition we go back
The sequence label of this sequence is also served as inputting, each label mapping into a vector (left side for seeing flow chart), here it is
The effect of input layer 2 and mapping layer 2, this is that do not have in baseline.
Compared to baseline systems only by x=c1...cnBe input to one it is LSTM layers two-way, we increase in systems
One two-way LSTM layers, is respectively designated as common and private.By the output of mapping layer 1 be separately input to common and
Two-way LSTM layers of private is calculated, and is respectively obtainedWithTwo outputs；Identical, the output of mapping layer 2
Another two-way LSTM is input to, entitled label is calculatedHere it is the effects of three two-way LSTM in flow chart.
It is only compared with baseline above with a two-way LSTM, two-way LSTM is increased three by our model.
Because the more several LSTM of our model, will have the step of output is spliced, this is in baseline models
Without：First, the output of entitled private and common is spliced intoBy entitled label and
The output of common is spliced intoTo this step, we just have two result of calculations, also just corresponding
Two branches in figure.
For ht, that is, the branch on the right side of flow chart, to be done below as baseline before：Input connects entirely
Layer is connect, then inputs final CRF layers and gives a forecast, it is seen that flow chart, the work of this part and the complete phase of 3-5 points of baseline
Together, it including formula, is here no longer described in detail, that is, model goes to this branch, predicts the sequence label of current input sequence.
ForThe CNN layers that it is inputted left side in figure by we again further extract feature
Afterwards, the classification score of multidigit mark person is calculated, that is, it is whom that the sequence currently inputted can be calculated according to following formula
Target：
It summarizes, in the training process, in short, by series of computation, model has two branches in left and right for input
Two outputs, that is, two prediction results.Two results and two correct answers are done into mathematic interpolation respectively, can be obtained
To two differences：Loss1 and loss2 makees systematic parameter reversely update further according to the two differences.We need to make as far as possible
Loss1 is minimized.Loss2 is maximized, in this way by being negated to loss2, i.e.,
The optimization aim of training is to minimize above formula as a result, the estimated performance of the right branch of model after handling in this way
Also it becomes better and better, has also just reached the target of our model work.
(2) it is the explanation of prediction process below：
It is a kind of name Entity recognition that confrontation study is carried out in crowdsourcing data provided by the embodiments of the present application refering to Fig. 3
The another flow chart of method.
Input and training stage difference during our model prediction.During the training stage, two inputs, forecast period are needed
The input on right side is only needed, that is, 1 is inputted, because not needing to input 2, then left branch just does not have to walk, and finally just has to yet
Retain to the output prediction result of right branch, and as correct result, be thus a complete prediction of a word
Flow.Certainly as baseline, the update work of parameter it is not related to during prediction.
Each technical characteristic of embodiment described above can be combined arbitrarily, to make description succinct, not to above-mentioned reality
It applies all possible combination of each technical characteristic in example to be all described, as long as however, the combination of these technical characteristics is not deposited
In contradiction, it is all considered to be the range of this specification record.
Embodiment described above only expresses the several embodiments of the present invention, and description is more specific and detailed, but simultaneously
It cannot therefore be construed as limiting the scope of the patent.It should be pointed out that those of ordinary skill in the art are come
It says, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the protection of the present invention
Range.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.
Claims (8)
1. a kind of name entity recognition method that confrontation study is carried out in crowdsourcing data, which is characterized in that including：
Training stage：
In first input layer, a training sequence is inputted；
In the first mapping layer, each word in the training sequence is mapped to a vector；
It is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two results
Splicing；
It is LSTM layers two-way second, the training sequence is calculated respectively according to forward and reverse as a result, by the two results
Splicing；
By step " it is LSTM layers two-way first, the training sequence is calculated according to forward and reverse respectively as a result, by this two
A result splicing；" output result and step " it is LSTM layers two-way first, by the training sequence according to forward and reverse point
It does not calculate as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the training sequence according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, by the instruction
Practice sequence to be calculated respectively according to forward and reverse as a result, the two results are spliced；" output result splicing；" as a result,
Output represents the vector of the score value of annotated sequence；
In Tag Estimation layer, " in full articulamentum, input step is " by step ", general LSTM layers two-way first for input step
The training sequence is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result and step
" it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spelled
It connects；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, use condition is random
The final sequence label of field prediction；
Calculating step, " in Tag Estimation layer, " in full articulamentum, input step is " " two-way first by step for input step
LSTM layers, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" output knot
Fruit and step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, using item
The final sequence label of the random field prediction of part；" output result and correct answer difference, i.e. the first difference；
In the second input layer, the corresponding sequence label of the training sequence is inputted；
In the second mapping layer, each word in the sequence label is mapped to a vector；
It is LSTM layers two-way second, the sequence label is calculated respectively according to forward and reverse as a result, by the two results
Splicing；
It is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two results
Splicing；
By step " it is LSTM layers two-way second, the sequence label is calculated according to forward and reverse respectively as a result, by this two
A result splicing；" output result and step " it is LSTM layers two-way in third, by the sequence label according to forward and reverse point
It does not calculate as a result, the two results are spliced；" output result splicing；
In convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to forward and reverse
It is calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, by the label
Sequence is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；" as a result, carrying
Feature is taken, calculates the score of multidigit mark person；
Calculate step " in convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to just
To with reversely calculate respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, will
The sequence label is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；"
As a result, extraction feature, the score of calculating multidigit mark person；" output result and correct answer difference, i.e. the second difference；
Judge whether first difference and second difference meet predetermined condition；
If so, terminate the training stage, into forecast period；Otherwise, using first difference and second difference to system
Parameter reversely updates, and continues back to step and " in the first input layer, inputs a training sequence；" continue to train；
Forecast period：
In the first input layer, a sequence to be predicted is inputted；
In the first mapping layer, each word in the sequence to be predicted is mapped to a vector；
It is LSTM layers two-way first, the sequence to be predicted is calculated respectively according to forward and reverse as a result, the two are tied
Fruit is spliced；
It is LSTM layers two-way second, the sequence to be predicted is calculated respectively according to forward and reverse as a result, the two are tied
Fruit is spliced；
By step " it is LSTM layers two-way first, the sequence to be predicted is calculated respectively according to forward and reverse as a result, by this
Two result splicings；" output result and step " it is LSTM layers two-way first, by the sequence to be predicted according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the sequence to be predicted according to positive and
Reversely calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, will described in
Sequence to be predicted is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；" knot
Fruit, output represent the vector of the score value of annotated sequence；
In Tag Estimation layer, the final sequence label of the random field prediction of use condition, retention forecasting result.
2. the name entity recognition method according to claim 1 that confrontation study is carried out in crowdsourcing data, feature exist
In, step " in convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to forward direction
It reversely calculates respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, by institute
Sequence label is stated to be calculated respectively according to forward and reverse as a result, the two results are spliced；" output result splicing；" knot
Fruit extracts feature, calculates the score of multidigit mark person；" in, calculating the formula of the score of multidigit mark person is：
。
3. the name entity recognition method according to claim 1 that confrontation study is carried out in crowdsourcing data, feature exist
In step " judges whether first difference and second difference meet predetermined condition；" specifically include：Described first is poor
The absolute value of the difference of value and second difference is less than predetermined value.
4. the training method of model, feature exist in a kind of name entity recognition method that confrontation study is carried out in crowdsourcing data
In, including：
In the first input layer, a training sequence is inputted；
In the first mapping layer, each word in the training sequence is mapped to a vector；
It is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two results
Splicing；
It is LSTM layers two-way second, the training sequence is calculated respectively according to forward and reverse as a result, by the two results
Splicing；
By step " it is LSTM layers two-way first, the training sequence is calculated according to forward and reverse respectively as a result, by this two
A result splicing；" output result and step " it is LSTM layers two-way first, by the training sequence according to forward and reverse point
It does not calculate as a result, the two results are spliced；" output result splicing；
In full articulamentum, input step " by step " it is LSTM layers two-way first, by the training sequence according to positive and anti-
To being calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way first, by the instruction
Practice sequence to be calculated respectively according to forward and reverse as a result, the two results are spliced；" output result splicing；" as a result,
Output represents the vector of the score value of annotated sequence；
In Tag Estimation layer, " in full articulamentum, input step is " by step ", general LSTM layers two-way first for input step
The training sequence is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result and step
" it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spelled
It connects；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, use condition is random
The final sequence label of field prediction；
Calculating step, " in Tag Estimation layer, " in full articulamentum, input step is " " two-way first by step for input step
LSTM layers, the training sequence is calculated respectively according to forward and reverse as a result, the two results are spliced；" output knot
Fruit and step " it is LSTM layers two-way first, the training sequence is calculated respectively according to forward and reverse as a result, by the two
As a result splice；" output result splicing；" as a result, output represent annotated sequence score value vector；" as a result, using item
The final sequence label of the random field prediction of part；" output result and correct answer difference, i.e. the first difference；
In the second input layer, the corresponding sequence label of the training sequence is inputted；
In the second mapping layer, each word in the sequence label is mapped to a vector；
It is LSTM layers two-way second, the sequence label is calculated respectively according to forward and reverse as a result, by the two results
Splicing；
It is LSTM layers two-way in third, the sequence label is calculated respectively according to forward and reverse as a result, by the two results
Splicing；
By step " it is LSTM layers two-way second, the sequence label is calculated according to forward and reverse respectively as a result, by this two
A result splicing；" output result and step " it is LSTM layers two-way in third, by the sequence label according to forward and reverse point
It does not calculate as a result, the two results are spliced；" output result splicing；
In convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to forward and reverse
It is calculated respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, by the label
Sequence is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；" as a result, carrying
Feature is taken, calculates the score of multidigit mark person；
Calculate step " in convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to just
To with reversely calculate respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, will
The sequence label is calculated according to forward and reverse respectively as a result, the two results are spliced；" output result splicing；"
As a result, extraction feature, the score of calculating multidigit mark person；" output result and correct answer difference, i.e. the second difference；
Judge whether first difference and second difference meet predetermined condition；
If so, terminate training；Otherwise, systematic parameter is reversely updated using first difference and second difference, continued
" in the first input layer, a training sequence is inputted back to step；" continue to train.
5. the name entity recognition method according to claim 4 that confrontation study is carried out in crowdsourcing data, feature exist
In, step " in convolutional layer, input step " by step " it is LSTM layers two-way second, by the sequence label according to forward direction
It reversely calculates respectively as a result, the two results are spliced；" output result and step " it is LSTM layers two-way in third, by institute
Sequence label is stated to be calculated respectively according to forward and reverse as a result, the two results are spliced；" output result splicing；" knot
Fruit extracts feature, calculates the score of multidigit mark person；" in, calculating the formula of the score of multidigit mark person is：
。
6. the name entity recognition method according to claim 4 that confrontation study is carried out in crowdsourcing data, feature exist
In step " judges whether first difference and second difference meet predetermined condition；" specifically include：Described first is poor
The absolute value of the difference of value and second difference is less than predetermined value.
7. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor realizes any one the method in claim 1-6 when performing described program
The step of.
8. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
The step of claim 1-6 any one the methods are realized during row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810007733.6A CN108228564B (en) | 2018-01-04 | 2018-01-04 | Named entity recognition method, device and readable storage medium for counterlearning on crowdsourced data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810007733.6A CN108228564B (en) | 2018-01-04 | 2018-01-04 | Named entity recognition method, device and readable storage medium for counterlearning on crowdsourced data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108228564A true CN108228564A (en) | 2018-06-29 |
| CN108228564B CN108228564B (en) | 2021-02-19 |
Family
ID=62642955
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810007733.6A Active CN108228564B (en) | 2018-01-04 | 2018-01-04 | Named entity recognition method, device and readable storage medium for counterlearning on crowdsourced data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108228564B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109063927A (en) * | 2018-08-28 | 2018-12-21 | 成都信息工程大学 | A kind of microblogging transfer amount prediction technique based on TS-LSTM and DNN |
| CN109086814A (en) * | 2018-07-23 | 2018-12-25 | 腾讯科技（深圳）有限公司 | A kind of data processing method, device and the network equipment |
| CN109583497A (en) * | 2018-11-29 | 2019-04-05 | 中电科嘉兴新型智慧城市科技发展有限公司 | A kind of confrontation generates the quality of data rule generation method and system of network intelligence judgement |
| CN110134959A (en) * | 2019-05-15 | 2019-08-16 | 第四范式（北京）技术有限公司 | Named Entity Extraction Model training method and equipment, information extraction method and equipment |
| CN110750523A (en) * | 2019-09-12 | 2020-02-04 | 苏宁云计算有限公司 | Data annotation method, system, computer equipment and storage medium |
| CN111222339A (en) * | 2020-01-13 | 2020-06-02 | 华南理工大学 | Medical consultation named entity identification method based on anti-multitask learning |
| CN112349370A (en) * | 2020-11-05 | 2021-02-09 | 大连理工大学 | Electronic medical record corpus construction method based on confrontation network and crowdsourcing |
| CN117436449A (en) * | 2023-11-01 | 2024-01-23 | 哈尔滨工业大学 | Crowd-sourced named entity recognition model and system based on multi-source domain adaptation and reinforcement learning |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106557462A (en) * | 2016-11-02 | 2017-04-05 | 数库（上海）科技有限公司 | Name entity recognition method and system |
| CN106569998A (en) * | 2016-10-27 | 2017-04-19 | 浙江大学 | Text named entity recognition method based on Bi-LSTM, CNN and CRF |
| US20170278514A1 (en) * | 2016-03-23 | 2017-09-28 | Amazon Technologies, Inc. | Fine-grained natural language understanding |
-
2018
- 2018-01-04 CN CN201810007733.6A patent/CN108228564B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170278514A1 (en) * | 2016-03-23 | 2017-09-28 | Amazon Technologies, Inc. | Fine-grained natural language understanding |
| CN106569998A (en) * | 2016-10-27 | 2017-04-19 | 浙江大学 | Text named entity recognition method based on Bi-LSTM, CNN and CRF |
| CN106557462A (en) * | 2016-11-02 | 2017-04-05 | 数库（上海）科技有限公司 | Name entity recognition method and system |
Non-Patent Citations (3)
| Title |
|---|
| GUL KHAN SAFI QAMAS 等: "基于深度神经网络的命名实体识别", 《技术研究》 * |
| YAOSHENG YANG等: "Adversarial Learning for Chinese NER from Crowd Annotations", 《PROCEEDINGS OF AAAI-2018》 * |
| 杨耀晟: "基于噪音训练数据的中文命名实体识别研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109086814A (en) * | 2018-07-23 | 2018-12-25 | 腾讯科技（深圳）有限公司 | A kind of data processing method, device and the network equipment |
| CN109086814B (en) * | 2018-07-23 | 2021-05-14 | 腾讯科技（深圳）有限公司 | Data processing method and device and network equipment |
| CN109063927B (en) * | 2018-08-28 | 2021-12-07 | 成都信息工程大学 | Microblog forwarding capacity prediction method based on TS-LSTM and DNN |
| CN109063927A (en) * | 2018-08-28 | 2018-12-21 | 成都信息工程大学 | A kind of microblogging transfer amount prediction technique based on TS-LSTM and DNN |
| CN109583497A (en) * | 2018-11-29 | 2019-04-05 | 中电科嘉兴新型智慧城市科技发展有限公司 | A kind of confrontation generates the quality of data rule generation method and system of network intelligence judgement |
| CN110134959A (en) * | 2019-05-15 | 2019-08-16 | 第四范式（北京）技术有限公司 | Named Entity Extraction Model training method and equipment, information extraction method and equipment |
| CN110134959B (en) * | 2019-05-15 | 2023-10-20 | 第四范式（北京）技术有限公司 | Named entity recognition model training method and equipment, and information extraction method and equipment |
| CN110750523A (en) * | 2019-09-12 | 2020-02-04 | 苏宁云计算有限公司 | Data annotation method, system, computer equipment and storage medium |
| CN111222339B (en) * | 2020-01-13 | 2023-05-23 | 华南理工大学 | Medical consultation named entity recognition method based on countermeasure multitask learning |
| CN111222339A (en) * | 2020-01-13 | 2020-06-02 | 华南理工大学 | Medical consultation named entity identification method based on anti-multitask learning |
| CN112349370A (en) * | 2020-11-05 | 2021-02-09 | 大连理工大学 | Electronic medical record corpus construction method based on confrontation network and crowdsourcing |
| CN112349370B (en) * | 2020-11-05 | 2023-11-24 | 大连理工大学 | Electronic medical record corpus construction method based on countermeasure network and crowdsourcing |
| CN117436449A (en) * | 2023-11-01 | 2024-01-23 | 哈尔滨工业大学 | Crowd-sourced named entity recognition model and system based on multi-source domain adaptation and reinforcement learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108228564B (en) | 2021-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108228564A (en) | The name entity recognition method of confrontation study is carried out in crowdsourcing data | |
| CN110188331B (en) | Model training method, dialogue system evaluation method, device, equipment and storage medium | |
| CN109522553B (en) | Named entity identification method and device | |
| CN111737476B (en) | Text processing method and device, computer readable storage medium and electronic equipment | |
| CN111046661B (en) | Reading understanding method based on graph convolution network | |
| CN110633730A (en) | Deep learning machine reading understanding training method based on course learning | |
| CN111008293A (en) | Visual question-answering method based on structured semantic representation | |
| Sonkar et al. | qdkt: Question-centric deep knowledge tracing | |
| CN106156003A (en) | A kind of question sentence understanding method in question answering system | |
| CN112613314A (en) | Electric power communication network knowledge graph construction method based on BERT model | |
| CN111563146B (en) | Difficulty controllable problem generation method based on reasoning | |
| US20210174204A1 (en) | System and method for natural language processing using neural network | |
| US20220261546A1 (en) | Method and apparatus for selecting answers to idiom fill-in-the-blank questions, and computer device | |
| CN111222318B (en) | Trigger word recognition method based on double-channel bidirectional LSTM-CRF network | |
| CN111897944A (en) | Knowledge map question-answering system based on semantic space sharing | |
| CN106202030A (en) | A kind of rapid serial mask method based on isomery labeled data and device | |
| CN107544960A (en) | A kind of inference method activated based on Variable-Bindings and relation | |
| CN114491039B (en) | Primitive learning few-sample text classification method based on gradient improvement | |
| CN112651324A (en) | Method and device for extracting semantic information of video frame and computer equipment | |
| CN109086463A (en) | A kind of Ask-Answer Community label recommendation method based on region convolutional neural networks | |
| CN110795934B (en) | Sentence analysis model training method and device and sentence analysis method and device | |
| CN113220908A (en) | Knowledge graph matching method and device | |
| CN111428502A (en) | Named entity labeling method for military corpus | |
| CN114154839A (en) | Course recommendation method based on online education platform data | |
| Wang et al. | Research and construction of junior high school subject Q&A system model based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |