CN109902756A - A kind of crowdsourcing mechanism auxiliary sort method and system based on Active Learning - Google Patents
A kind of crowdsourcing mechanism auxiliary sort method and system based on Active Learning Download PDFInfo
- Publication number
- CN109902756A CN109902756A CN201910172856.XA CN201910172856A CN109902756A CN 109902756 A CN109902756 A CN 109902756A CN 201910172856 A CN201910172856 A CN 201910172856A CN 109902756 A CN109902756 A CN 109902756A
- Authority
- CN
- China
- Prior art keywords
- training data
- training
- data
- module
- disaggregated model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007246 mechanism Effects 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000012549 training Methods 0.000 claims abstract description 154
- 238000013145 classification model Methods 0.000 claims abstract description 19
- 238000012360 testing method Methods 0.000 claims abstract description 12
- 238000002372 labelling Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 6
- 238000013441 quality evaluation Methods 0.000 claims description 6
- 238000001746 injection moulding Methods 0.000 claims 2
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000009193 crawling Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention proposes a kind of crowdsourcing mechanism auxiliary sort method based on Active Learning, comprising: the answer of collection is formed answer pair, composing training data；One group of training data input training set is selected from training data, obtains preliminary classification model；Crowdsourcing mark is carried out by the training data that selection strategy picks out high-class contribution degree；The data of mark are updated into training set, new disaggregated model is obtained；If disaggregated model precision is not up to setting value, continues through selection strategy and update training set, the new disaggregated model of training；If disaggregated model precision reaches setting value, stop updating；By test set input precision disaggregated model up to standard, marking sequence is carried out to result；Representative sample is carried out crowdsourcing mark and reduces artificial mark cost by the present invention, improves efficiency of algorithm.
Description
Technical field
The present invention relates to big data sorting algorithm fields more particularly to a kind of crowdsourcing mechanism based on Active Learning to assist row
Sequence method and system.
Background technique
Crowdsourcing mechanism is the product of information age, and crowdsourcing can make full use of the creativeness of the masses, low in cost, deep accredited
The favor of Xi Hua company.But excessive monistic data mark task is not only expended the time, can also be consumed by the way of crowdsourcing
Take a large amount of human resources, influences the efficiency of work.
In addition, some sorting algorithms lack Active Learning mechanism, causes nicety of grading and efficiency relatively low, how both to save people
Power can improve nicety of grading again becomes a great problem of current big data field.
Summary of the invention
In view of the above problem of the existing technology, the present invention proposes a kind of crowdsourcing mechanism auxiliary row based on Active Learning
Sequence method and system mainly solves the problems, such as that existing algorithm classification is inconvenient.
To achieve the goals above and other purposes, the technical solution adopted by the present invention are as follows.
A kind of crowdsourcing mechanism auxiliary sort method based on Active Learning, comprising:
The answer of collection forms answer pair, composing training data；
One group of training data input training set is selected from training data, obtains preliminary classification model；
The training data for obtaining high-class contribution degree carries out crowdsourcing mark；
The data of mark are updated into training set, new disaggregated model is obtained；
If the new disaggregated model precision is not up to setting value, continue to update training set, the new disaggregated model of training；
If updated disaggregated model precision reaches setting value, stop updating；
By test set input precision disaggregated model up to standard, marking sequence is carried out to result.
Optionally, described that one group of training data input training set is selected to specifically include to the trained number from training data
Feature vector is obtained according to feature extraction is carried out, passes through feature vector train classification models.To feature vector carry out dimensionality reduction operation or
Other reduce complicated processing, and the training effectiveness of model can be improved.
Optionally, the disaggregated model includes multiple sub-classifiers, and the sub-classifier selects high score from training data
The training data of class contribution degree.Ballot mode can be used in evaluation method, and voting results result is used to judge the classification of training data
Contribution degree.
Optionally, the judgment criteria of the classification contribution degree is that the inconsistent accounting of sub-classifier voting results is higher
Training data, classification contribution degree it is higher.The complexity of disaggregated model can be improved for training in the high data of classification contribution degree
And precision.
Optionally, the collected answer does not repeat in pairs each other.Prevent duplicate training data improve artificial mark at
Originally it and has a negative impact to disaggregated model.
Optionally, the crowdsourcing mechanism is labeled training data, and training data is sent to specific people to each
Quality of the answer in locating answer pair is evaluated, and two classification quality labels are formed.The specific people is expert group,
The data of high-class contribution degree are manually marked, the accuracy of answer quality evaluation is improved, reduce erroneous human's mark
Interference.Quality tab is for being ranked up marking to training data.
A kind of crowdsourcing mechanism auxiliary ordering system based on Active Learning, comprising:
Training dataset collects answer, constructs answer to as training data；
Labeling module is labeled training data；
Training module carries out feature extraction to labeled data, obtains feature vector for train classification models；
Selecting module selects one group of training data for evaluation module selection from training data concentration；
Module, including multiple sub-classifiers are evaluated, the data of high-class contribution degree are selected from one group of training data, is exported
To labeling module；
Marking sorting module；Output result for reaching the categorization module of setting accuracy to training precision scores,
And it sorts according to score.
Optionally, each decision of the selecting module obtains one group of data from training data；By evaluation module to instruction
The quality for practicing data is evaluated, and obtains the high training data of the inconsistent accounting of quality evaluation as high-class contribution degree
Data.
Optionally, the labeling module is labeled using crowdsourcing mechanism, the high-class contribution degree that selecting module is obtained
Data are sent to specific people and are manually marked, and the data of mark are sent to training module, update disaggregated model.
Optionally, it if disaggregated model nicety of grading is below standard, evaluates module and obtains training data to selecting module again
Quality evaluation is carried out, high-class contribution degree data are sent to labeling module, the data after mark are used to update training module
Obtain new disaggregated model, iteration movement, until updated disaggregated model nicety of grading reaches setting value.To initial instruction
Practice model and be iterated update, the training data of high-class contribution degree is constantly used for train classification models, classification can be improved
The precision and efficiency of model.
As described above, a kind of crowdsourcing mechanism auxiliary sort method and system based on Active Learning of the present invention, has as follows
Beneficial effect.
The sample for selecting high-class contribution degree carries out crowdsourcing mark, reduces the workload manually marked, improves work effect
Rate.
Detailed description of the invention
Fig. 1 is the flow chart that a kind of crowdsourcing mechanism based on Active Learning of the present invention assists sort method.
Specific embodiment
Illustrate embodiments of the present invention below by way of specific specific example, those skilled in the art can be by this specification
Other advantages and efficacy of the present invention can be easily understood for disclosed content.The present invention can also pass through in addition different specific realities
The mode of applying is embodied or practiced, the various details in this specification can also based on different viewpoints and application, without departing from
Various modifications or alterations are carried out under spirit of the invention.It should be noted that in the absence of conflict, following embodiment and implementation
Feature in example can be combined with each other.
It should be noted that illustrating the basic structure that only the invention is illustrated in a schematic way provided in following embodiment
Think, only shown in schema then with related component in the present invention rather than component count, shape and size when according to actual implementation
Draw, when actual implementation kenel, quantity and the ratio of each component can arbitrarily change for one kind, and its assembly layout kenel
It is likely more complexity.
Referring to Fig. 1, the present invention provides a kind of crowdsourcing mechanism auxiliary sort method based on Active Learning, comprising:
The answer of collection is formed into answer pair, composing training data；
One group of training data input training set is selected from training data, obtains preliminary classification model；
The training data for obtaining high-class contribution degree carries out crowdsourcing mark；
The data of mark are updated into training set, new disaggregated model is obtained；
If the new disaggregated model precision is not up to setting value, continue to update training set, the new disaggregated model of training；
If updated disaggregated model precision reaches setting value, stop updating；
By test set input precision disaggregated model up to standard, marking sequence is carried out to result.
In one embodiment, answer data can be obtained by way of the Ask-Answer Communities such as crawling Baidu, knowing, will be collected
Answer form answer list.
Answer in answer list is formed into answer pair, and guarantees that the data of each answer centering do not repeat in pairs each other.
Obtained answer is to as training data.
The training data of selection setting quantity from training data, and selected data are inputted in training set, to selected
Training data carry out feature extraction, obtain feature vector.In one embodiment, dimensionality reduction can be carried out to feature vector, reduced
The complexity of feature vector improves training effectiveness.
The corresponding feature vector of obtained training data is used for train classification models, obtains preliminary classification model.
The disaggregated model includes several sub-classifiers, and the sub-classifier is according to selection strategy from one group of training
The training data of high-class contribution degree is selected in data.
The judgment criteria of the high-class contribution degree is the higher instruction of the inconsistent accounting of sub-classifier voting results
Practice data, classification contribution degree is higher.This is that the classification information that contains in view of the sample of easier classification error is abundanter.For spending
The selection strategy for measuring this inconsistent degree includes: ballot entropy (Vote entropy), KL divergence (Kull-back-Leibler
divergence)、JS(Jensen-Shannon)。
In one embodiment, high-class contribution can be selected in such a way that sub-classifier is voted using ballot entropy strategy
The training data of degree.The ballot entropy of input training data e is defined as follows.
Wherein V (c, e) indicates that training data e is classified as the quantity of the sub-classifier of classification c.K indicates sub-classifier
Total quantity, C are class categories sum, and c is specific classification.
It is least consistent that several voting results in training data are selected by selection strategy by the sub-classifier
Training data is sent to specific personnel and is manually marked.Here specific people is generally expert group.Due to training data
Contribution degree of classifying is high, and common mark personnel are difficult to provide the judgement of accurate training data quality, and the mark of mistake will be right
The training of disaggregated model generates interference, and therefore, setting expert group is manually marked, and evaluates the quality of training data,
Be conducive to improve training precision and efficiency.
Due to marking the difference of personnel, the most instruction of poll is selected using voting mechanism for controversial annotation results
Practice the corresponding mark label of data, the final mass label as training data.
If training data is answer to<Ai, Aj>, enables<Ai, Aj>=1 indicate the quality of answer Ai higher than answer Aj,<Ai,
Aj >=0 expression answer Ai is of low quality in answer Aj.
The training data of mark is inputted into training set, the feature vector of training set is updated, and updated feature
Vector train classification models obtain new disaggregated model.Judge whether the precision of new disaggregated model reaches setting value.
If disaggregated model nicety of grading is below standard, sub-classifier carries out quality evaluation to the training data newly selected again,
High-class contribution degree data are sent to expert group manually to be marked, the data after mark are used to update training set and are obtained newly
Disaggregated model, iteration movement, until updated disaggregated model nicety of grading reaches setting value.To initial training model
It is iterated update and the essence of disaggregated model can be improved constantly according to the training data train classification models of high-class contribution degree
Degree and efficiency.
Test set is constructed, the method for constructing test set is identical as the building method of training dataset, is not repeating here.
Test set is predicted using trained disaggregated model, 0 or 1 quality tab will be obtained after prediction, by 0 or 1
The score between [0,100] is converted to, is given a mark by scoring functions to classification results.In one embodiment, the marking
Function is as follows.
WhereinFor the classification results of answer pair,For the final mass score of answer At.
After obtaining the quality score of each test answers, answer is ranked up according to quality score.
The present invention provides a kind of crowdsourcing mechanism auxiliary ordering system based on Active Learning, comprising:
Training dataset collects answer, constructs answer to as training data；
Labeling module is labeled training data；
Training module carries out feature extraction to labeled data, obtains feature vector for train classification models；
Selecting module selects one group of training data for evaluation module selection from training data concentration；
Module, including multiple sub-classifiers are evaluated, the data of high-class contribution degree are selected from one group of training data, is exported
To labeling module；
Marking sorting module；Output result for reaching the categorization module of setting accuracy to training precision scores,
And it sorts according to score.
In one embodiment, answer data can be obtained by way of the Ask-Answer Communities such as crawling Baidu, knowing, will be collected
Answer form answer list.
Answer in answer list is formed into answer pair, and guarantees that the data of each answer centering do not repeat in pairs each other.
Obtained answer is to as training data, composing training data set.
The training data of selection setting quantity is concentrated from training data, and selected data are inputted in training module, it is right
Selected training data carries out feature extraction, obtains feature vector.In one embodiment, dimensionality reduction can be carried out to feature vector,
The complexity of feature vector is reduced, training effectiveness is improved.
The corresponding feature vector of obtained training data is used for train classification models, obtains preliminary classification model, is constituted
Evaluate module.
The preliminary classification model includes several sub-classifiers.
The selecting module selects one group of training data from training data concentration every time, and the sub-classifier is according to selection plan
The training data of high-class contribution degree is slightly selected from one group of training data.
The judgment criteria of the high-class contribution degree is the higher instruction of the inconsistent accounting of sub-classifier voting results
Practice data, classification contribution degree is higher.This is that the classification information that contains in view of the sample of easier classification error is abundanter.For spending
The selection strategy for measuring this inconsistent degree includes: ballot entropy (Vote entropy), KL divergence (Kull-back-Leibler
divergence)、JS(Jensen-Shannon)。
In one embodiment, high-class contribution can be selected in such a way that sub-classifier is voted using ballot entropy strategy
The training data of degree.The ballot entropy of input training data e is defined as follows.
Wherein V (c, e) indicates that training data e is classified as the quantity of the sub-classifier of classification c.K indicates sub-classifier
Total quantity, C are class categories sum, and c is specific classification.
It is least consistent that several voting results in training data are selected by selection strategy by the sub-classifier
Training data, enters data into labeling module, and obtained training data is sent to specific personnel and carried out manually by labeling module
Mark.Here specific people is generally expert group.Since training data classification contribution degree is high, common mark personnel be difficult to
The judgement of accurate training data quality out, and the mark of mistake will generate interference to the training of disaggregated model, therefore, setting is special
Family's group is manually marked, and is evaluated the quality of training data, is conducive to improve training precision and efficiency.
Due to marking the difference of personnel, the most instruction of poll is selected using voting mechanism for controversial annotation results
Practice the corresponding mark label of data, the final mass label as training data.
If training data is answer to<Ai, Aj>, enables<Ai, Aj>=1 indicate the quality of answer Ai higher than answer Aj,<Ai,
Aj >=0 expression answer Ai is of low quality in answer Aj.
The training data of mark is inputted into training module, the feature vector in training module is updated, and after update
Feature vector train classification models, obtain new disaggregated model.Judge whether the precision of new disaggregated model reaches setting value.
If disaggregated model nicety of grading is below standard, evaluates module and obtain again quality is carried out to the training data of selecting module
Evaluation, high-class contribution degree data are sent to labeling module, by the data after mark be used to update training module obtain it is new
Disaggregated model, iteration movement, until updated disaggregated model nicety of grading reaches setting value.To initial training model into
Row iteration updates, and the training data of high-class contribution degree is constantly used for train classification models, the essence of disaggregated model can be improved
Degree and efficiency.
Test set is constructed, the method for constructing test set is identical as the building method of training dataset, is not repeating here.
Test set is predicted using trained disaggregated model, 0 or 1 quality tab will be obtained after prediction, by 0 or 1
The score between [0,100] is converted to, is given a mark by scoring functions to classification results.In one embodiment, the marking
Function is as follows.
WhereinFor the classification results of answer pair,For the final mass score of answer At.
After obtaining the quality score of each test answers, answer is ranked up according to quality score.
In conclusion a kind of crowdsourcing mechanism auxiliary sort method and system based on Active Learning of the present invention, introduces selection
Strategy selects the data of high-class contribution degree as training sample, improves training effectiveness while reducing trained cost, subtracts
Few artificial mark cost.So the present invention effectively overcomes various shortcoming in the prior art and has high industrial utilization value.
The above-described embodiments merely illustrate the principles and effects of the present invention, and is not intended to limit the present invention.It is any ripe
The personage for knowing this technology all without departing from the spirit and scope of the present invention, carries out modifications and changes to above-described embodiment.Cause
This, institute is complete without departing from the spirit and technical ideas disclosed in the present invention by those of ordinary skill in the art such as
At all equivalent modifications or change, should be covered by the claims of the present invention.
Claims (10)
1. a kind of crowdsourcing mechanism based on Active Learning assists sort method characterized by comprising
The answer of collection forms answer pair, composing training data；
One group of training data input training set is selected from training data, obtains preliminary classification model；
The training data for obtaining high-class contribution degree carries out crowdsourcing mark；
The data of mark are updated into training set, new disaggregated model is obtained；
If the new disaggregated model precision is not up to setting value, continue to update training set, the new disaggregated model of training；If more
Disaggregated model precision after new reaches setting value, then stops updating；
By test set input precision disaggregated model up to standard, marking sequence is carried out to result.
2. crowdsourcing mechanism according to claim 1 based on Active Learning assists sort method, which is characterized in that it is described from
It selects one group of training data input training set to specifically include in training data and spy is obtained to training data progress feature extraction
Vector is levied, feature vector train classification models are passed through.
3. the crowdsourcing mechanism according to claim 1 based on Active Learning assists sort method, which is characterized in that described point
Class model includes multiple sub-classifiers, and the sub-classifier selects the training data of high-class contribution degree from training data.
4. the crowdsourcing mechanism according to claim 1 based on Active Learning assists sort method, which is characterized in that described point
The judgment criteria of class contribution degree is the higher training data of the inconsistent accounting of sub-classifier voting results, classification contribution
It spends higher.
5. the crowdsourcing mechanism according to claim 1 based on Active Learning assists sort method, which is characterized in that the receipts
The answer of collection does not repeat in pairs each other.
6. the crowdsourcing mechanism according to claim 1 based on Active Learning assists sort method, which is characterized in that the crowd
Machine contracting system is labeled training data, and training data is sent to matter of the specific people to each answer in locating answer pair
Amount quality is evaluated, and two classification quality labels are formed.
7. a kind of crowdsourcing mechanism based on Active Learning assists ordering system characterized by comprising
Training dataset collects answer, constructs answer to as training data；
Labeling module is labeled training data；
Training module carries out feature extraction to labeled data, obtains feature vector for train classification models；
Selecting module selects one group of training data for evaluation module selection from training data concentration；
Module, including multiple sub-classifiers are evaluated, the data of high-class contribution degree are selected from one group of training data, output is to mark
Injection molding block；
Marking sorting module；Output result for reaching the categorization module of setting accuracy to training precision scores, and presses
It sorts according to score.
8. the crowdsourcing mechanism according to claim 7 based on Active Learning assists ordering system, which is characterized in that the choosing
It selects each decision of module and obtains one group of data from training data；The quality of training data is evaluated by evaluating module,
Obtain data of the high training data of the inconsistent accounting of quality evaluation as high-class contribution degree.
9. the crowdsourcing mechanism according to claim 7 based on Active Learning assists ordering system, which is characterized in that the mark
Injection molding block is labeled using crowdsourcing mechanism, and the high-class contribution degree data that selecting module obtains are sent to specific people and are carried out
Artificial mark, and the data of mark are sent to training module, update disaggregated model.
10. the crowdsourcing mechanism according to claim 7 based on Active Learning assists ordering system, which is characterized in that if point
Class model nicety of grading is below standard, then evaluates module and obtain again and carry out quality evaluation to the training data of selecting module, by high score
Class contribution degree data are sent to labeling module, are used to update training module for the data after mark and obtain new disaggregated model, weight
Multiple iterative action, until updated disaggregated model nicety of grading reaches setting value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910172856.XA CN109902756A (en) | 2019-03-07 | 2019-03-07 | A kind of crowdsourcing mechanism auxiliary sort method and system based on Active Learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910172856.XA CN109902756A (en) | 2019-03-07 | 2019-03-07 | A kind of crowdsourcing mechanism auxiliary sort method and system based on Active Learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109902756A true CN109902756A (en) | 2019-06-18 |
Family
ID=66946572
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910172856.XA Pending CN109902756A (en) | 2019-03-07 | 2019-03-07 | A kind of crowdsourcing mechanism auxiliary sort method and system based on Active Learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109902756A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110443292A (en) * | 2019-07-24 | 2019-11-12 | 北京交通大学 | The crowdsourcing answer decision-making technique of more influence factors |
| CN110795072A (en) * | 2019-10-16 | 2020-02-14 | 北京航空航天大学 | Crowd-sourcing competition platform framework system and method based on crowd intelligence |
| CN110909880A (en) * | 2019-11-20 | 2020-03-24 | 北京航空航天大学 | Crowdsourcing worker performance prediction method based on deep knowledge tracking |
| CN111030764A (en) * | 2019-10-31 | 2020-04-17 | 武汉大学 | Crowdsourcing user information age management algorithm based on random game online learning |
| CN112562153A (en) * | 2020-11-04 | 2021-03-26 | 重庆恢恢信息技术有限公司 | Construction site behavior personnel optimization method based on intelligent cloud platform |
| CN114611715A (en) * | 2022-05-12 | 2022-06-10 | 之江实验室 | Crowd-sourcing active learning method and device based on annotator reliability time sequence modeling |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446287A (en) * | 2016-11-08 | 2017-02-22 | 北京邮电大学 | Answer aggregation method and system facing crowdsourcing scene question-answering system |
| CN107169001A (en) * | 2017-03-31 | 2017-09-15 | 华东师范大学 | A kind of textual classification model optimization method based on mass-rent feedback and Active Learning |
| CN108629183A (en) * | 2018-05-14 | 2018-10-09 | 南开大学 | Multi-model malicious code detecting method based on Credibility probability section |
| CN109389180A (en) * | 2018-10-30 | 2019-02-26 | 国网四川省电力公司广元供电公司 | A power equipment image-recognizing method and inspection robot based on deep learning |
-
2019
- 2019-03-07 CN CN201910172856.XA patent/CN109902756A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446287A (en) * | 2016-11-08 | 2017-02-22 | 北京邮电大学 | Answer aggregation method and system facing crowdsourcing scene question-answering system |
| CN107169001A (en) * | 2017-03-31 | 2017-09-15 | 华东师范大学 | A kind of textual classification model optimization method based on mass-rent feedback and Active Learning |
| CN108629183A (en) * | 2018-05-14 | 2018-10-09 | 南开大学 | Multi-model malicious code detecting method based on Credibility probability section |
| CN109389180A (en) * | 2018-10-30 | 2019-02-26 | 国网四川省电力公司广元供电公司 | A power equipment image-recognizing method and inspection robot based on deep learning |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110443292A (en) * | 2019-07-24 | 2019-11-12 | 北京交通大学 | The crowdsourcing answer decision-making technique of more influence factors |
| CN110443292B (en) * | 2019-07-24 | 2021-12-07 | 北京交通大学 | Multi-influence-factor crowdsourcing answer decision method |
| CN110795072A (en) * | 2019-10-16 | 2020-02-14 | 北京航空航天大学 | Crowd-sourcing competition platform framework system and method based on crowd intelligence |
| CN110795072B (en) * | 2019-10-16 | 2021-10-29 | 北京航空航天大学 | Crowd-sourcing competition platform framework system and method based on crowd intelligence |
| CN111030764A (en) * | 2019-10-31 | 2020-04-17 | 武汉大学 | Crowdsourcing user information age management algorithm based on random game online learning |
| CN111030764B (en) * | 2019-10-31 | 2021-02-02 | 武汉大学 | Crowdsourcing user information age management method based on random game online learning |
| CN110909880A (en) * | 2019-11-20 | 2020-03-24 | 北京航空航天大学 | Crowdsourcing worker performance prediction method based on deep knowledge tracking |
| CN110909880B (en) * | 2019-11-20 | 2022-10-21 | 北京航空航天大学 | Crowdsourcing task prediction method based on deep knowledge tracking |
| CN112562153A (en) * | 2020-11-04 | 2021-03-26 | 重庆恢恢信息技术有限公司 | Construction site behavior personnel optimization method based on intelligent cloud platform |
| CN112562153B (en) * | 2020-11-04 | 2023-05-02 | 重庆恢恢信息技术有限公司 | Building site behavior personnel optimization method based on intelligent cloud platform |
| CN114611715A (en) * | 2022-05-12 | 2022-06-10 | 之江实验室 | Crowd-sourcing active learning method and device based on annotator reliability time sequence modeling |
| CN114611715B (en) * | 2022-05-12 | 2022-08-23 | 之江实验室 | Crowd-sourcing active learning method and device based on annotator reliability time sequence modeling |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902756A (en) | A kind of crowdsourcing mechanism auxiliary sort method and system based on Active Learning | |
| CN110378818B (en) | Personalized exercise recommendation method, system and medium based on difficulty | |
| Yao et al. | Text classification model based on fasttext | |
| CN109918642A (en) | The sentiment analysis method and system of Active Learning frame based on committee's inquiry | |
| CN103559504A (en) | Image target category identification method and device | |
| CN106202891A (en) | A kind of big data digging method towards Evaluation of Medical Quality | |
| CN107545038A (en) | A kind of file classification method and equipment | |
| CN108536784A (en) | Comment information sentiment analysis method, apparatus, computer storage media and server | |
| CN106960017A (en) | E-book is classified and its training method, device and equipment | |
| CN109214444B (en) | Game anti-addiction determination system and method based on twin neural network and GMM | |
| CN107885849A (en) | A kind of moos index analysis system based on text classification | |
| CN109409530A (en) | A kind of intelligent decision making model and method for mahjong | |
| CN109711424A (en) | A kind of rule of conduct acquisition methods, device and equipment based on decision tree | |
| CN112148994B (en) | Information push effect evaluation method and device, electronic equipment and storage medium | |
| CN108509588B (en) | Lawyer evaluation method and recommendation method based on big data | |
| Lisiński et al. | Principles of the application of strategic planning methods | |
| Yan et al. | The pyramidsnapshot challenge: Understanding student process from visual output of programs | |
| CN109816010A (en) | A kind of CART increment study classification method based on selective ensemble for flight delay prediction | |
| WO2020135054A1 (en) | Method, device and apparatus for video recommendation and storage medium | |
| CN105931055A (en) | Service provider feature modeling method for crowdsourcing platform | |
| CN112785156B (en) | Industrial collar and sleeve identification method based on clustering and comprehensive evaluation | |
| CN117892220A (en) | Error question classification analysis and promotion method based on big data | |
| Lonij et al. | Open-world visual recognition using knowledge graphs | |
| Suh et al. | Creativity, job performance and their correlates: a comparison between nonprofit and profit‐driven organizations | |
| CN109033378A (en) | A kind of application method of Zero-shot Learning in intelligent customer service system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190618 |