CN110096569A - A kind of crowd survey personnel set recommended method - Google Patents
A kind of crowd survey personnel set recommended method Download PDFInfo
- Publication number
- CN110096569A CN110096569A CN201910280296.XA CN201910280296A CN110096569A CN 110096569 A CN110096569 A CN 110096569A CN 201910280296 A CN201910280296 A CN 201910280296A CN 110096569 A CN110096569 A CN 110096569A
- Authority
- CN
- China
- Prior art keywords
- personnel
- crowd
- survey
- many
- past
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/335—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06311—Scheduling, planning or task assignment for a person or group
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Computational Linguistics (AREA)
- Economics (AREA)
- Artificial Intelligence (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Gather recommended methods the invention discloses a kind of many survey personnel, step includes: 1) to survey many of task according to history crowd to survey one technical term libraries of report generation and each crowd observes and predicts and accuses corresponding 5 tuple:<submitter, submission time, whether be defect, whether be duplicate reports, technical term list>；2) crowd for surveying task based on history crowd observes and predicts announcement, generates personnel's experience and domain background information；3) corresponding pretreated new many survey tasks, generate 2 tuples of new many survey tasks:<issuing time, the list of requirement technology term>；4) it is based on personnel's experience and domain background, computing staff's Flaw detectability, personnel's activity, personnel and Xin crowd survey the correlation of task；5) the corresponding recommendation personnel of the new many survey task are generated according to the correlation to gather.The present invention can preferably play many survey personnel efficiency, promote many survey service modes that formation efficiency is excellent, efficiency is high.
Description
Technical field
The invention belongs to field of computer technology, are related to software testing technology, and especially crowdsourcing software test is (referred to as many
Survey), for surveying the suitably many survey personnel of one group of task recommendation for new crowd, the defects detection rate of test is improved, for a kind of many survey people
Member's set recommended method.
Background technique
Crowd, which surveys, to be referred to before software is formally issued, and test assignment is published to many survey platforms on internet by software company,
Many survey personnel on platform execute test, and crowd is submitted to observe and predict announcement.Since software error will lead to customer churn and economic loss,
In the case where software company's professional test personnel's relative shortage, many survey technologies are researched and developed or are updated in current internet company software
It is widely adopted in the process.
Since professional software test background, ability be not irregular mostly by many survey personnel, different personnel survey in crowd to be appointed
Performance difference in business is obvious；Inappropriate crowd survey personnel may omit defect or submit repeated defects, cause resource unrestrained
Take.Therefore how to be new many suitably many survey personnel of one group of task recommendations are surveyed, reduces repeated defects, improves defects detection rate, more
It is vital for playing the efficiency of personnel well.
The personnel that other kinds of soft project activity is related to recommend problem (such as recommending developer, defect repair people),
It is independent of each other between the personnel of recommendation, task can be executed there is usually one people or multiple personnel execute task most respectively
One result of selection eventually；But for many survey tasks, need that one group of crowd is recommended to survey personnel to complete task, the performance between personnel jointly
It is interactional, the final test result of their performance co-determination.This is because test assignment requires to reach as far as possible
High program coverage, so many survey personnel answer the covering of program code between personnel it can be found that while defect
The coincidence as few as possible improves defects detection rate to reduce repeated defects.
Existing crowd survey personnel recommend in the related technology, and only the feature of many survey personnel of part modeling and personnel are special
The influence factor for defects detection is levied, the present invention passes through more fully many survey personnel modelings, portrays many survey defects detections of influence
Personnel characteristics, be respectively formed based on accuracy and based on multifarious personnel sort, pass through balanced personnel's accuracy and multiplicity
Property hybrid-sorting policy recommendation personnel set, be able to ascend defects detection rate.
Summary of the invention
The problem to be solved in the present invention is: proposing that a kind of many survey personnel gather recommended method, surveys task recommendation one for new crowd
The many survey personnel of group, improve the defects detection rate of test.
Modeling personnel experience and domain background are accused the technical solution of the present invention is as follows: observing and predicting based on history crowd, is based on this, is calculated
Personnel's Flaw detectability, personnel's activity, personnel and Xin crowd survey the correlation of task, generate the personnel row based on accuracy
Sequence；Based on personnel's domain background, generates and sorted based on multifarious personnel；Pass through balanced personnel's accuracy and multifarious mixing
Ordering strategy, for new many survey task recommendations, one group of crowd surveys personnel；Method flow of the invention as shown in Figure 1, the specific steps are that:
1) it collects and the crowd for pre-processing history crowd survey task observes and predicts announcement, and obtain technical term library, including following son
Step:
The issuing time that each history crowd surveys task 1a) is obtained, obtains following attributes that each crowd observes and predicts announcement: submitter,
Submission time, whether be defect, whether be duplicate reports, report natural language description；
Natural language description 1b) based on all reports, obtains technical term library；Firstly, the natural language to report is retouched
It states and carries out participle operation, be divided into independent word；Secondly, calculating the document frequencies of all words, (each word is more
A few crowd observes and predicts to be occurred in announcement)；Again, n% after the word and document frequency of m% (such as 5%) is filtered out before document frequency
The word of (such as 5%), remaining word are technical term library；Filter out before document frequency 5% word be because they
It appears in many documents, hardly there is distinction, filter out after document frequency 5% word similarly because of these words
Language can hardly bring distinction information.
1c) natural language description after each report participle is filtered based on technical term library, filters out and does not go out
Word in present technical term library, obtains the technical term list of each report；
History crowd 1d) is observed and predicted into announcement and is expressed as 5 tuples: <submitter, submission time, whether be defect, whether be repeat report
Accuse, technical term list >；
2) it collects and pre-processes new many survey tasks, including following sub-step:
2a) obtain following attributes of new many survey tasks: the natural language description of issuing time, testing requirement；
Participle operation 2b) is carried out to the natural language description of testing requirement, independent word is divided into, obtains demand
Word list；
It 2c) is filtered based on word list of the technical term library to demand, filters out and do not appear in technical term library
In word, obtain the technical term list of demand.
It is 2d) 2 tuples:<issuing time, the list of requirement technology term>by new many task presentations of surveying；
3) announcement is observed and predicted based on history crowd, models personnel's experience and domain background, including following sub-step:
Announcement 3a) is observed and predicted to history crowd according to submitter to be grouped, and all history crowds of the same submitter are observed and predicted into announcement
It is classified as one group；One group of history crowd corresponding for each submitter observes and predicts announcement, extracts following 21 in 5 tuples reported from these
Feature, for portraying personnel's experience.
Submitter and technical term list 3b) are extracted from 5 tuples that history crowd observes and predicts announcement, based on submitter to technical term
The technical term list that list merges to obtain each personnel is indicated for portraying personnel's domain background with vector；
4) it is based on personnel's experience and domain background, computing staff's Flaw detectability, personnel's activity, personnel and Xin crowd survey
The correlation of task；Including following sub-step:
It 4a) establishes the Flaw detectability of machine learning model prognosticator: history crowd is surveyed into task as training set.It is right
Whether Mr. Yu crowd surveys all crowds in task A and observes and predicts corresponding 5 tuple of announcement, obtain each crowd and observe and predict the submitter X of announcement, be defect two
A attribute；Many survey report sets that submitter X is submitted before crowd's survey task A issuing time are obtained, and are observed and predicted based on the crowd
Whether accuse the value for gathering relevant 21 features of extraction submitter X experience as independent variable will be defect as dependent variable
It establishes Logic Regression Models, predicts the Flaw detectability of many survey personnel (i.e. whether it can be found that defect)；
4b) the activity of computing staff: the 3rd feature (defect counts that the past submits, the note of many survey personnel's experiences are obtained
For bugNum) and the 21st feature (the last one crowd observes and predicts the interval of the submission time of announcement and the issuing time of new many survey tasks,
It is denoted as intv), personnel's activity is calculated as
4c) computing staff and Xin crowd survey the correlation of task: obtaining the technical term list, new of many survey personnel's domain backgrounds
The requirement technology term list of many survey tasks, personnel and the correlation calculations of task are that the cosine of two technical term lists is similar
Property；
5) correlation that task is surveyed based on personnel's Flaw detectability, personnel's activity, personnel and Xin crowd, is generated based on standard
Personnel's sequence Rank of true propertyacu；It is specific as follows:
Scoreacu(i)=k1 × bugProb (i)+k2 × act (i)+k3 × rev (i)；
Rankacu(i)=Scoreacu(i)-1
Wherein i indicates i-th of candidate many survey personnel, and it is living that bugProb represents personnel's Flaw detectability, act represents personnel
Jump property, rev represent personnel and Xin crowd surveys the correlation of task；The weight of k1, k2, k3 for items in the ranking, and k1+k2+k3
=1；Many survey personnel that -1 times table is shown with maximum accuracy come RankacuFirst；
6) it is based on personnel's domain background, is generated based on multifarious personnel sequence Rankdiv；It is specific as follows:
By Rankacu(i)=1 many survey personnel come RankdivFirst, iteration obtain RankdivSubsequent sequence；
Each candidate personnel are calculated in each iteration, and current Rank is addeddivThe diversity score Score of listdiv(i,Rankdiv), choosing
It obtains a point highest candidate personnel and Rank is addeddivList continues iteration until covering all many survey personnel；
Wherein i indicates i-th of candidate many survey personnel, Scorediv(i,Rankdiv) indicate to enter candidate personnel i to current
RankdivThe diversity score of list, i ∪ RankdivIndicate current RankdivPersonnel and candidate personnel i set, cosine
(x, y) indicates personnel x, the cosine similarity of y domain background；
7) personnel's sequence Rank based on accuracy is integratedacuWith based on multifarious personnel sort Rankdiv, generate mixing
Personnel's sequence Rankcmb, and personnel is recommended to gather.Hybrid-sorting strategy is as follows:
Wherein
Wherein, ScIndicate the threshold value being previously set, i indicates i-th of candidate many survey personnel, Rankcmb(i,Sc) indicate in threshold
Value ScUnder setting, the hybrid-sorting of many survey personnel i, W indicates all candidate many survey personnel.
In simple terms, threshold value S is greater than for accuracy valuecMany survey personnel, hybrid-sorting strategy be based on diversity arrange
The result of sequence；Threshold value S is less than for accuracy valuecMany survey personnel, hybrid-sorting strategy be based on accuracy sequence result；
And all accuracy values are greater than threshold value ScMany survey personnel come accuracy value less than threshold value ScMany survey personnel before；Increase
Big sequence threshold value ScThe accuracy of personnel's recommendation results can be made to increase, diversity is reduced (closer to the sequence based on accuracy)；
Reduce sequence threshold value ScThe diversity of personnel's recommendation results can be made to increase, accuracy is reduced (closer to based on multifarious row
Sequence).
Compared with prior art, the present invention more fully models many survey personnel, portrays the personnel for influencing many survey defects detections
Feature is sorted based on multifarious personnel and is focused wherein main focus of the sequence of the personnel based on accuracy improves defects detection rate
It reduces repeated defects and improves defects detection rate, balanced accuracy and multifarious hybrid-sorting strategy can reduce repetition and lack
It falls into, improve defects detection rate, the better many survey service modes for playing many survey personnel's efficiency, promoting that formation efficiency is excellent, efficiency is high.
Detailed description of the invention
Fig. 1 is that many survey personnel gather recommended method frame diagram.
Specific embodiment
This method is described further below by specific embodiment；
Step 1 collect and pre-process history crowd survey task crowd observe and predict announcement；After the completion of each crowd's survey task, it will receive many surveys
Many crowds that personnel submit observe and predict announcement, and history crowd can be obtained in the database for survey platform of comforming and surveys task and relevant many surveys
Report；5 attributes that the crowd of collection observes and predicts announcement are the common Report Properties of many survey processes；Wherein " submitter " indicates to submit and be somebody's turn to do
Crowd observes and predicts many survey personnel of announcement, is mostly used giver identification (id) to indicate, the effect of the attribute is to correspond to whole historical acts
On each crowd survey personnel, thus the personnel of progress modeling；" submission time " indicates the time for submitting the crowd to observe and predict announcement, the work of the attribute
With being for portraying personnel's experience and computing staff's activity；Crowd observes and predicts the defects of announcement and is only test real concern,
Whether " being defect " indicates that the crowd observes and predicts the defect accused and whether described, which is the important feature for portraying personnel's experience, and
Establish the dependent variable of machine learning model prognosticator's Flaw detectability；Whether " being duplicate reports " indicates that the crowd observes and predicts announcement and is
No and other reports are repeated and are repeated with which report, which is mainly used for preferably portraying personnel's experience；" report
Natural language description " indicates that the crowd observes and predicts content description, such as the description of operating procedure, problem of announcement etc., which is mainly used for
Portray personnel's domain background；
Step 2 collects and pre-processes new many survey tasks；2 attributes of many survey tasks of collection are that many survey processes are common
Attribute；Wherein " issuing time " indicates that new many survey tasks are published to the time on platform, the attribute for portray personnel's experience, with
And computing staff's activity；" natural language description of testing requirement " refers to the description in many survey tasks for test content, leads to
Often comprising tested function declaration；
Step 3 is based on history crowd and observes and predicts announcement modeling personnel experience and domain background；It is main for 21 features of personnel's experience
Be divided into following several classes: " many of participation survey number of items " indicates to submit the number of many survey projects of report；" many of submission survey
Reported number " indicates that the crowd submitted observes and predicts the number (may submit multiple reports in a project) of announcement；" the defect number of submission
It is the number of the report of defect in announcement that mesh ", which indicates that the crowd submitted observes and predicts,；" defective proportion of submission " is with the defect counts submitted
Many divided by submission survey reported number；" the repetition ratio for submitting defect " is the defect counts with repetition factor divided by submission,Wherein r indicates the report that certain many survey personnel submits, and r ' dup indicates the duplicate reports of report r；Such as
Certain many survey personnel has submitted two reports R1, R2, and report R1 has 2 duplicate reports, and report R2 has 6 duplicate reports.So should
It is (1/2+1/6)/2 that many survey personnel, which submit the repetition ratio of defect,；Above-mentioned 5 features be based respectively on over it is all, the past 2
2 moon, past 1 month, past Zhou Jinhang statistics, the activity for the more fine-grained different time of portraying over is to personnel's experience
Influence；In addition, personnel's empirical features further include " when the last one crowd observes and predicts submission time and the publication of new many survey tasks of announcement
Between interval ", issuing time and many submission times that observe and predict announcement of this feature based on new many survey tasks obtain, for modeling personnel
Flaw detectability and computing staff's activity；
Step 4 is based on personnel's experience and domain background, computing staff's Flaw detectability, personnel's activity, personnel and Xin
The correlation of many survey tasks；For personnel's Flaw detectability, history crowd is surveyed into task as training set, extracts expression personnel warp
21 features tested establish machine learning model and predict to obtain；Specifically, history crowd task is surveyed to arrange according to issuing time
Sequence surveys task T for history crowdi(i > 1, because the 1st task does not have historical data), obtains the issuing time of the task, obtains
Each crowd in the task is taken to observe and predict the submitter of announcement, from 1-Ti-1In task, extract in the task issuing time, with the submission
The value of relevant 21 features of people's experience, whether as independent variable, obtaining is defect attribute as dependent variable, establishes logistic regression
Model；Newly many survey tasks 21 features of submitter will be extracted in the same way as test set, whether predict many survey personnel
Defect can be submitted, whether what regression model obtained is the probability of defect as Flaw detectability；
Step 5 surveys the correlation of task based on personnel's Flaw detectability, personnel's activity, personnel and Xin crowd, generates base
In personnel's sequence Rank of accuracyacu；For the Flaw detectability of all personnel, activity, correlation, returned respectively
One changes, and numerical value is mapped to the section [0-1], then calculates accuracy value.Such as Flaw detectability, transfer function isWherein min, max respectively indicate the minimum value and most of the Flaw detectability of all personnel
Big value, bugProb (i) ' and bugProb (i) respectively indicate the defects detection energy after initial Flaw detectability and normalization
Power；Personnel's activity and correlation use identical normalization mode；For weight k1, history crowd can be surveyed task by k2, k3
It marks off verifying to collect, optimal weight is selected on verifying collection, or every importance is judged really based on expertise
Determine weight, weight is bigger, and the effect for indicating this is bigger；
Step 6 is based on personnel's domain background, generates based on multifarious personnel sequence Rankdiv；Selection accuracy row first
Many survey personnel of sequence first, and the ranked personnel's situation of basis of iteration, determine next sequence personnel；The algorithm is greedy
Center algorithm can make entirely sequence change since first selects the difference of personnel；This method is by accuracy value highest
(Rankacu(i)=1 many survey personnel) can obtain preferably effect as first sorted；
The comprehensive personnel's sequence Rank based on accuracy of step 7acuWith based on multifarious personnel sort Rankdiv, generate
Mixing personnel sequence Rankcmb, and personnel is recommended to gather；The advantage of the hybrid-sorting method is that user passes through given threshold Sc,
Accuracy and multifarious relative weighting can be customized；The threshold value can be by marking off verifying for history crowd survey task
Collection selects optimal threshold on verifying collection, or based on expertise for accuracy and multifarious importance judge into
The setting of row threshold value；Following examples intuitively provide the influence of hybrid-sorting strategy and threshold value to sequence (assuming that needing to recommend 6
The many survey personnel of name, deepening background color indicates many survey personnel recommended).
Personnel's sequence based on accuracy:
| 0.9 | 0.82 | 0.79 | 0.71 | 0.65 | 0.64 | 0.58 | 0.53 | 0.48 | 0.39 | Accuracy value |
| 1 | 5 | 8 | 3 | 6 | 2 | 9 | 4 | 7 | 10 | Diversity sequence |
It is sorted based on multifarious personnel:
| 0.9 | 0.64 | 0.71 | 0.53 | 0.82 | 0.65 | 0.48 | 0.79 | 0.58 | 0.39 | Accuracy value |
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | Diversity sequence |
Threshold value Sc=0.6 mixing personnel sequence:
| 0.9 | 0.64 | 0.71 | 0.82 | 0.65 | 0.79 | 0.58 | 0.53 | 0.48 | 0.39 | Accuracy value |
| 1 | 2 | 3 | 5 | 6 | 8 | 9 | 4 | 7 | 10 | Diversity sequence |
Threshold value Sc=0.7 mixing personnel sequence:
| 0.9 | 0.71 | 0.82 | 0.79 | 0.65 | 0.64 | 0.58 | 0.53 | 0.48 | 0.39 | Accuracy value |
| 1 | 3 | 5 | 8 | 6 | 2 | 9 | 4 | 7 | 10 | Diversity sequence |
Although disclosing particular content of the invention for the purpose of illustration, implementing algorithm and attached drawing, its object is to help
Understand the contents of the present invention and implements accordingly, but it will be appreciated by those skilled in the art that: it is of the invention and appended not departing from
Spirit and scope of the claims in, various substitutions, changes and modifications are all possible.The present invention should not be limited to this explanation
Book most preferred embodiment and attached drawing disclosure of that, the scope of protection of present invention are with the range that claims define
It is quasi-.
Claims (10)
1. a kind of crowd survey personnel gather recommended method, step includes:
1) many survey one technical term libraries of report generation of task are surveyed according to history crowd and each crowd observes and predicts and accuses corresponding 5 tuple: < mention
Hand over people, submission time, whether be defect, whether be duplicate reports, technical term list >；
2) crowd for surveying task based on history crowd observes and predicts announcement, generates personnel's experience and domain background information；
3) corresponding pretreated new many survey tasks, generate 2 tuples of new many survey tasks: < issuing time, requirement technology term list
>；
4) it is based on personnel's experience and domain background, computing staff's Flaw detectability, personnel's activity, personnel and Xin crowd survey task
Correlation；
5) the corresponding recommendation personnel of the new many survey task are generated according to the correlation to gather.
2. the method as described in claim 1, which is characterized in that the method for modeling personnel's experience are as follows: surveyed according to submitter to crowd
Report is grouped, and all history crowds of same submitter are observed and predicted announcement and are classified as one group；Then corresponding for each submitter
One group of crowd observes and predicts announcement, observes and predicts extraction feature in 5 tuples of announcement from group crowd, for portraying personnel's experience of the submitter；Modeling
The method of personnel's domain background are as follows: Cong Gezhong, which is observed and predicted, extracts submitter and technical term list in 5 tuples of announcement, be then based on and mention
People is handed over to merge to obtain the technical term list of each personnel to technical term list, for portraying personnel's domain background.
3. method according to claim 2, which is characterized in that the feature of extraction include: participate in the past many survey number of items,
Many weights for surveying reported numbers, the defect counts that the past submits, the defective proportion that the past submits, past submission defect of past submission
Compound proportion, was submitted many many survey reported numbers surveying number of items, submitting for the past 2 months participated in for the past 2 months for the past 2 months
Defect counts, submitted within the past 2 months defective proportion, the past 2 months submit defect repetition ratio, past 1 month participation
Many many survey reported numbers, the defect counts submitted for the past 1 month, the past 1 month surveying number of items, submitting for past 1 month
The defective proportion of submission, many survey number of items, past 2 for submitting the repetition ratio of defect, 2 weeks of past to participate in for the past 1 month
The crowd that a week submits surveys reported numbers, the defect counts that 2 weeks of past submit, the defective proportion of 2 week submissions of past, past 2
The repetition ratio of defect, the last one crowd is submitted to observe and predict between the submission time of announcement and the issuing time of new many survey tasks in a week
Every.
4. the method as described in claim 1, which is characterized in that the method for computing staff's Flaw detectability are as follows: by history crowd
Survey task is as training set；All crowds, which observe and predict, in survey task A many for one accuses corresponding 5 tuple, obtains each crowd and observes and predicts announcement
It submitter X, whether is two attributes of defect；It obtains the crowd that submitter X is submitted before crowd's survey task A issuing time and observes and predicts announcement
Set, and the relevant characteristic value of report set extraction submitter X experience is surveyed based on crowd, as independent variable, will whether be
Defect establishes Logic Regression Models as dependent variable, predicts the Flaw detectability of many survey personnel.
5. the method as described in claim 1, which is characterized in that according to formulaComputing staff's activity act；Its
In, bugNum is the defect counts submitted in the past of many survey personnel, and intv is that the last one crowd of many survey personnel observes and predicts mentioning for announcement
Hand over the interval of time and new many survey task issuing times.
6. the method as described in claim 1, which is characterized in that the method for the correlation of computing staff and Xin crowd's survey task are as follows:
Personnel are determined according to the cosine similarity of the technical term list of many survey personnel and the requirement technology term list of new many survey tasks
With the correlation rev of new many survey tasks.
7. the method as described in claim 1, which is characterized in that in step 5), generate the method that the recommendation personnel gather are as follows:
It is primarily based on the correlation that personnel's Flaw detectability, personnel's activity, personnel and Xin crowd survey task, is generated based on accuracy
Personnel's sequence Rankacu；Based on personnel's domain background, generate based on multifarious personnel sequence Rankdiv；It is then based on personnel row
Sequence RankacuWith personnel's sequence RankdivGenerate mixing personnel sequence Rankcmb, and generate recommendation personnel set.
8. the method for claim 7, which is characterized in that personnel's sequence Rank based on accuracyacu(i)=Scoreacu
(i)-1, Scoreacu(i)=k1 × bugProb (i)+k2 × act (i)+k3 × rev (i), i indicate i-th of candidate many survey personnel,
Personnel's Flaw detectability, the act (i) that bugProb (i) represents many survey personnel i represent personnel's activity of many survey personnel i, rev
(i) correlation of crowd survey personnel i and new many survey tasks are represented；K1, k2, k3 are weight, and k1+k2+k3=1；It generates based on more
Personnel's sequence Rank of sampledivMethod are as follows: by Rankacu(i)=1 many survey personnel come RankdivFirst, iteration
Obtain RankdivSubsequent sequence, each candidate many survey personnel are calculated in each iteration, current Rank are addeddivThe diversity of list
Score Scorediv(i,Rankdiv), Rank is added in many survey personnel of candidate for choosing highest scoringdivList continues iteration until covering
All many survey personnel are covered, personnel's sequence Rank is obtaineddiv；Mixing personnel are ordered asWhereinW(Sc)
=i ∈ W | Scoreacu(i)≥Sc, ScIndicate the threshold value being previously set, Rankcmb(i,Sc) indicate in threshold value ScThe lower crowd of setting
The hybrid-sorting of survey personnel i, W indicate all candidate many survey personnel.
9. the method as described in claim 1, which is characterized in that the method for generating the technical term library are as follows: obtain first every
A history crowd surveys the issuing time of task and each crowd observes and predicts the attribute of announcement: submitter, submission time, whether be defect, be
No is the natural language description of duplicate reports, report；The natural language description that each crowd observes and predicts announcement is then based on to segment
Operation obtains multiple words, and calculates the document frequency of each word；Then the word and document of m% before document frequency are filtered out
The word of n% after frequency, remaining word are technical term library.
10. method as claimed in claim 9, which is characterized in that the method for generation technique term list are as follows: be based on technical term
The natural language description that each crowd is observed and predicted after accusing participle in library is filtered, and filters out the word not appeared in technical term library
Language obtains each crowd and observes and predicts the corresponding technical term list of announcement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910280296.XA CN110096569A (en) | 2019-04-09 | 2019-04-09 | A kind of crowd survey personnel set recommended method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910280296.XA CN110096569A (en) | 2019-04-09 | 2019-04-09 | A kind of crowd survey personnel set recommended method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110096569A true CN110096569A (en) | 2019-08-06 |
Family
ID=67444534
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910280296.XA Pending CN110096569A (en) | 2019-04-09 | 2019-04-09 | A kind of crowd survey personnel set recommended method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110096569A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886651A (en) * | 2019-02-20 | 2019-06-14 | 福建省农村信用社联合社 | A kind of in-house many test tubes reason method and system |
| CN111090585A (en) * | 2019-12-09 | 2020-05-01 | 中国科学院软件研究所 | Crowd-sourcing task closing time automatic prediction method based on crowd-sourcing process |
| CN111522733A (en) * | 2020-03-16 | 2020-08-11 | 中国科学院软件研究所 | Crowdsourcing tester recommending and crowdsourcing testing method and electronic device |
| CN111666207A (en) * | 2020-05-18 | 2020-09-15 | 中国科学院软件研究所 | Crowdsourcing test task selection method and electronic device |
| CN112346954A (en) * | 2019-08-07 | 2021-02-09 | 南京慕测信息科技有限公司 | Crowdsourcing test platform-oriented cooperation method |
| CN114021758A (en) * | 2022-01-04 | 2022-02-08 | 北京磁浮有限公司 | Operation and maintenance personnel intelligent recommendation method and device based on fusion of gradient lifting decision tree and logistic regression |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150095084A1 (en) * | 2012-12-05 | 2015-04-02 | Matthew Cordasco | Methods and systems for connecting email service providers to crowdsourcing communities |
| CN108804319A (en) * | 2018-05-29 | 2018-11-13 | 西北工业大学 | A kind of recommendation method for improving Top-k crowdsourcing test platform tasks |
| CN108874655A (en) * | 2017-05-15 | 2018-11-23 | 华为技术有限公司 | A kind of method and device handling crowdsourcing test data |
-
2019
- 2019-04-09 CN CN201910280296.XA patent/CN110096569A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150095084A1 (en) * | 2012-12-05 | 2015-04-02 | Matthew Cordasco | Methods and systems for connecting email service providers to crowdsourcing communities |
| CN108874655A (en) * | 2017-05-15 | 2018-11-23 | 华为技术有限公司 | A kind of method and device handling crowdsourcing test data |
| CN108804319A (en) * | 2018-05-29 | 2018-11-13 | 西北工业大学 | A kind of recommendation method for improving Top-k crowdsourcing test platform tasks |
Non-Patent Citations (2)
| Title |
|---|
| 刘莹等: "移动应用众包测试人员评价模型", 《计算机应用》 * |
| 崔强等: "众测中的工作者选择方法研究", 《软件学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886651A (en) * | 2019-02-20 | 2019-06-14 | 福建省农村信用社联合社 | A kind of in-house many test tubes reason method and system |
| CN112346954A (en) * | 2019-08-07 | 2021-02-09 | 南京慕测信息科技有限公司 | Crowdsourcing test platform-oriented cooperation method |
| CN111090585A (en) * | 2019-12-09 | 2020-05-01 | 中国科学院软件研究所 | Crowd-sourcing task closing time automatic prediction method based on crowd-sourcing process |
| CN111522733A (en) * | 2020-03-16 | 2020-08-11 | 中国科学院软件研究所 | Crowdsourcing tester recommending and crowdsourcing testing method and electronic device |
| CN111522733B (en) * | 2020-03-16 | 2021-06-01 | 中国科学院软件研究所 | Crowdsourcing tester recommending and crowdsourcing testing method and electronic device |
| CN111666207A (en) * | 2020-05-18 | 2020-09-15 | 中国科学院软件研究所 | Crowdsourcing test task selection method and electronic device |
| CN114021758A (en) * | 2022-01-04 | 2022-02-08 | 北京磁浮有限公司 | Operation and maintenance personnel intelligent recommendation method and device based on fusion of gradient lifting decision tree and logistic regression |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110096569A (en) | A kind of crowd survey personnel set recommended method | |
| Feng et al. | An expert recommendation algorithm based on Pearson correlation coefficient and FP-growth | |
| Ambreen et al. | Empirical research in requirements engineering: trends and opportunities | |
| Zhi et al. | Cost, benefits and quality of software development documentation: A systematic mapping | |
| Chen et al. | “Sampling” as a baseline optimizer for search-based software engineering | |
| Thakurta | Understanding requirement prioritization artifacts: a systematic mapping study | |
| EP1939749A2 (en) | Software testing capability assessment framework | |
| US20210286708A1 (en) | Method and electronic device for recommending crowdsourced tester and crowdsourced testing | |
| Wu et al. | Maintenance effort estimation for open source software: A systematic literature review | |
| Molléri et al. | Cerse-catalog for empirical research in software engineering: A systematic mapping study | |
| US20160217427A1 (en) | Systems, methods, and devices for implementing a referral processing engine | |
| Riabov et al. | Scalable Planning for Distributed Stream Processing Systems. | |
| Dal Sasso et al. | What makes a satisficing bug report? | |
| Wang et al. | Predicting product co-consideration and market competitions for technology-driven product design: a network-based approach | |
| CN111159171A (en) | Data auditing method and system | |
| Naderifar et al. | A review on conformance checking technique for the evaluation of process mining algorithms | |
| Ji et al. | Complexity analysis approach for prefabricated construction products using uncertain data clustering | |
| CN115879829A (en) | Evaluation expert screening method applied to platform innovation capability examination and verification | |
| Zhang et al. | Uncertainty-wise requirements prioritization with search | |
| CN115860572A (en) | Supplier evaluation method and system based on flexible configuration of multi-dimensional operation | |
| US20050278301A1 (en) | System and method for determining an optimized process configuration | |
| Perera et al. | A rule-based system for automated generation of serverless-microservices architecture | |
| Wnuk et al. | Replication of an experiment on linguistic tool support for consolidation of requirements from multiple sources | |
| US20160217425A1 (en) | Systems, methods, and devices for implementing a referral processor | |
| US20160217216A1 (en) | Systems, methods, and devices for implementing a referral search |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190806 |
|
| RJ01 | Rejection of invention patent application after publication |