WO2013003861A3 - Evaluating a worker in performing crowd sourced tasks and providing in-task training through programmatically generated test tasks - Google Patents
Evaluating a worker in performing crowd sourced tasks and providing in-task training through programmatically generated test tasks Download PDFInfo
- Publication number
- WO2013003861A3 WO2013003861A3 PCT/US2012/045321 US2012045321W WO2013003861A3 WO 2013003861 A3 WO2013003861 A3 WO 2013003861A3 US 2012045321 W US2012045321 W US 2012045321W WO 2013003861 A3 WO2013003861 A3 WO 2013003861A3
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- job
- tasks
- worker
- evaluating
- providing
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Educational Administration (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Systems and methods for evaluating a worker in performing crowd sourced tasks and providing in-task training are disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, for selecting a job distributed through a job distribution platform for workers to work on, for use to generate a test task, the job being associated with a known correct result, associating a manipulated result, known to be an incorrect result for the job, with the job to generate the test task, and/or presenting the job with the manipulated result as the test task to a worker for evaluation of work quality of the worker. The job distribution platform crowd sources tasks online to workers to work on via their respective computing devices.
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161503483P | 2011-06-30 | 2011-06-30 | |
US61/503,483 | 2011-06-30 | ||
US13/275,290 US8554605B2 (en) | 2011-06-29 | 2011-10-17 | Evaluating a worker in performing crowd sourced tasks and providing in-task training through programmatically generated test tasks |
US13/275,290 | 2011-10-17 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2013003861A2 WO2013003861A2 (en) | 2013-01-03 |
WO2013003861A3 true WO2013003861A3 (en) | 2013-03-14 |
Family
ID=47424833
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2012/045321 WO2013003861A2 (en) | 2011-06-30 | 2012-07-02 | Evaluating a worker in performing crowd sourced tasks and providing in-task training through programmatically generated test tasks |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2013003861A2 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10755221B2 (en) * | 2015-12-29 | 2020-08-25 | Workfusion, Inc. | Worker answer confidence estimation for worker assessment |
CN110853557B (en) * | 2019-11-26 | 2022-09-02 | 武汉帆茂电子科技有限公司 | FPGA-based liquid crystal module bomb inspection system and inspection method |
CN116016293B (en) * | 2023-03-22 | 2023-08-25 | 深圳市亿联无限科技股份有限公司 | Multi-task queue testing method, system and platform for network communication product production |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030050829A1 (en) * | 2001-09-13 | 2003-03-13 | International Business Machines Corporation | Method and system for collecting and distributing data evaluating the job performances of short term job contractors through a computer controlled centralized database |
US20050154600A1 (en) * | 2004-01-08 | 2005-07-14 | American International Group, Inc. | Extended work program |
US20070124363A1 (en) * | 2004-07-21 | 2007-05-31 | The Mathworks, Inc. | Instrument-based distributed computing systems |
US7778865B1 (en) * | 2003-05-30 | 2010-08-17 | Kane Jeffrey S | Distributional assessment system |
US20110313820A1 (en) * | 2010-06-17 | 2011-12-22 | CrowdFlower, Inc. | Using virtual currency to compensate workers in a crowdsourced task |
-
2012
- 2012-07-02 WO PCT/US2012/045321 patent/WO2013003861A2/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030050829A1 (en) * | 2001-09-13 | 2003-03-13 | International Business Machines Corporation | Method and system for collecting and distributing data evaluating the job performances of short term job contractors through a computer controlled centralized database |
US7778865B1 (en) * | 2003-05-30 | 2010-08-17 | Kane Jeffrey S | Distributional assessment system |
US20050154600A1 (en) * | 2004-01-08 | 2005-07-14 | American International Group, Inc. | Extended work program |
US20070124363A1 (en) * | 2004-07-21 | 2007-05-31 | The Mathworks, Inc. | Instrument-based distributed computing systems |
US20110313820A1 (en) * | 2010-06-17 | 2011-12-22 | CrowdFlower, Inc. | Using virtual currency to compensate workers in a crowdsourced task |
Also Published As
Publication number | Publication date |
---|---|
WO2013003861A2 (en) | 2013-01-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2011159433A3 (en) | Using virtual currency to compensate workers in a crowdsourced task | |
MX2018004593A (en) | Adjusting industrial vehicle performance. | |
EP3096235A4 (en) | Information processing system, information processing server, information processing program, and fatigue evaluation method | |
EP2657863A3 (en) | Methods and computer-program products for generating grasp patterns for use by a robot | |
EP3182091A4 (en) | Electromagnetic multiaxial fatigue testing machine | |
MY186462A (en) | Training ensurance method and system for computer directed assembly and manufacturing | |
AU2012272977A8 (en) | System and method for building and managing user experience for computer software interfaces | |
DE102011105141A8 (en) | METHOD AND SYSTEM FOR SIMULATING A WORKING PROCESS ON A TOOL MACHINE | |
WO2011159434A3 (en) | Distributing a task to multiple workers over a network for completion while providing quality control | |
MX2013001171A (en) | System and method for determining a lubricant discard interval. | |
DE112012003977T8 (en) | Non-intrusive method and apparatus for automatically distributing security rules in a cloud environment | |
WO2015101461A3 (en) | Method and apparatus for design of a metrology target | |
TW201614556A (en) | Method, apparatus and computer program product for presenting cooking schedule | |
WO2013188511A3 (en) | System and method for online hiring and virtual interview questioning | |
WO2012131329A3 (en) | Improvements in and relating to methods and systems for investigating radioactive sources in locations | |
WO2014184710A3 (en) | Virtual reality based welding simulator | |
EP3327701A4 (en) | Online test method and online test server for evaluating creativity for ideas | |
WO2015127170A3 (en) | Method and system for providing a robust and efficient virtual asset vulnerability management and verification service | |
MX2016003852A (en) | Platform for generating personalized career pathways. | |
GB201219722D0 (en) | X-ray analysis apparatus | |
EP3327717A4 (en) | Information processing device, method for evaluating fatigue level, and program | |
WO2014107489A3 (en) | System and method for crowdsourcing map production | |
WO2013003861A3 (en) | Evaluating a worker in performing crowd sourced tasks and providing in-task training through programmatically generated test tasks | |
GB201206932D0 (en) | Comparing event data sets | |
MY193347A (en) | Mantle phenotype detection in palm |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 12804489 Country of ref document: EP Kind code of ref document: A2 |