CN112862303A

CN112862303A - Crowdsourcing quality evaluation system and method based on block chain

Info

Publication number: CN112862303A
Application number: CN202110152939.XA
Authority: CN
Inventors: 王建荣; 孙浩然; 徐天一; 刘贺; 邱铁
Original assignee: Tianjin University
Current assignee: Tianjin University
Priority date: 2021-02-04
Filing date: 2021-02-04
Publication date: 2021-05-28
Anticipated expiration: 2041-02-04
Also published as: CN112862303B

Abstract

The invention relates to a crowd-sourcing quality evaluation system and method based on a block chain, which integrates a block chain technology and a crowd-sourcing system, realizes a safe and reliable distributed crowd-sourcing system, and effectively solves the problems of single-point failure, data leakage and data credibility; more reliable worker nodes are obtained through a credit mechanism of historical behaviors of the nodes, and the quality of crowdsourcing results is further improved; a more reasonable reward mechanism is constructed through the intelligent contract, and compared with a traditional fixed pricing mode, the incentive becomes more transparent and reasonable.

Description

Crowdsourcing quality evaluation system and method based on block chain

Technical Field

The invention belongs to the field of crowdsourcing quality evaluation systems in distributed environments, and particularly relates to a crowdsourcing quality evaluation system and method based on a block chain.

Background

Crowdsourcing is a new business model that transforms human knowledge, experience, ability, etc. into human real profits over the internet. Crowdsourcing reduces the labor cost of the party issuing the package, improves the working efficiency, and can select a proper task and complete the task to obtain a corresponding reward according to the interest and hobbies of the worker.

The core of crowdsourcing is to solve the problem of machine difficulty by using group intelligence. At present, the task types on the crowdsourcing platform are various, and the application field is wide. The task categories can be divided into macro tasks and micro tasks according to granularity. Macro tasks are challenging and innovative tasks that take a significant amount of time to complete, and the amount of rewards is large. Micro-tasks refer to small and repetitive tasks, usually by collecting worker answers and then integrating to arrive at the final result. The micro-tasks are simple and easy to operate, and gradually occupy the main crowdsourcing market. The current popular crowdsourcing platform Amazon Mechanical turn is a typical representative of the crowdsourcing micro task platform. The worker can participate in the task of the crowdsourcing platform by utilizing the idle time, and the reward set by the task can be obtained after the task is completed.

However, although the crowdsourcing system is well developed, the traditional trust-based mode has the weakness, and due to the wide population, background knowledge, professional ability and the like of workers participating in the crowdsourcing platform are different, so that the crowdsourcing result is poor in quality. This also presents some unavoidable challenges. Most of the existing crowdsourcing platforms (such as MTurk) adopt a fixed pricing incentive mechanism, under the mechanism, no matter how good or bad the quality of the tasks completed by workers, the workers can obtain the same reward, and the amount of the benefits obtained by the workers is only related to the number of the tasks completed. In this case, many malicious workers may freely submit answers in order to maximize their profits, regardless of the quality of the task results, and thus the final answers obtained by the contracting party are of low quality and cannot be utilized. Although some platforms have quality control methods, most crowdsourcing systems are based on centralized third parties, complete credibility cannot be guaranteed, problems of data leakage, data loss and the like exist, single-point faults exist, and problems of data loss, data leakage and the like can also be caused.

The blockchain technology is honored as an innovative technology that can change most industries. The block chain has the characteristics of decentralization, tamper resistance, traceability, openness and transparency and the like, and can overcome the defects that a centralized platform is easy to attack and generate single-point faults. In the current blockchain 2.0 era, revenue rules and condition codes can be released to a blockchain network by using intelligent contracts in blockchains, so that the standardization of each industry is transparent and standardized.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a crowdsourcing quality assessment system and method based on a block chain, wherein the right of quality assessment is completed by an intelligent contract by introducing the block chain idea, so that a decentralized platform is realized; the data security is ensured by utilizing the anti-tampering characteristic of the block chain, and the problem of data loss of a centralized platform is solved; by constructing a credit model, a credit value which accurately and objectively reflects the behavior characteristics of the user is used as a basis for screening workers in crowdsourcing quality evaluation; and finally, according to the quality evaluation result, a two-stage reward mechanism based on the finished quality is constructed, so that the traditional incentive mode of fixed pricing is broken, and a more reasonable and transparent reward distribution mode is provided.

The technical problem to be solved by the invention is realized by the following technical scheme:

a system for crowd sourcing quality assessment based on blockchains, comprising: the system model comprises a task requester, a worker and a task evaluator, wherein the task requester, the worker and the task evaluator act at different periods of the whole crowdsourcing process; the task requester, the worker and the task evaluator participate in the quality evaluation process of the block chain system together, and the quality evaluation process comprises three stages of task issuing and execution, task evaluation and reward distribution.

A crowd-sourcing quality assessment method based on blockchains is characterized by comprising the following steps: the method comprises the following steps:

1) a task requester issues tasks and relevant information of the tasks in a distributed block chain system;

2) the crowdsourcing system selects worker nodes for the current task through a credit mechanism, and selects judges from the worker nodes through a random selection algorithm;

3) carrying out crowdsourcing work by workers, and storing results into a block chain;

4) the selected judgers evaluate the tasks submitted by the workers;

5) and the task publisher distributes the rewards to the workers according to the evaluation result of the judgers.

Moreover, the method comprises the following specific operations:

1) when a requester issues a task, relevant parameters are set, descriptions such as specific content description of the task, a task storage position, a reward amount of the task, the number of workers required by the task, a task deadline, skills required by the workers, credit value requirements of the workers and the like are written in an intelligent contract in detail, and then a transaction is sent to a block chain to issue the task; the requester needs to deploy the intelligent contract on the block chain, in order to ensure the smooth operation of the intelligent contract, the requester transfers enough Ethernet coins in the intelligent contract, actual task data can be stored in the IPFS, and a pointer for acquiring the task data is stored in the intelligent contract;

2) after the task is released to the block chain, only workers meeting the requirements have the opportunity to participate in crowdsourcing, the workers meeting the requirements are placed in a buffer pool, firstly, a judger is selected through a random selection algorithm, corresponding calculation work is completed before the deadline of a quality evaluation stage, and a final evaluation result is given;

3) selecting workers with the quantity required by the requester through a node matching algorithm, wherein all the selected workers need to complete the work of the first stage between the deadline time, when submitting the task result, signing the task result by using a private key of the selected workers, and then recording the signed message and the hash of the task result to a block chain through a transaction; if the worker does not submit the task result on time, the deposit mortgage mortgaged on the block chain cannot be returned to the account of the worker, and the deposit mortgage can be directly sent to the task requester, and meanwhile, the credit value of the worker can be reduced according to a credit updating mechanism;

4) after the workers finish the task, the public key of the requester and the public key of the evaluator are used for encrypting the answers respectively, then the encrypted answers are sent to a down-link database for storage, meanwhile, each worker signs the result by using the private key of the worker, then the signed message and the hash value of the task result are sent to and stored in the block chain, and the requester and the evaluator can find the storage position of the task result in the IPFS according to the pointer on the block chain; after the storage position of the task is obtained, the requester and the judge can decrypt the data stored in the position respectively by using own private keys, and the requester and the judge can verify the validity of the transaction according to the data signature and the public key of a worker to ensure that the transaction content is not tampered;

5) the assessor begins the assessment work after the task submission deadline. The assessment task also has an expiration date, and if the assessment is not completed before the expiration date, the deposit and reputation values are lost like a worker. The judging person can collect information submitted by all workers on the block chain, and then decrypt the information by using a private key of the judging person to obtain a specific storage position of the task; after the original data of all task results are obtained, triggering a quality evaluation algorithm in a corresponding intelligent contract, and identifying the results together to generate a final task evaluation table, wherein a judge can encrypt the task evaluation table by using a public key of a requester and store the encrypted task evaluation table in a linked database, and meanwhile, can store a hash value and a pointer in a block chain, and the requester can find a specific storage position of the data according to the task result pointer and then decrypt the data by using a private key of the requester to obtain task answers and the original data of the task evaluation;

6) after the task evaluation is completed, corresponding rewards, including rewards for the tokens and rewards for the reputation value, are issued to the workers according to a reward distribution mechanism and a reputation update mechanism. The amount of rewards earned by the worker is closely related to the quality of the task completed by the worker, and the better the quality of the completed task, the more rewards earned; the awards of the judges are also distributed as specified in the smart contracts.

The invention has the advantages and beneficial effects that:

1. the invention discloses a crowd-sourcing quality evaluation system and method based on a block chain, which integrates a block chain technology and a crowd-sourcing system, realizes a safe and reliable distributed crowd-sourcing system, and effectively solves the problems of single-point failure, data leakage and data credibility;

2. according to the crowd-sourcing quality evaluation system and method based on the block chain, more reliable worker nodes are obtained through a credit mechanism of node historical behaviors, and the quality of crowd-sourcing results is further improved.

3. According to the crowd-sourcing quality assessment system and method based on the block chain, a more reasonable reward mechanism is constructed through an intelligent contract, and compared with a traditional fixed pricing mode, incentives become more transparent and reasonable.

Drawings

FIG. 1 is a diagram of a system model of the present invention;

FIG. 2 is a system architecture diagram of the present invention;

FIG. 3 is a flow chart of the quality assessment of the present invention.

Detailed Description

The present invention is further illustrated by the following specific examples, which are intended to be illustrative, not limiting and are not intended to limit the scope of the invention.

A crowd-sourcing quality assessment system based on block chains is modeled as shown in FIG. 1, and the specific functions of the entities are described as follows:

the task requester: i.e. providers of crowdsourced tasks. Task requesters are generally companies or organizations that have crowd-sourced task requirements that cannot use computers or other machines to accomplish existing tasks, and therefore, may achieve their end purpose in a crowd-sourced manner. If a requester wants to participate in the CQAM-BC model, the requester must first become a legitimate node of the blockchain by registering an identity, because only the legitimate node can initiate a transaction in the blockchain. The requester may need to freeze a portion of the funds on the blockchain that includes the remuneration that the task needs to pay for the corresponding worker when initiating the transaction.

Workers: i.e. the executing party of the crowdsourced task. As with the requestor, all workers must also obtain a legitimate identity through registration to enter the blockchain as a legitimate node. Typically, a worker is a node in a blockchain that is active and has some computing power. If the worker meets the various requirements of the requester, the worker may participate in some crowdsourcing task. The worker completes the task by providing computing power or otherwise and submits the final result to the blockchain. The blockchain system receives a corresponding reward based on the quality of the task performed by the worker.

The judger: i.e. the evaluators of the crowdsourcing tasks. The reviewer is also a legitimate node in the blockchain system, and differs from the worker in that they are at two different stages of the crowdsourcing process. After the worker submits the task results, the reviewer begins working.

A crowd-sourcing quality assessment system based on block chains is disclosed, wherein a model architecture is shown in FIG. 2, and the model architecture is divided into four layers in total, namely a storage layer, a block chain layer, an interface layer and an upper application layer;

a storage layer: the storage layer is used for storing original data of tasks issued by requesters and task answers submitted by workers. The storage of the block chain is limited, the data size of tasks (such as image labeling tasks, image recognition tasks and the like) issued by a task publisher is usually large, and if all data are stored in the block chain, a large burden is caused on network synchronization, and excessive disk space is occupied, so that all original task data are not generally stored in the block chain, and the problem of insufficient storage capacity of the block chain is solved by introducing a distributed database such as an IPFS (internet protocol file system). IPFS and blockchain are complementary and have many similar characteristics, making IPFS an ideal place to store data, and can be referenced and time stamped using blockchain techniques. IPFS stores large amounts of data at different nodes, which in combination with blockchain techniques can be guaranteed to be online.

Block chain layer: the blockchain layer stores metadata of tasks, including data size, hash value of data, pointer of data and the like, so that a user can verify the integrity and authenticity of the data without trusting the data stored in the storage layer. The blockchain layer is a core layer of the system architecture, all participants acquire unique identities on the blockchain through registration, and each participant who successfully registers owns a key pair and a unique address. The private key of the key pair is private and is kept by the user separately. The requestor may send a transaction to the blockchain to issue a particular task. The role of any participant on the blockchain can be varied, and miners pack the transactions generated into blocks, which are linked to the ledger of the blockchain. In order to ensure that the data maintained by each node is consistent, the consensus mechanism plays an irreplaceable role.

Interface layer: the interface layer provides various APIs connected by an upper application layer and a block chain layer. Through the provided API, various device terminals of the application layer can interact with the blockchain.

An application layer: the application layer comprises a PC terminal, a mobile device and other terminals. All complex calculations are encapsulated in the client of the application layer device, including a reputation calculation algorithm, a matching degree calculation algorithm, a quality assessment algorithm and a reward distribution algorithm.

The invention provides a credit mechanism more suitable for a crowdsourcing system by combining the characteristics of a block chain on the basis of a traditional credit model. The reputation mechanism calculates based on a reputation reward function and a reputation penalty function. This will be described in detail below:

nodes in the blockchain need to lock a portion of the deposit on the blockchain before accepting the task. If the data submitted by a node is of low quality, the deposit that the node mortises on the blockchain may not be withdrawn. Thus, the model considers that the number of deposits that a node mortgage will affect the quality of the task results that it ultimately submits. If the deposit of the node mortgage is large, the larger the confidence that the node submits high-quality data is, the more the credit value reward is obtained. Therefore, the specific calculation of the reputation reward function is as follows.

Φ(BaseV,E_i＝H)＝BaseV+(1+f(y)+p)

Wherein BaseV represents a base reputation value before a worker executes a task; e_iRepresents the evaluation obtained by the worker after the task is completed; p represents the proportion of the deposit of the worker on the block chain to the total deposit of all the workers; Φ (BaseV, E)_iH) represents the update of the reputation value of the worker after obtaining the evaluation of the H level, and f (y) represents the reward factor of the user. From the reputation reward function, it can be known that the continuous credible service and the mission deposit share together determine the final reward strength.

The specific calculation of the reputation penalty function is as follows.

Φ(BaseV,E_i＝L)＝BaseV-g(x)

Wherein, phi (BaseV, E)_iL) represents the update of the reputation value after the worker obtains the evaluation of the L level, and g (x) represents a penalty factor. According to the calculation formula of the credit penalty function, the reputation value is reduced due to the first doing and doing activities of the workers, and the reduction amplitude of the reputation value is larger and larger along with the accumulation of the doing and doing times and gradually reaches a specific value. From the credit penalty function, the penalty degree of the task can be adjusted by obtaining the total times of low evaluation from the historical behavior of the worker.

The invention provides a crowd-sourcing quality assessment method and system based on a block chain, and provides a new idea for block chain and crowd-sourcing quality assessment, namely, the quality assessment is completed by an intelligent contract through a block chain technology, and a decentralized platform is further realized. And by constructing a credit model, the credit value of the user behavior characteristics is used as a basis for screening workers in crowdsourcing quality evaluation. And finally, constructing a more reasonable and transparent reward distribution mode according to the quality evaluation result.

Although the embodiments of the present invention and the accompanying drawings are disclosed for illustrative purposes, those skilled in the art will appreciate that: various substitutions, changes and modifications are possible without departing from the spirit and scope of the invention and the appended claims, and therefore the scope of the invention is not limited to the disclosure of the embodiments and the accompanying drawings.

Claims

1. A system for crowd sourcing quality assessment based on blockchains, comprising: the system model comprises a task requester, a worker and a task evaluator, wherein the task requester, the worker and the task evaluator act at different periods of the whole crowdsourcing process; the task requester, the worker and the task evaluator participate in the quality evaluation process of the block chain system together, and the quality evaluation process comprises three stages of task issuing and execution, task evaluation and reward distribution.

2. The method of blockchain-based crowdsourcing quality assessment according to claim 1, wherein: the method comprises the following steps:

4) the selected judgers evaluate the tasks submitted by the workers;

3. The method of blockchain-based crowdsourcing quality assessment according to claim 2, wherein: the method comprises the following specific operations: