JP2021039756A - Method of and apparatus for measuring level of operator skill by using mean difference return rate - Google Patents

Abstract

To provide a method of and an apparatus for measurement with using a return rate which can determine an operator's level of skill.SOLUTION: A method according to the present invention has a step of giving a work for a crowdsourcing structure to one return project to a plurality of operators to receive input of work results from the plurality of operators, a step of having the input work results checked by a plurality of inspectors to receive the result of inspection, and a step of measuring a skill level of the operators based on the inspection results. The step for measuring the skill level of the operators has a step of measuring the skill level of the operators by using a work point corresponding to an amount of works of the operator which passed inspection and a return point corresponding to a return amount. A weight value is not applied to the return point upon calculation of the return point until the work amount of the operator exceeds the amount of the predetermined skill level point, but the weight value based on a mean difference return rate is applied to the return point upon calculation of the return rate if the work amount of the operator exceeds the amount of the predetermined skill level point.SELECTED DRAWING: Figure 3

Description

The present invention relates to a method and apparatus for measuring worker skill using an average remand rate.

In recent years, an increasing number of companies are collecting and processing large amounts of data based on crowdsourcing, which involves the general public in some processes of corporate activities. That is, a company opens one project and allows the general public, that is, a worker, to participate in the project, thereby collecting necessary information through the work results completed by the worker. ..

Specifically, when a company opens a project, it selects suitable workers according to the type and difficulty of the project and assigns the work.

At this time, the company selects the workers based on the number of works completed by the workers. That is, the larger the number of work completed by the worker, the more skilled the worker is judged to be, and the corresponding worker is given an opportunity to participate in more work.

However, simply judging the skill level of a worker based on the number of completed work cannot reflect the actual skill level of the worker because the judgment is made without considering the actual work ability of the worker at all. ..

That is, there is a problem that a lot of work is assigned to the corresponding worker because the number of completed works is large even though the worker has a low practical work ability, and the work ability of the corresponding worker is higher than that of the corresponding worker. However, because work that requires a high level of work capacity is assigned, the number of work to be remanded increases, which prolongs the execution period of the project and increases the remand cost (acceptance cost). There is.

Korean Publication No. 10-2014-095956 (2014.08.04.)

The present invention has been made in view of the above circumstances, and an object of the present invention is to use an average remand rate, which can determine a worker's work skill level by comparing a worker's remand rate and an average remand rate. The purpose is to provide a method and an apparatus for measuring the skill level of an existing worker.

The problem to be solved by the present invention is not limited to the problem mentioned above, and other problems not mentioned can be clearly understood by ordinary engineers from the following description.

The worker proficiency measurement method using the average remand rate according to one aspect of the present invention for solving the above-mentioned problems gives the work of the crowdsourcing platform for one project to a plurality of workers, and a plurality of workers. The stage of receiving the input of the work result from the plurality of workers, the stage of receiving the input of the acceptance result from the plurality of inspectors by allowing a plurality of inspectors to inspect the work result input from the plurality of workers, and the acceptance of the inspection. The step of measuring the skill level of the worker, including the step of measuring the skill level of the worker based on the result, is the number of work points corresponding to the acceptance acceptance (passing) quantity of the worker and the remand quantity. The skill level of the worker is measured using the difference from the corresponding remand point, and the remand point is the remand before the work quantity of the worker exceeds the predetermined skill time quantity. The weighted value is not applied when calculating the score, and after the work quantity of the worker exceeds the predetermined quantity at the time of skill, the weighted value based on the average remand rate is applied when calculating the remand score. It is characterized in that it is calculated.

In one embodiment of the present invention, the quantity at the time of skill is derived by a pilot project in which workers with higher skill level, workers with medium skill level, and workers with lower skill level are evenly selected. It can be.

In one embodiment of the present invention, if the worker's work quantity exceeds the predetermined skilled time quantity and then the worker's remand rate is greater than the average remand rate, it is greater than 1. The weighted value of the first value is applied to calculate the remand points, and when the remand rate of the worker is smaller than the average remand rate, the weighted value of the second value smaller than 1 is applied to the above. The number of remand points can be calculated.

In one embodiment of the present invention, when the work quantity of the worker exceeds the predetermined quantity at the time of skill and the remand rate of the worker is larger than the average remand rate, the total remand quantity ( Of a), the weighted value is not applied in relation to the quantity (b) corresponding to the average remand quantity (b), and in relation to the remaining quantity (ab) exceeding the average remand quantity (b). The number of remand points can be calculated by applying the weighted value of the first value.

In one embodiment of the present invention, if the remand rate of the worker is smaller than the average remand rate after the work quantity of the worker exceeds the predetermined quantity at the time of skill, the total remand quantity (c). ), The weighted value of the second value is applied in relation to the quantity (bc) less than the average remand quantity (b), and in relation to the remaining quantity (c- (bc)). Can calculate the remand points without applying the weighted value.

In one embodiment of the present invention, the number of work points can be calculated using the work cost paid to the worker when passing the acceptance inspection.

In one embodiment of the present invention, the remand points can be calculated using the remand cost generated at the time of remand.

In one embodiment of the present invention, the measurement result of the skill level of the worker can be used to provide an opportunity to work on the crowdsourcing platform for other projects that require a skill level of a predetermined grade or higher.

A computer program according to another aspect of the present invention for solving the above-mentioned problems is combined with a computer which is hardware, executes a worker skill measurement method using the average remand rate, and is stored in a medium. ..

The worker proficiency measuring device using the average remand rate according to another aspect of the present invention for solving the above-mentioned problems transmits the work of the crowdsourcing platform for one project to a plurality of workers, and the plurality of workers. The work result is received from the worker, the work result input from the plurality of workers is transmitted to the plurality of inspectors so that the plurality of inspectors can inspect, and the acceptance result is received from the plurality of inspectors. Based on the acceptance inspection result, the number of work points corresponding to the acceptance pass quantity of the worker is calculated, the number of remand points corresponding to the remand quantity is calculated, and the difference between the work points and the remand points is used. The calculation unit includes a calculation unit for calculating the skill level of the worker, and the calculation unit applies a weighted value when calculating the remand points before the work quantity of the worker exceeds a predetermined quantity at the time of skill. Instead, after the work quantity of the worker exceeds the predetermined quantity at the time of skill, the weighted value based on the average remand rate is applied and calculated.

Other specific matters of the present invention are included in the detailed description and drawings.

According to the worker proficiency measuring method and apparatus using the average remand rate of the present invention as described above, the proficiency of the worker is measured by comparing the remand rate of the worker with the average remand rate. , The skill level that accurately reflects the work ability of the worker can be measured, so that when the work is assigned to the worker, the level of work suitable for the work ability of the worker can be efficiently assigned.

In addition, since the skill level is measured using the worker's remand rate, a worker with a high remand rate is measured with a lower skill level than a worker with a low remand rate even if the number of works that have passed the acceptance inspection is the same. obtain. As a result, the worker can improve the quality of the work by performing the work more carefully so that the work is not returned.

Furthermore, by reducing the number of operations to be remanded, the project can be completed within a specified period, and the remand cost (acceptance cost) can be reduced.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned above can be clearly understood by ordinary engineers from the following description.

It is a conceptual diagram of the crowdsourcing service by one Example of this invention. It is a flowchart explaining the progress process of the project of the crowdsourcing platform by one Example of this invention. It is a flowchart of the worker skill degree measurement method using the average remand rate by one Example of this invention. It is a flowchart of the remand point calculation method which differs depending on the work quantity and the remand rate of a worker at the time of measuring the worker skill level by one Example of this invention. It is a graph explaining the quantity at the time of proficiency derived by the pilot project by one Example of this invention. It is a block diagram of the worker skill level measuring apparatus using the average return rate by one Example of this invention.

The advantages and features of the present invention, and the methods for achieving them, will be clarified with reference to the examples described in detail with the accompanying drawings. However, the present invention is not limited to the examples disclosed below, and can be realized in various forms different from each other. However, the present embodiment is provided to complete the disclosure of the present invention and to allow ordinary engineers in the technical field to which the present invention belongs to fully understand the scope of the present invention, and the present invention is claimed. It is only defined by the scope.

The terms used herein are for illustration purposes only and are not intended to limit the invention. In the present specification, the singular type also includes a plural type unless otherwise specified. As used herein, "comprises" and / or "comprising" does not preclude the presence or addition of one or more other components other than those mentioned. The same drawing reference throughout the specification indicates the same component, and "and / or" includes each of the mentioned components and any combination of one or more. Of course, even if "first", "second", etc. are used to describe various components, these components are not limited by these terms. These terms are used solely to distinguish one component from the other. Therefore, it goes without saying that the first component referred to below can also be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms used herein (including technical and scientific terms) are used as meanings commonly understood by ordinary engineers in the technical field to which the present invention belongs. Also, terms defined in commonly used dictionaries are not ideally or over-interpreted unless explicitly specifically defined.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a conceptual diagram of a crowdsourcing service according to an embodiment of the present invention.

With reference to FIG. 1, the crowdsourcing service is composed of 10 clients, 20 service providers, and 30 masses.

The client 10 means a company or an individual who requests a crowdsourcing-based project (hereinafter referred to as a project).

The requester 10 requests a project for the purpose of collecting source data or data annotation. Collecting source data means collecting unprocessed data, such as collecting recorded audio, collecting photographs, and so on. Data annotation means entering annotation data related to source data such as text, photos, and videos. For example, data annotation may include, but is not limited to, searching for an individual in a given text, searching for similar sentences, and so on.

The service provider 20 means a company that provides a crowdsourcing service.

When the service provider 20 receives a project request for a product or service from the client 10, the service provider 20 allocates the work for the project to the general public 30 and receives the work result from the public 30. After that, the output (final product) extracted based on the work result is provided to the client 10.

At this time, the service provider 20 provides the crowdsourcing service to the client 10 and the public 30 via the crowdsourcing platform (hereinafter referred to as the platform). That is, when the service provider 20 receives a project request from the client 10, the service provider 20 opens the project on the platform. After that, when the work result for the project opened by the public 30 is provided, the project can be completed on the platform, the output (final product) can be extracted and provided to the client 10.

Mass 30 means the general public participating in a project opened on the platform. Here, the public 30 can participate in a project opened on the platform through an application or website provided by the service provider 20.

The mass 30 is composed of a worker 32 and an inspector 34.

Worker 32 decides to participate in a specific project among a plurality of projects opened on the platform. After that, the worker 32 performs work such as collecting source data or data annotation, and sends this to the platform.

The inspector 34 decides to participate in a specific project among a plurality of projects opened on the platform. After that, the inspector 34 inspects the work result performed by the worker 32. As a result of the acceptance inspection, the inspector 34 can perform pass certification (passage) or remand of the work, and can input the reason for remand when remand the work.

FIG. 2 is a flowchart illustrating a progress process for a crowdsourcing-based project according to an embodiment of the present invention.

First, the client 10 requests the service provider 20 for the project (S41).

After that, the service provider 20 opens the requested project on the platform (S42). At this time, the service provider 20 determines the grade in consideration of the difficulty level of the project before opening the project. That is, depending on the difficulty level, it is determined what grade or higher the public 30 is exposed to the project. This makes it possible to increase the reliability of the work results of the project.

Also, the service provider 20 can open a pilot project before officially opening the project. Before officially opening the project, the pilot project should check whether there are any problems with the progress of the project, whether there are exceptional situations that are not in the work guide, what quality the client 10 wants, etc. Or, to obtain the data necessary to formally carry out or evaluate a project.

After that, the service provider 20 allocates work to workers 32 of the corresponding grade or higher according to the grade of the project (S43).

After that, the worker 32 performs the assigned work (S44). At this time, the worker 32 can input the reason why the work is not possible without performing the work for the work whose work itself is impossible for some reason.

After that, the service provider 20 receives the work result provided by the worker 32 (S45), and assigns the acceptance inspection work for the work result to the inspector 34 (S46).

After that, the inspector 34 performs the assigned acceptance inspection (S47). At this time, if it is determined that the work has been performed normally, the inspector 34 decides to pass the acceptance inspection, and if it is determined by the acceptance inspection that the work is wrong, it decides to remand. At the time of determining the remand, the inspector 34 inputs the reason for the remand as to why the work was determined to be incorrect.

After that, the service provider 20 receives the acceptance result from the inspector 34 (S48).

If the acceptance result is pass certification (pass), the service provider 20 uses the result of the corresponding work as valid data and extracts the output (final product) at the end of the project based on this.

If the acceptance result is a remand, the service provider 20 can internally perform the acceptance again or assign the work to the worker 32 again to perform the work.

After that, when the project period ends or sufficient valid data is secured, the service provider 20 terminates the relevant project (S49) and calculates the final result based on the secured valid data. And provide it to the client 10 (S50).

At this time, before the end of the project, the service provider 20 evaluates the execution results of the worker 32 and the inspector 34, calculates the work cost and the acceptance cost according to the evaluation, and pays the worker 32 and the inspector 34. ..

In FIGS. 1 and 2, the terms client, service provider, worker, and inspector are simply expressed, but these are smartphones, tablet PCs, PDAs, laptop PCs, desktop PCs, which are operated by each participant. It means a computer device such as a server or a telecommunications device.

FIG. 3 is a flowchart of a worker skill level measuring method using an average remand rate according to an embodiment of the present invention, and FIG. 4 is a flow chart of a worker skill level measurement according to an embodiment of the present invention. It is a flowchart about the calculation method of the remand point which differs according to the work quantity of a worker and the remand rate, and FIG. 5 explains the work quantity at the time of skill derived by the pilot project by one Example of this invention. It is a graph to do.

The steps shown in FIG. 3 can be understood to be performed by a platform server (hereinafter referred to as a server) operated by the service provider 20, but are not limited thereto.

Referring to FIG. 3, in step S110, the server assigns the work of the crowdsourcing infrastructure for one project to the plurality of workers 32, and receives the input of the work results from the plurality of workers 32.

The server receives a plurality of work results provided by a plurality of workers 32 who participate in the project. Here, one project includes a plurality of tasks, but the plurality of tasks can be grouped into a plurality of tasks according to the type, or each task is a different type of task. It may be the same type of work, but it is not limited to this.

That is, when each worker 32 participates in one project, he / she is assigned to perform a plurality of tasks.

After that, the worker 32 receives points for the work that has passed the acceptance inspection by the inspector 34. That is, the worker 32 receives points for the number of work points calculated by using the work cost generated by the completion of the work due to the completion of the work (passing the acceptance inspection).

On the other hand, the worker 32 cannot pass the acceptance inspection by the inspector 34, and is penalized for the returned work, and the returned work must be reassigned and the work must be performed again. .. That is, the worker receives the remand points calculated by using the remand cost generated due to the remand as a penalty.

Here, the worker 32 performs the work using a predetermined terminal device. The terminal device of the worker 32 may be a computer device such as a smartphone, a tablet PC, a PDA, a laptop PC, a desktop PC, or a telecommunications device, but is not limited thereto.

Next, in step S120, the server causes the plurality of inspectors 34 to inspect the work results input from the plurality of workers 32, and then receives the inspected results from the plurality of inspectors 34. That is, when each inspector 34 participates in one project, it is determined whether or not the work performed by the plurality of workers 32 is normally performed.

If it is determined that the work has been performed by mistake, the inspector 34 remands the work. At this time, the inspector 34 also inputs the reason for the remand so that the worker 32 can refer to it when re-working the work.

The inspector 34 receives points for each acceptance inspection, and the points provided to the inspector 34 at this time correspond to the penalty that the worker 32 receives for the returned work.

Here, the inspector 34 performs the work using a predetermined terminal device. The terminal device of the inspector 34 may be, but is not limited to, a computer device such as a smartphone, tablet, PDA, laptop, desktop, or telecommunications device.

Next, in step S130, the server measures the skill level of the worker 32 based on the acceptance inspection result.

The server has the number of work points corresponding to the number of works that have passed the acceptance inspection among the works performed by the worker 32 (hereinafter referred to as the number of works that have passed the acceptance inspection), and the work that has been returned from the work performed by the worker 32. The skill level of the worker 32 is measured by using the number of remand points corresponding to the quantity (hereinafter referred to as the remand quantity).

Specifically, the server measures the skill level of the worker by using the value of the difference obtained by subtracting the return point from the work score of each worker 32. At this time, it is measured that the larger the difference value, the higher the skill level of the worker.

Therefore, in order to measure the skill level so as to sufficiently match the working ability of the worker, it is important to accurately calculate the remand points. That is, the server can accurately measure the skill level of the worker by calculating the return points so as to be different from each other according to the ratio of the work quantity to be returned to the work quantity of the worker.

Hereinafter, it will be described that different remand points are calculated depending on the case with reference to FIG.

In step S210, the server calculates the number of work points of the worker.

As described above, the number of work points corresponds to the number of works that have passed the acceptance inspection among the works performed by the worker 32 (hereinafter referred to as the number of works that have passed the acceptance inspection). Specifically, the number of work points means the value obtained by multiplying the acceptance inspection passing quantity by the work unit price. This value is the same as the points provided to the worker 32 when the work is completed (passed acceptance).

For example, if the specified worker 32 has passed the acceptance inspection of 300 pieces and the work unit price is 50 won, the number of work points of the worker is 15,000 points, and the worker 32 receives 15,000 points from the server. ..

Next, in step S220, the server compares the work quantity of the worker 32 with the quantity at the time of skill. Here, the work quantity of the worker 32 indicates the quantity of the work that has passed the acceptance inspection among the works performed by the worker 32 in the project. The quantity at the time of proficiency is a reference value at which the worker 32 starts to become proficient in performing the work after the project is opened and the worker 32 performs the work to some extent, and is predetermined by the server.

Specifically, the server can derive the quantity at the time of skill by the pilot project. The server selects workers 32 from among previously undertaken projects, based on a project similar to the one currently in progress.

That is, the server sorts the workers of the upper skill level, the workers of the middle skill level, and the workers of the lower skill level evenly based on the skill level measured by the previous similar projects. To do. The workers of the upper skill level, the workers of the middle skill level, and the workers of the lower skill level selected in this way participate in the pilot project and perform the work.

At this time, the server may select the number of workers 32 who participate in the pilot project so as to be the same as the number of workers 32 who formally participate in the project, or the work which officially participates in the project. Only a predetermined percentage of the number of persons 32 may be selected.

With reference to FIG. 5, the server can determine the quantity of work in which the average acceptance completion rate in the pilot project sharply increases and the average remand rate sharply decreases as the quantity at the time of skill. In the specific example of FIG. 5, the skill time is set slightly before the curve of the average remand rate and the curve of the average acceptance completion rate intersect (for example, when the quantity is 5 to 20% less than the quantity at the time of intersection). However, it can also be at or near the intersection.

Further, referring to FIG. 4, in step S230, when the quantity of work of the worker 32 is smaller than the quantity of work of the worker 32, that is, the quantity of work of the worker 32 exceeds the quantity of work at the time of skill. Before, the number of remand points to which the weighted value is not applied is calculated.

There may be a large difference in the work ability at the initial stage of work due to individual differences and experience differences for each worker. In addition, there is a risk that the work may be mistakenly performed at the initial stage of the work due to the interpretation of the work guide.

Therefore, the server calculates the remand points for the work quantity smaller than the quantity at the time of skill without applying the weighted value. For example, if the work quantity (acceptance pass quantity) of the specific worker 32 is 300 at the beginning of the project and the quantity at the time of skill is 500, the server determines that the quantity of work is the quantity at the time of skill. Judging that it does not exceed, the remand points are calculated without applying the weighted value.

Specifically, when the quantity of work of the worker is smaller than the quantity at the time of skill, the server calculates the number of remand points by multiplying the remand work quantity (a) by the remand unit price (u).

For example, if the quantity of the returned work is 200 out of the 300 work quantity (acceptance passing quantity) of the specific worker 32 and the remand unit price is 20 won, the remand score of the corresponding worker 32 is 4. It is 000 points.

On the other hand, the server calculates the remand points to which the weighted value is applied when the work quantity of the worker 32 is larger than the quantity at the time of skill, that is, after the work quantity of the worker 32 exceeds the quantity at the time of skill. At this time, the weighted value is determined based on the average remand rate.

The average remand rate is calculated by the pilot project. Referring to FIG. 5, the server calculates the average remand rate for each work quantity by the pilot project, and the calculated average remand rate sharply decreases after the quantity at the time of skill.

Further, referring to FIG. 4, in step S240, the server determines the worker's remand rate and the average remand rate calculated by the pilot project when the number of workers' work exceeds the quantity at the time of skill. To compare.

Specifically, when the quantity of work completed (passed by acceptance) by the worker 32 is 70 and the quantity at the time of skill is 30, the server determines the remand rate of the worker with the work quantity of 70. , Compare with the average remand rate.

As an embodiment, when the worker 32 participating in the formal project is a low grade worker, the server can make a comparison based only on the work results of the lower skill workers who participated in the pilot project.

That is, the server has a low grade by comparing the remand rate of the low-grade workers 32 who have no particular work performance experience with the average remand rate of the low-skilled workers who participated in the pilot project. For the worker 32, the comparison standard can be slightly lowered.

Next, in step S250, when the worker's remand rate is larger than the average remand rate, the server applies the weighted value of the first value to calculate the remand points. Here, the first value is a value larger than 1 (for example, 2).

That is, when the work quantity exceeds the quantity at the time of skill and the remand rate of the worker is higher than the average remand rate, the server applies a double weighted value to the remand quantity larger than the average remand quantity. Calculate the number of remand points.

Specifically, referring to the following [Formula 1], the server is related to the quantity (b) corresponding to the average remand quantity (b) in the total remand quantity (a) of the worker. The weighted value of the first value (w1; eg 1.3 to 3 times, especially 2 times) in relation to the remaining quantity (ab) that exceeds the average remand quantity (b) without applying the weighted value. The number of remand points can be calculated by applying the weighted value of) and multiplying by the remand unit price (u). In other words, a penalty is added to the base remand points calculated by the remand quantity (a) x remand unit price (u), and this penalty addition is the excess (a-) from the average remand quantity (b). It is calculated by multiplying b) by the value (w1-1) obtained by subtracting 1 from the first value together with the remand unit price (u).

[Formula 1]
Average remand quantity x remand unit price + (remand quantity-average remand quantity) x remand unit price x 2
= Remand quantity x Remand unit price + (Remand quantity-Average remand quantity) x Remand unit price x (2-1)
b × u ＋ (ab) × u × w1 ＝ a × u ＋ (ab) × u × (w1-1)

For example, if the total remand quantity (a) of the worker for 70 work quantities is 50, the average remand quantity (b) for 70 work quantities is 35, and the remand unit price is 10 won, the number of remand points Can be calculated as 35 × 10 + (50-35) × 10 × 2 = 650 points.

Next, in step S260, when the worker's remand rate is smaller than the average remand rate, the server applies the weighted value of the second value to calculate the remand points. Here, the second value is a value smaller than 1 (for example, 0.5).

That is, when the work quantity exceeds the quantity at the time of skill and the remand rate of the worker is smaller than the average remand rate, the server applies a weighted value of 0.5 times to the remand quantity smaller than the average remand quantity. Then, the number of remand points is calculated.

Specifically, referring to the following [Formula 2], the server sets the reduced quantity (bc) from the average remand quantity (b) based on the entire remand quantity (c) of the worker. In connection with this, a weighted value of the second value (w2; for example, a weighted value of 0.3 to 0.7 times, particularly 0.5 times) is applied, and the quantity (c-(b-)) offset by this reduction is applied. In relation to c)), the remand points can be calculated by multiplying and adding the remand unit price (u) without applying the weighted value. In other words, the incentive subtraction is performed on the base remand points calculated by the remand quantity (a) × the remand unit price (u), and the incentive subtraction is the reduction amount (bc) from the average remand quantity (b). ), The value obtained by subtracting the second value from 1 (1-w2) is calculated by multiplying the value (1-w2) together with the remand unit price (u).

[Formula 2]
(Average remand quantity-Return quantity) x Remand unit price x 0.5 + (Remand quantity- (Average remand quantity-Remand quantity)) x Remand unit price = Remand quantity x Remand unit price- (Average remand quantity-Remand quantity) x Remand unit price x (1-0.5)
(bc) × u × w2 ＋ [c- (bc)] × u ＝ c × u− (bc) × u × (1-w2)

For example, if the total remand quantity (c) of the worker for 70 work quantities is 20, the average remand quantity (b) for 70 work quantities is 35, and the remand unit price is 10 won, the remand points are ( It can be calculated as 35-20) × 10 × 0.5 + (20- (35-20)) × 10 = 125 points.

Next, in step S270, the server measures the skill level of the worker 32 using the value of the difference between the calculated number of work points and the number of return points. At this time, it is determined that the larger the difference value (final score), the higher the skill level of the worker 32.

The server can classify the worker 32 into a worker with a higher skill level, a worker with a medium skill level, and a worker with a lower skill level according to the final score.

If 0 points is the lowest score and 300 points is the highest score, and the final score of the worker 32 is within the range of 0 to 100 points or 0 to 119 points, the lower skill level. It can be classified as a worker with a medium skill level if it is in the range of 101 to 200 points or 120 to 200 points, and it can be classified as a worker with a medium skill level, from 201 points to 300 points. If it is within the range of up to, it can be classified as a worker with a higher skill level.

After that, the server uses the proficiency measurement result of the worker 32 to give the worker 32 an opportunity to work on the crowdsourcing platform for other projects that require a proficiency level equal to or higher than a predetermined grade. Can be done.

That is, the worker 32 classified as a worker with a higher skill level as a result of measuring the skill level can later participate in all projects requiring a higher skill level, a middle skill level, or a lower skill level. However, the worker 32 classified as a worker with a lower skill level as a result of the measurement of the skill level can participate only in the project in which the lower skill level is required thereafter.

FIG. 6 is a block diagram of a worker skill level measuring device using an average remand rate according to an embodiment of the present invention.

Referring to FIG. 6, the worker skill level measuring device 300 (hereinafter referred to as the worker skill level measuring device) using the average reversal rate includes a communication unit 310 and a calculation unit 320.

The communication unit 310 transmits the work of the crowdsourcing platform for one project to a plurality of workers 32, receives the work results from the plurality of workers 32, and receives the work results input from the plurality of workers 32. It plays a role of transmitting to a plurality of inspectors 34 so that a plurality of inspectors 34 can inspect and receiving acceptance results from the plurality of inspectors 34.

Based on the acceptance inspection result, the calculation unit 320 calculates the number of work points corresponding to the acceptance inspection pass (passing) quantity of the worker 32, calculates the remand points corresponding to the remand quantity, and the difference between the work points and the remand points. Plays a role in calculating the skill level of the worker 32 using.

Further, when calculating the remand points, the calculation unit 320 does not apply the weighted value before the work quantity of the worker 32 exceeds the predetermined quantity at the time of skill, and the work quantity of the worker 32 is predetermined. After exceeding the quantity at the time of skill, a weighted value based on the average remand rate is applied to calculate.

The worker proficiency measuring device 300 described with reference to FIG. 6 can be provided as a component of the server described above.

The worker proficiency measurement method using the average remand rate according to the embodiment of the present invention described above is embodied by a program (or application) in order to be executed in combination with a computer as hardware. Can be stored in the medium.

The above-mentioned program is C, C ++, JAVA, which can be read by the processor (CPU) of the computer through the device interface of the computer so that the computer can read the program and execute the method embodied by the program. A code encoded in a computer language such as (registered trademark), Ruby, or machine language can be included. Such code can include a function that defines the functions required to execute the method and related functional code (Funcational Code), and each of the functions is performed by the processor of the computer. It can include control code related to the execution procedure required for execution according to a predetermined procedure. In addition, such code should refer to additional information or media necessary for the processor of the computer to execute each of the functions at any position (address) in the internal or external memory of the computer. Further memory reference related code can be included. In addition, when the processor of the computer needs to communicate with some other computer or server which is remote (remote) in order to execute each of the functions, the code is the communication of the computer. Modules can be used to further include communication-related codes for how to communicate with some (any) remote computer, server, etc., what information and media should be sent and received during communication, etc. ..

The stored medium is not a medium that stores data for a short time such as a register, a cache, or a memory, but a medium that stores data semipermanently and can be read by an apparatus. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. That is, the program can be stored in various recording media on various servers to which the computer can be connected or in various recording media on the user's computer. Further, the media may be distributed to computer systems connected to each other by a network, and a code that can be read by a computer in a distributed manner may be stored.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, ordinary engineers in the technical field to which the present invention belongs can use the present invention without changing its technical idea or essential features. It can be understood that it can be carried out in a concrete form of. Therefore, the examples described above should be understood as exemplary in all respects and not restrictive.

10 Client 20 Service provider 30 Mass 32 Worker 34 Inspector 300 Worker proficiency measuring device 310 Communication unit 320 Calculation unit

Claims (9)

It ’s a computer-based method,
At the stage of assigning crowdsourcing infrastructure work for one project to multiple workers and receiving input of work results from multiple workers,
A stage in which a plurality of inspectors are allowed to inspect the work results input from the plurality of workers and the acceptance results are input from the plurality of inspectors.
Including the step of measuring the skill level of the worker based on the acceptance inspection result.
At the stage of measuring the skill level of the worker, the skill level of the worker is measured by using the difference between the number of work points corresponding to the acceptance inspection quantity of the worker and the remand point corresponding to the remand quantity.
As for the remand points, the weighted value is not applied when calculating the remand points before the work quantity of the worker exceeds the predetermined quantity at the time of skill, and the work quantity of the worker is the pre-determined quantity. After the quantity at the time of skill is exceeded, the weighted value based on the average remand rate is applied when calculating the remand points.
After the work quantity of the worker exceeds the predetermined quantity at the time of skill
When the remand rate of the worker is larger than the average remand rate, the weighted value of the first value larger than 1 is applied to calculate the remand points.
When the remand rate of the worker is smaller than the average remand rate, a weighted value of a second value smaller than 1 is applied to calculate the remand points. .. The average according to claim 1, wherein the quantity at the time of proficiency is derived by a pilot project in which the upper proficiency worker, the middle proficiency worker, and the lower proficiency worker are evenly selected. A method for measuring worker skill using the remand rate. After the work quantity of the worker exceeds the predetermined quantity at the time of skill
When the remand rate of the worker is larger than the average remand rate,
Of the total remand quantity (a), the weighted value is not applied to the quantity (b) corresponding to the average remand quantity (b), and the remaining quantity (a-) exceeding the average remand quantity (b). The method for measuring worker proficiency using the average remand rate according to claim 1, wherein the weighted value of the first value is applied in relation to b) to calculate the remand points. After the work quantity of the worker exceeds the predetermined quantity at the time of skill
When the remand rate of the worker is smaller than the average remand rate,
Based on the entire remand quantity (c), the weighted value of the second value was applied in relation to the quantity (bc) of the reduction from the average remand quantity (b), and this reduction was offset. The worker skill level measurement using the average remand rate according to claim 1, wherein the weighted value is not applied in relation to the quantity (c- (bc)) and the remand points are calculated. Method. The worker skill level measuring method using the average remand rate according to claim 1, wherein the number of work points is calculated by using the work cost paid to the worker at the time of passing the acceptance inspection. The operator skill level measuring method using the average remand rate according to claim 5, wherein the remand points are calculated by using the remand cost generated at the time of remand. The first aspect of the present invention, wherein the measurement result of the skill level of the worker is used to provide an opportunity to work on the crowdsourcing platform for other projects that require a skill level of a predetermined grade or higher. A method for measuring worker skill using the average remand rate. A computer program that is combined with a computer that is hardware and stored in a medium to execute a method for measuring worker proficiency using the average reversal rate according to any one of claims 1 to 7. It is a worker skill level measuring device that uses the average remand rate.
The work of the crowdsourcing platform for one project can be transmitted to a plurality of workers, the work results can be received from the plurality of workers, and the work results input from the plurality of workers can be inspected by the plurality of inspectors. As described above, the communication unit that transmits to the plurality of inspectors and receives the acceptance results from the plurality of inspectors.
Based on the acceptance inspection result, the number of work points corresponding to the acceptance pass quantity of the worker is calculated, the number of remand points corresponding to the remand quantity is calculated, and the skill of the worker is used by using the difference between the work points and the remand points. Including the calculation unit that calculates the degree
When calculating the number of remand points, the calculation unit
The weighted value is not applied before the work quantity of the worker exceeds the predetermined skill time quantity, and the average remand after the work quantity of the worker exceeds the predetermined skill time quantity. Calculated by applying a weighted value based on the rate
After the work quantity of the worker exceeds the predetermined quantity at the time of skill
When the remand rate of the worker is larger than the average remand rate, the weighted value of the first value larger than 1 is applied to calculate the remand points.
When the remand rate of the worker is smaller than the average remand rate, the worker skill level measuring device using the average remand rate calculates the remand points by applying a weighted value of a second value smaller than 1.