KR102155839B1 - Method and apparatus for monitoring worker based on the reason for not wokring - Google Patents

Abstract

According to the present invention, a method for monitoring workers based on reasons for work inability includes the steps of: allocating crowdsourcing-based work regarding one single project to a plurality of workers; receiving a work outcome from the plurality of workers, while selectively receiving a work inability reason and a work inability reason keyword written in a work inability reason keyword list in regard to a work inability case of the work outcome; classifying work inability cases of each of the workers by the work inability reason keyword; and selecting workers with a fault in accordance with a ratio to a work inability case including the at least one work inability reason keyword of all work inability reasons of each of the workers. Therefore, the present invention is capable of preventing an increase in entire working time caused by a rise in inspection time.

Description

Worker monitoring method and device based on reason for work impossibility {METHOD AND APPARATUS FOR MONITORING WORKER BASED ON THE REASON FOR NOT WOKRING}

The present invention relates to a method and apparatus for monitoring a worker based on a reason for inability to work.

Recently, more and more companies are collecting and processing large amounts of data on a crowdsourcing basis that engages the general public in some process of corporate activities. In other words, by opening a project and allowing the general public, that is, workers to participate in the project, necessary information is collected through the work results completed by the workers.

And, in order to collect more reliable information, the company assigns the results of work completed by the worker to the inspector to perform the inspection work.

On the other hand, there are various types of tasks provided to workers, and various work capabilities are required to perform each task. Therefore, each worker naturally generates weak and superior parts among various types of work for each individual.

In order to compensate for this, for all jobs, the operator is allowed to select a job that cannot be done.

FIG. 1 is a diagram for explaining contents of selecting items that are not allowed to work provided by a crowdsourcing-based platform.

All tasks are assigned a'No Work' button that allows you to select No Work. When an operator does not work, the work becomes data that cannot be used.

However, from a worker's point of view, since it is merely a distinction between workable data and non-workable data, the crowdsourcing-based platform pays a certain point for the case of inability to work.

However, by exploiting this, bad workers who handle the unsuccessful work despite the fact that the work is possible to obtain more unworkable points are occurring.

In this case, the amount of usable data decreases, and as a result, a problem arises that more cost is consumed to secure valid data.

In addition, since the wrong work is rejected at the inspection stage, the rejection rate of the entire project increases, and the inspection time increases, thereby lowering the overall work speed of the inspector.

On the other hand, since there is a possibility that a job impossibility may occur, there is a problem that even if a large number of job impossibility is selected, it is not necessarily considered a defective worker.

Unexamined Patent Publication No. 10-2014-0095956, 2014.08.04.

According to an embodiment of the present invention, the keyword for the reason for the work impossibility, which is listed in the keyword list for the reason for the work impossibility, must be selected and input for the case of the worker's inability to work, and the defective worker is selected according to the ratio of the case of the non-work case including the keyword for the reason for the work impossibility. Provides a method and device for monitoring workers based on the reason why they cannot work.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

As a technical means for achieving the above-described technical problem, the method for monitoring workers based on reasons for inability to work according to the first aspect of the present invention includes the steps of granting a crowd-sourcing-based work for one project to a plurality of workers, the plurality of The work result is inputted from the operator of the above, but in the case of a work impossible case among the work results, the reason for work impossibility and the reason for work impossibility are selected and inputted in the keyword list of the reason for work impossibility, and the reason for work impossibility by each worker And classifying by keyword, and selecting a defective worker according to a ratio of the cases in which one or more of the keywords for the reason for non-work is included among the total reasons for non-work of each worker.

In one embodiment, the step of selecting and inputting the keyword for reason for the reason not to be able to work and the keyword list for reason for reason not to be able to work may include providing the keyword list for the reason for not working to each worker, and the providing from each worker. It may include the step of selecting and inputting the keyword for reason why the work is impossible based on the received keyword list of reason for not working.

In an embodiment, when the keyword for reason why the job impossibility to be input is not included in the list of keywords for the reason why the job impossibility is not included, the worker may directly input the keyword for the reason why the job is impossible.

In one embodiment, the step of selecting the defective workers may include, if the ratio of the non-working cases including any one of the work impossible reason keywords in the list of the work impossible reason keyword to the total work impossible cases of each worker is equal to or greater than a preset ratio, The worker can be selected as a defective worker.

In an embodiment, the preset ratio may be set according to reliability.

In one embodiment, the worker monitoring method according to the present invention is a step of allowing a plurality of inspectors to inspect the work results inputted from the plurality of workers, but collecting data describing the reason for work impossibility among the work results from the inspected work. , And generating a keyword list of reasons for work impossibility based on the collected data on which reasons for work impossibility are described.

In one embodiment, the step of generating the keyword list of reasons for non-work may include: extracting a plurality of keywords for reasons of non-work from the collected reason for non-work, and a keyword for main reasons for non-work among previously extracted keywords It may include the step of selecting, and updating the main reason keyword for the reason why the work impossible to the keyword list.

In an embodiment, in the step of extracting the plurality of keywords for the reason for non-workability, a word, which is a noun, may be extracted from the reason for inability to work composed of sentences.

In one embodiment, the step of selecting the main reason for unworkable keywords includes calculating an appearance frequency for each word, and selecting a word within a predetermined ranking based on the appearance frequency as the main reason for unworkability. can do.

In an embodiment, the predetermined ranking may be determined based on reliability.

In an embodiment, the main keyword for reason for unworkable may be updated in real time on the list of keywords for reason for non-work whenever it is selected.

In an embodiment, the default value of the keyword list for reason for non-work may include keywords for main reason for non-work reason of another project having a predetermined similarity to the project.

In an embodiment, in the step of collecting data describing the reason for work impossibility among the work results from the inspected work, data describing the reason for work impossibility may be collected based on the same project ID.

In one embodiment, a penalty including one or more of a warning and a suspension of work qualification may be given to a worker selected as the defective worker.

The computer program according to the second aspect of the present invention for solving the above-described problem is combined with a computer that is hardware to execute the worker monitoring method based on the reason for inability to work, and is stored in a medium.

In addition, the worker monitoring device based on the reason for the inability to work according to the third aspect of the present invention transmits a crowdsourcing-based work for one project to a plurality of workers, and receives work results from the plurality of workers, In the case of an unsuccessful case among the results, a communication module that receives the keyword of the reason for the reason not to work and the reason for the reason not to work listed in the keyword list of the reason for the work impossibility, a memory in which a program for selecting defective workers based on the data received from the communication module is stored, and the According to the execution of the program stored in the memory, each worker's unworkable cases are classified according to the above-mentioned reason for unworkable keywords, and the ratio of the non-working cases including one or more of the above-mentioned reasons for non-working among all the reasons of each worker's inability to work It includes a processor that selects bad workers according to.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

Other specific details of the present invention are included in the detailed description and drawings.

According to any one of the problem solving means of the present invention described above, by extracting the keyword for reason for work impossibility from the reason for work impossibility collected for the same project and managing it as a list of reasons for reason for work impossibility, the actual worker is unable to work when a problem occurs. The reason keyword is selected and entered, but the ratio of the case of the work impossible case including the reason keyword for the work impossible is calculated, and based on this, it is possible to select whether the corresponding worker is a defective worker or not.

In addition, there is an advantage in that the keyword for the reason for the work impossibility can be quickly and accurately updated and completed by allowing the worker to write the reason for the work impossibility and extracting and selecting the keyword for the reason for the work impossibility from the reason for the work impossibility.

In addition, by quickly identifying a defective worker and temporarily stopping the work for the defective worker, the rejection rate is increased and the inspection time is increased, thereby preventing a problem that the total working time is increased.

As a result, by selecting the defective workers, the amount of defective work is reduced, and accordingly, a bottleneck in the inspection work can be prevented, and the inspector can efficiently perform the inspection.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

FIG. 1 is a diagram for explaining contents of selecting items that are not allowed to work provided by a crowdsourcing-based platform.
2 is a conceptual diagram of a crowd sourcing service according to an embodiment of the present invention.
3 is a flowchart illustrating a crowdsourcing-based project progress process according to an embodiment of the present invention.
4 is a flowchart of a method for monitoring a worker based on a reason for inability to work according to an embodiment of the present invention.
5A to 5C are diagrams for explaining a case in which work is impossible.
FIG. 6 is an exemplary diagram for explaining inputting keywords for reason for work impossibility and reason for work impossibility according to an embodiment of the present invention.
7 is an exemplary view for explaining a monitoring result of a worker according to an embodiment of the present invention.
8 is a flowchart of a method of selecting a keyword for the reason why the work cannot be performed using the reason for non-work according to an embodiment of the present invention.
9 is a diagram illustrating a timeline from project open to end according to an embodiment of the present invention.
FIG. 10 is an exemplary diagram for explaining a pool of reason for unworkable data created for the same project according to an embodiment of the present invention.
FIG. 11 is an exemplary diagram for explaining a keyword for reason not to be able to work and a keyword for reason not to be able to work according to an embodiment of the present invention.
12 is a view for explaining an apparatus for monitoring a worker based on a reason for inability to work according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments are intended to complete the disclosure of the present invention, It is provided to fully inform the technician of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and “and/or” includes each and all combinations of one or more of the mentioned elements. Although "first", "second", and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a conceptual diagram of a crowd sourcing service according to an embodiment of the present invention.

Referring to FIG. 2, a crowd sourcing service is performed by being composed of a client 10, a service provider 20, and a public 30.

The sponsor 10 refers to a company or individual who requests a crowdsourcing-based project (hereinafter, referred to as a project).

The requester 10 requests a project for the purpose of collecting source data or annotating data. Collection of source data refers to collecting raw data such as recorded voice collection and photo collection. Data annotation refers to inputting relevant annotation data into source data such as text, photos, and videos. For example, the data annotation may include finding an entity in a given fingerprint or finding a similar sentence, but is not limited thereto. Meanwhile, the types of the above-described projects are only one embodiment, and various projects may be handled in the present invention according to the design of the client.

The service provider 20 refers to a company that provides crowdsourcing services.

When the service provider 20 receives a request for a project for a product or service from the client 10, the service provider 20 allocates a task for the project to the general public 30 and receives the work result from the public 30. Thereafter, the final product extracted based on the work result is provided to the client 10.

At this time, the service provider 20 provides a crowd sourcing service to the client 10 and the public 30 through a crowd sourcing platform (hereinafter, the platform). That is, when the service provider 20 receives a request for a project from the client 10, the service provider 20 opens the project on the platform. Thereafter, when the work result for the open project is received from the public 30, the project is terminated on the platform, and the final product may be extracted and provided to the client 10.

The public 30 refers to the general public participating in the project open on the platform. Here, the public 30 may participate in a project opened on the platform through an application or website provided by the service provider 20.

The public 30 is composed of a worker 32 and an inspector 34.

The worker 32 decides to participate in a specific project among a plurality of projects open on the platform. Thereafter, the operator 32 performs an operation such as collection of source data or data annotation, and transmits it to the platform.

The inspector 34 decides to participate in a specific project among a plurality of projects open on the platform. Thereafter, the inspector 34 inspects the result of the work performed by the operator 32. As a result of performing the inspection, the inspector 34 may pass or reject the job, and may input a reason for rejection when returning the job.

3 is a flowchart illustrating a crowdsourcing-based project progress process according to an embodiment of the present invention.

First, the requester 10 requests one or more projects to the service provider 20 (S11).

Thereafter, the service provider 20 opens the requested project on the platform (S12). At this time, the service provider 20 may determine the grade in consideration of the difficulty of the project before opening the project. That is, depending on the difficulty level, it is possible to determine which level or higher public 30 to expose the project. Accordingly, it is possible to increase the reliability of the work result of the project.

Thereafter, the service provider 20 allocates a task to the workers 32 of the corresponding grade or higher according to the grade of the project (S13).

Thereafter, the worker 32 performs the assigned task (S14). In this case, the worker 32 may input the reason for the inability to work without performing the work for a work in which the work itself is impossible for some reason. In this case, an embodiment of the present invention is characterized in that it is possible to deliberately select a reason for work impossibility to select a defective worker who does not perform some work and a worker who is truly difficult to perform the work as will be described later.

Thereafter, the service provider 20 receives the work result from the worker 32 (S15), and assigns the inspection work on the work result to the inspector 34 (S16).

Thereafter, the inspector 34 performs the assigned inspection (S17). At this time, the inspector 34 decides to pass the inspection when it is determined that the work has been properly performed, and determines rejection when it is determined that the inspection work is wrong. When deciding to reject, the inspector 34 inputs the rejection reason for what reason the task was judged to be wrong.

Thereafter, the service provider 20 receives the inspection result from the inspector 34 (S18).

If the inspection result is Pass, the service provider 20 uses the work result as valid data, and based on this, extracts the final product at the end of the project.

If the inspection result is rejected, the service provider 20 may internally perform the inspection again or assign the job again to the worker 32 to perform the job.

Thereafter, when the project period ends or sufficient valid data is secured, the service provider 20 terminates the project (S19), calculates a final result based on the secured valid data, and provides it to the client 10 ( S20).

At this time, before the end of the project, the service provider 20 evaluates the performance results of the worker 32 and the inspector 34, calculates the work cost and the inspection cost according to the evaluation, to the worker 32 and the inspector 34. give.

On the other hand, in Figures 2 and 3, although simply expressed as the client 10, the service provider 20, the worker 32, and the inspector 34, these are smartphones, tablets, PDAs, laptops, operated by each participant. It means a computer device or a telecommunication device such as a desktop or a server.

4 is a flowchart of a method for monitoring a worker based on a reason for inability to work according to an embodiment of the present invention.

Meanwhile, the steps shown in FIG. 4 may be understood as being performed by a platform server (hereinafter, referred to as a server) operated by the service provider 20, but are not limited thereto.

Referring to FIG. 4, the server assigns a crowdsourcing-based work for one project to a plurality of workers 32 (S110), and receives work results from the plurality of workers 32 (S120).

Here, one project may contain multiple jobs, and the plurality of jobs may be grouped into a plurality of tasks according to their type, or each job may be a different type of job or the same job. , But is not limited thereto.

In addition, a plurality of workers 32 or a plurality of inspectors 34 perform work using a predetermined terminal device. The terminal device of the worker 32 may be a computer device or a telecommunication device such as a smart phone, tablet, PDA, laptop, desktop, etc., but is not limited thereto.

At this time, in step S120, in the case of a work impossible case, the workers 32 select the work impossibility reason keyword listed in the work impossibility reason and the work impossibility reason keyword list, and the server selects the work impossibility reason selected by the worker 32 You will be prompted for a keyword.

Meanwhile, according to an exemplary embodiment of the present invention, the server may provide a list of keywords for reasons why the work is impossible to each worker 32 in advance.

5A to 5C are diagrams for explaining a case in which work is impossible.

In one embodiment of the present invention, the reason for work impossibility refers to a reason that the object of work cannot be performed, that is, an objective problem of source data. In addition, an embodiment of the present invention is to determine a worker who performs a job impossibility process as a defective worker even though there is no objective problem with the source data.

For example, in the case of FIG. 5A, in a project for extracting gestures from a video, the worker 32 may perform a job impossibility because the gesture does not exist in the video, which is the source data.

In the case of FIG. 5B, a project for extracting text from an image, or because the text does not exist in the image, which is the corresponding source data, the worker 32 may perform a job impossibility process.

In the case of FIG. 5C, the operator 32 works because there is no human voice in the project for recording utterances in the voice file or the voice file as the source data, and only animal cries and wind noises exist. Impossible treatment can be done.

FIG. 6 is an exemplary diagram for explaining inputting keywords for reason for work impossibility and reason for work impossibility according to an embodiment of the present invention.

Referring to FIG. 6, the server provides a list of keywords for reasons why the work is impossible and an input window for reasons why the work is impossible to the worker 32.

The worker 32 selects one keyword for reason for work impossibility from the list of keywords for reason for work impossibility, and inputs a specific reason for work impossibility.

In one embodiment, when the operator 32 is unable to perform the corresponding job and presses the job impossibility button, a list of keywords for the reason for the job impossibility is output, and the worker 32 is Select a keyword for the reason why you cannot work. In addition, the worker 32 inputs a sentence containing the selected work impossibility reason keyword or a sentence of the content related to the selected work impossibility reason keyword as the work impossibility reason.

For example, the worker 32 selects a "gesture" from a list of keywords for reasons why the work is impossible, and inputs the reason for the work impossible. In other words, it is possible to input "no gesture in the video" to include the keyword (gesture) for the reason why the selected work is impossible. Alternatively, you can enter "The video seems to have been sent incorrectly" to relate to the keyword (gesture) for the reason you selected.

As described above, when each worker 32 selects the item that is not allowed to work, they must select the keyword for the reason why it is impossible to work, which is included in the list of the reason for the reason not to work, and must describe the reason for the reason not to work in a sentence form.

If the keyword for the reason why the job impossibility to be input is not included in the keyword list for the reason why the job is impossible, the worker 32 selects "direct input" at the bottom of the list of the reason keyword for the job impossibility, as shown in FIG. 6, You can also directly enter the reason for the non-work in the input window for the reason for non-work.

In this way, the server may store the reason for the work impossible and the keyword for the reason why the work is impossible, input from the worker 32, in the reason for the work impossible data pool. The data pool for reasons why the work is impossible may be continuously updated whenever keywords for reasons why the work cannot be performed and the reason for the work impossible are directly input by the plurality of workers 32.

Accordingly, the list of keywords for reasons why the work is impossible may also be updated in real time or according to a preset period, and a detailed description thereof will be described later.

Next, the server classifies each worker 32's work impossibility by keyword (S130).

In the case of a job impossible case, it may be the case that it is included in the keyword list of reasons why the work is not possible, or it may be the case that the keyword was extracted due to the reason not included in the keyword list of the reason for the job failure. Are classified by keyword.

7 is an exemplary view for explaining the monitoring result of the operator 32 according to an embodiment of the present invention.

Referring to FIG. 7, when the total number of unworkable cases among the tasks completed by worker A is 120, the server classifies 120 unworkable cases according to the reason for the unworkable keyword.

Worker A has a total of 120 unworkable cases, 90 of which include the keyword'voice' for the reason for the inability to work, 20 unsuccessful cases that include'text', and 10 non-working cases that include'gesture'. Can be confirmed.

Next, the defective workers are selected according to the ratio of the cases in which at least one keyword of the reason for the work impossible is included among the total reasons for the work impossible of each worker 32 (S140).

At this time, when the ratio of the number of cases in which the number of cases in which any one of the keywords for reasons for inability to work is included in the list of reasons for work impossibility is greater than or equal to a preset ratio to the total number of cases in which each worker 32 is unable to work, the corresponding worker 32 is designated as a defective worker. Can be selected.

Here, the preset ratio may be preset by the server according to the reliability. That is, the server may set the corresponding ratio lower as the reliability of the level requested by the client 10 or the level required according to the nature of the project is higher.

For example, referring to FIG. 7, the total number of work impossibility of worker A is 90 cases that include the keyword'voice' for the reason for the work impossibility, 20 cases that include the keyword'text' for the reason for the work impossibility, and the keyword' It is classified into 10 cases including'gesture'.

At this time, the ratio of non-working cases containing the keyword'voice' for reasons of non-work, compared to the total non-working cases was 75%, the ratio of non-working cases containing the keyword'text' as the reason for not working is 17%, and The percentage of non-work cases involving'gesture' is 8%. In other words, worker A is applying for work impossibility due to the reason for being unable to work at a large rate.

At this time, if the ratio set in advance by the server is 60%, the ratio of non-work cases containing the keyword'voice' for the reason why the work is not possible is 60% or more, so the server cannot request work due to the reason that worker A is'voice'. Recognizing that it occurs repeatedly, worker A is judged as a defective worker. Accordingly, it is possible to select a worker who is applying for work impossibility for the same reason.

On the other hand, if the ratio set in advance by the server is 80%, the ratio of all the cases of unsuccessful work including each of the keywords for the reason for unsuccessful work is 80% or less, so that worker A is determined to be a normal worker. In this case, the server does not judge worker A as a bad worker even though there is a very high number of requests for rejection of work.

In order to prevent such a phenomenon, an embodiment of the present invention calculates the ratio of the total number of jobs to the total number of jobs, not only the number of the specific job impossibility reason keyword in this case, and when the corresponding rate is more than the aforementioned preset ratio. The worker 32 may be determined as a defective worker.

That is, the determination as a defective worker may be determined according to the ratio of each keyword for the reason for the work impossibility, or may be determined according to the ratio of the number of work impossible to the total number of work. This can be determined by the person who implements the present invention through the nature of the project, the total number of work, and preparation for the case of impossible work for each worker 32.

As described above, the worker 32 selected as a defective worker may be given a penalty including one or more of a warning and a suspension of work qualification.

For example, the server may give attention to the worker 32 selected as a defective worker through a warning, or prevent the worker 32 selected as a defective worker from performing a task until the project is finished. have.

By stopping the qualification of the worker 32 selected as a defective worker in this way, it is possible to prevent the occurrence of unsuccessful work for the same reason continuously, thereby reducing useless data, and the inspection work of the inspector 34 There is an advantage that speed and efficiency are increased.

Until now, the server allows the worker 32 to select one of the list of reasons for work impossibility, and by using the selected keyword to create the reason for work impossibility, it has been explained that the worker monitors the worker according to a preset ratio and selects defective workers. I did.

In this case, the list of keywords for reason for unworkable provided to the worker 32 is generated only when a sufficient amount of reasons for unintentional work are input from the plurality of workers 32. That is, the list of keywords for reasons why the work is impossible may be generated after the middle of a project in which a certain amount of work has been performed. Accordingly, the workers 32 are provided with a list of reasons for work impossibility after a certain amount of time elapses after the project is opened, and can select and input the reason for work impossibility.

However, if a project similar to the corresponding project has been previously carried out, a list of keywords for reasons of unworkability including the keywords of major reasons for unworkability of the previous project can be used as a default value when the project is opened. That is, the default value of the keyword list of reasons for unworkable reasons may include keywords for major reasons for unworkable reasons of other projects having a predetermined similarity to the project.

For example, the server may provide the workers 32 with a list of keywords for reasons why the previous project is impossible until the beginning of a project, that is, a list of reasons for reasons why they cannot work, is generated.

In the case of a project that is requested or performed for the first time, there is no keyword for reason for work impossibility.

In this case, there may be a case in which the workers 32 abuse this situation to deliberately perform a job impossibility for the cases in which the job can be performed.

To prevent this, according to an embodiment of the present invention, the main unworkable reason keyword of another project having a predetermined similarity to a specific project that has already been carried out may be included in the list of reasons for unworkable reason of the corresponding project and utilized.

In one embodiment, a preset similarity may be treated as having a preset similarity when it has a degree of agreement greater than or equal to a threshold value as a result of performing one or more analysis of all or part of keywords, image analysis, and summary analysis between other projects and the project. .

Alternatively, when another project is specified by the client 10 or the manager, the specified other project may be treated as having a preset similarity to the corresponding project.

Hereinafter, with reference to FIGS. 8 to 11, it will be described that a list of keywords for reasons why the work is impossible is generated at the beginning of a project, and a list of reasons for reasons why a work is not possible is updated in the middle and the second half of the project.

FIG. 8 is a flowchart of a method of selecting a keyword for the reason why the work cannot be performed using the reason for non-work according to an embodiment of the present invention, and FIG. 9 is a view showing a timeline from project opening to completion according to an embodiment of the present invention. And FIG. 10 is an exemplary diagram for explaining a pool of reason data created for the same project according to an embodiment of the present invention, and FIG. 11 is a keyword and main work impossible to work according to an embodiment of the present invention. This is an example diagram for explaining the impossible reason keywords.

Meanwhile, the steps illustrated in FIG. 8 may be understood as being performed by a platform server (hereinafter, referred to as a server) operated by the service provider 20, but are not limited thereto.

Referring to FIG. 8, according to an embodiment of the present invention, a process of collecting and updating a list of keywords for reasons that cannot be performed may be performed.

In one embodiment, the server assigns a crowdsourcing-based work for one project to a plurality of workers 32 (S110), and receives the work results from the plurality of workers 32 (S120). The above step is a step that is carried out continuously throughout the early, mid, and late of the project, and the server receives work results from multiple workers until the project is opened and closed.

Next, the server allows the plurality of inspectors 34 to inspect the work results input from the plurality of workers 32, but collects data describing the reason for the work impossibility among the work results from the finished work (S230).

That is, at the beginning of the project, since there is not enough data to generate the keyword list of reasons for work impossibility, the server receives the reason for work impossibility from the worker 32 and collects sufficient data.

At this time, the server may collect data describing the reason for the work impossibility based on the same project ID.

In one embodiment, in the case where a keyword list of reasons for unworkable reasons including a keyword for the reason why a previous project or a similar project is not included is provided as a default, the worker 32 is not allowed to work in one of the list of reasons for unworkable keywords. Select the reason keyword, and enter the reason for not working. In this case, the server may generate a keyword list of reasons for work impossibility of the project by using only the reason for work impossibility.

The collected reason for non-work may be stored in the data pool of the reason for non-work generated in the server.

That is, as shown in FIG. 9, the server may build a pool of reason data for reasons not to be able to work by collectively collecting reasons for work impossibility input from a plurality of workers at the beginning of the project.

Referring to FIG. 10, a data pool for reasons for non-work, in which reasons for non-work to which the same project ID is assigned, are recorded one row at a time, may be constructed.

For an ongoing (opened) project, the server receives the work result from the worker 32 in real time, and in this process, the data pool for reasons of work impossibility for each project is updated by collecting the reason for work impossibility in real time.

Thereafter, referring to FIG. 9, after the project is opened and a certain period of time has elapsed, the server generates a keyword list of reasons for unworkable when it is determined that the reason for unworkable has been sufficiently collected.

For example, if there are n workers 32, at least n reasons for inability to work are collected, reasons for inability to work corresponding to at least m tasks are collected, or work collected from a previous project or similar project When the reason for the work impossibility or the reason for the work impossibility of more than the same number as the keyword for the reason for the work impossibility is collected, the server may determine that the reason for the work impossibility has been sufficiently collected.

Referring to FIG. 8 again, in detail, the server may generate a keyword for the reason why the work cannot be performed based on the collected data describing the reason for the work impossible (S240).

In an exemplary embodiment, in the process of generating keywords for reasons for non-work, a plurality of keywords for reasons for non-work are extracted from the previously collected reason. In this case, a word, which is a noun, may be extracted from the reasons for impossible work composed of a plurality of keywords.

For example, referring to FIG. 10, the noun “speech” may be extracted as a keyword for the reason for the inability to work from the reason for inability to work in the sentence “no speech in the voice file”. In addition, the noun "gesture" may be extracted as a keyword for the reason why it is impossible to work from the reason for impossible work in the sentence "no gesture in the video".

Next, a major keyword for reason for unworkable reason is selected from among the plurality of keywords for reason for non-working, and the selected keyword is updated on the list of keywords for reason for non-work (S250).

In an embodiment, in the case of selecting a keyword for the reason why the main job impossibility is selected, the frequency of occurrence for each word is first calculated, and a word within a predetermined ranking may be selected as the main keyword for the reason for the inability to work based on the frequency of appearance.

For example, referring to FIG. 11, words such as'voice, text, gesture, speech, file', etc., are extracted as keywords for the reason for work impossibility from the entire reason for work impossibility, and a ranking is given and listed according to the frequency of occurrence of each word. do. In this case, the server may select a word in the 3 rankings, that is,'voice, text, gesture' as a keyword for the main reason for impossibility of work.

In this case, a predetermined ranking selected as a keyword for the main reason why the work is impossible may be a value previously set by the server according to the reliability. That is, the server may adjust the number of words selected as the main reason for unworkable keywords according to the level requested by the client 10 or the level of reliability required according to the nature of the project.

Thereafter, the server provides the worker 32 with a list of keywords for reasons why the work is impossible, including the selected keywords for reasons why the work is impossible. The worker 32 may select one keyword for the reason why the job cannot be performed using the provided keyword list for the reason why the job cannot be performed, and input the reason for the reason for the non-work by using the selected keyword for the reason for the job impossible.

Referring back to FIG. 9, the generated keyword for the main reason for non-work may be updated every predetermined period or in real time on the list of keywords for the reason for non-work whenever the keyword for the main reason for non-work is selected.

That is, the list of keywords for reason why the work is impossible to determine the defective worker is continuously updated according to the progress of the work.

From the middle of the project onwards, the server collects in real-time the unworkable reason keywords and the reason for unworkable in a pool of unworkable reason data for unworkable cases input from the plurality of workers 32. The server updates the list of keywords for reasons that cannot be worked by counting keywords for reasons that are not allowed to work in real time, collected in real-time in the data pool because of reasons for non-working.

In this case, the server allows the worker 32 to directly input the reason for the work impossibility without selecting the keyword for the reason for the work impossibility and the keyword for the reason for the work impossibility when the work is not possible.

In other words, if only the keyword for the reason why the work is impossible is selected and input, the server can no longer extract a new keyword for the reason why the work is impossible. In other words, since only the existing keywords for the selected main reason for work impossibility are continuously selected, the reason for the work impossibility of a new content cannot be input, and accordingly, the accuracy of the operator 32 monitoring is degraded.

Therefore, the server allows the worker 32 to select one keyword of the reason for non-work from the list of keywords for reason of non-work, and inputs the reason for non-work related to this in the form of a sentence. Alternatively, the reason for the work impossibility can be entered directly without selecting the keyword for the reason for the work impossibility.

Accordingly, a new keyword for the reason for the work impossibility is extracted, the main keyword for the reason for the non-work is updated in real time, and the operator is also provided with a list of keywords for the reason for the work impossibility that is updated at a predetermined time or in real time to select and input the reason for the work impossibility. This allows the server to more accurately monitor the worker 32.

For example, referring to FIG. 11, as the keyword for the reason why the work is impossible and the reason for the reason for the work impossible are input in real time, the “text” that was the second priority may be updated to the first priority. In addition, a new keyword "file" is extracted and can be ranked.

Meanwhile, in the above description, steps S11 to S250 may be further divided into additional steps or may be combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be changed. In addition, even if other contents are omitted, the contents of FIG. 12 to be described later may also be applied to the worker monitoring method based on the reason for work impossibility of FIGS. 1 to 11.

Hereinafter, with reference to FIG. 12, a description will be given of a worker monitoring apparatus 100 based on a reason for inability to work according to an embodiment of the present invention.

12 is a view for explaining the worker monitoring device 100 based on the reason for inability to work according to an embodiment of the present invention.

Referring to FIG. 12, the worker monitoring apparatus 100 based on the reason for the inability to work includes a communication module 110, a memory 120, and a processor 130.

The communication module 110 transmits a crowdsourcing-based work for one project to a plurality of workers 32 and receives work results from the plurality of workers 32. In this process, the communication module 110 receives the work impossibility reason keywords listed in the work impossibility reason and the work impossibility reason keyword list in case of a work impossibility among the work results of the worker 32.

The memory 120 stores a program for selecting a defective worker based on the data received from the communication module 110.

The processor 130 executes a program stored in the memory 120. According to the execution of the program stored in the memory 120, the processor 130 categorizes each worker 32's work impossibility by the keyword, and at least one of the total work impossibility of each worker 32 Defective workers may be selected according to the ratio of the cases in which the work impossibility reason keyword is included.

The worker monitoring apparatus 100 described with reference to FIG. 12 may be provided as a component of the above-described server.

The above-described method for monitoring a worker based on a reason for inability to work according to an embodiment of the present invention may be implemented as a program (or application) and stored in a medium to be executed by being combined with a computer that is hardware.

The above-described program includes C, C++, JAVA, Ruby, which can be read by a processor (CPU) of the computer through the device interface of the computer, in order for the computer to read the program and execute the methods implemented as a program. It may include a code (Code) coded in a computer language such as machine language. Such code may include a functional code related to a function that defines necessary functions for executing the methods, and includes a control code related to an execution procedure necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, these codes may further include additional information required for the processor of the computer to execute the functions, or code related to a memory reference to which location (address address) of the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server in the remote in order to execute the functions, the code uses the communication module of the computer to determine how It may further include a communication-related code for whether to communicate, what kind of information or media should be transmitted and received during communication.

The stored medium is not a medium that stores data for a short moment, such as a register, cache, memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. That is, the program may be stored in various recording media on various servers to which the computer can access, or on various recording media on the user's computer. In addition, the medium may be distributed over a computer system connected through a network, and computer-readable codes may be stored in a distributed manner.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains can understand that it is possible to easily transform it into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

10: Client
20: service provider
30: public
32: worker
34: inspector
100: operator monitoring device
110: communication module
120: memory
130: processor

Claims (16)

As a method performed by a computer,
Granting a crowdsourcing-based work for one project to a plurality of workers;
Receiving work results from the plurality of workers, and in the case of a work impossible case among the work results, inputting a reason for work impossibility and selecting and inputting any one of the work impossibility reason keywords included in the work impossibility reason keyword list;
Classifying each worker's work impossibility by the keyword; And
A method for monitoring a worker based on a reason for non-work, comprising the step of selecting a defective worker according to a ratio of the number of non-work cases including at least one of the keywords for the reason for the non-work relative to the total number of jobs of each worker. The method of claim 1,
The step of receiving the input of the reason for work impossibility and selecting and inputting any one of the reasons for work impossibility included in the keyword list of reasons for impossibility of work,
Providing each worker with a list of keywords for reasons why the work is impossible; And
And receiving a selection input of any one of the reasons for work impossible keywords included in the list of reasons for work impossible from the respective workers. The method of claim 1,
The worker monitoring method based on the reason for non-work, if the keyword to be input is not included in the keyword list for the reason for non-work, the worker can directly input the keyword. The method of claim 1,
The step of selecting the defective worker,
Worker monitoring method based on the reason for non-work, which selects the corresponding worker as a defective worker when the ratio of the number of non-work cases including one or more of the keywords for the reason for non-work relative to the total number of non-work cases of each worker is more than a preset ratio. The method of claim 4,
The preset ratio is set according to the reliability of the worker monitoring method based on the reason for the inability to work. The method of claim 1,
Allowing a plurality of inspectors to inspect the work results inputted from the plurality of workers, and collecting data describing the reason for non-work among the work results from the inspected work; And
The worker monitoring method based on the reason for work impossibility, further comprising the step of generating a keyword list of reasons for work impossibility based on the collected data describing the reason for the work impossibility. The method of claim 6,
The step of generating the keyword list for reasons why the work is impossible,
Extracting a plurality of unworkable reason keywords from the collected work impossibility reason;
Selecting a main keyword for reason for non-working among the extracted keywords for reason for non-working; And
And the step of updating the main reason keywords for reason not possible to work in the list of reason keywords for reason not possible to work. The method of claim 7,
The step of extracting the plurality of reasons why keywords cannot be worked,
A worker monitoring method based on the reason for the inability to work, which is to extract a word that is a noun from the reason for inability to work composed of sentences. The method of claim 8,
The step of selecting the main keyword for the reason why the work is impossible,
Calculating an appearance frequency for each word; And
And selecting a word in a predetermined ranking as a keyword for the main reason for the inability to work based on the frequency of appearance. The method of claim 9,
The predetermined ranking is determined based on the reliability of the worker monitoring method based on the reason for the inability to work. The method of claim 7,
The main reason for unworkable reason keyword is to be updated in real time on the keyword list of reasons for unworkability whenever the keyword is selected. The method of claim 11,
The default value of the keyword list of reasons for non-work is to include keywords for main reasons for non-work of other projects having a predetermined similarity to the project. The method of claim 6,
The step of collecting data describing the reason for the inability to work among the work results from the inspection completed work,
A method of monitoring workers based on reasons for non-work, which collects data on the reason for work impossibility based on the same project ID. The method of claim 1,
The worker monitoring method based on the reason for the inability to work is given a penalty including at least one of a warning and a job qualification suspension to the worker selected as the defective worker. A computer program stored in a medium in order to execute a method for monitoring a worker based on the reason for inability to work according to any one of claims 1 to 14, combined with a computer as hardware. In the worker monitoring device based on the reason for not working
Crowdsourcing-based work for one project is sent to a plurality of workers, and work results are received from the plurality of workers, but in the case of a work impossible case among the work results, the reason for work impossibility is received and included in the keyword list of the reason for work impossibility. A communication module for receiving any one of the keywords for reason why the work is impossible;
A memory storing a program for selecting defective workers based on the data received from the communication module; And
According to the execution of the program stored in the memory, each worker's unworkable cases are classified according to the work impossibility reason keyword, and the ratio of the number of unworkable cases including one or more of the above work impossibility reason keywords relative to the total work impossibility reason keywords of each worker Worker monitoring device based on the reason for the inability to work, including a processor that selects defective workers according to.

Images