KR102155854B1 - Keyword management method and apparatus for collecting various sentences - Google Patents

Abstract

Provided are a keyword management method for collecting various sentences and a device thereof. The method comprises the steps of: requesting a task by assigning a crowdsourcing-based sentence generation task for one project to a plurality of workers; providing the plurality of workers with recommended keywords for sentence input; receiving sentences from the plurality of workers as a result of the sentence generation task; extracting a first keyword from the received sentences; determining whether the result of the sentence generation task is approved by comparing the extracted first keyword with a control keyword; and requesting an inspection by assigning the approved result of the sentence generation task to a plurality of inspectors.

Description

Keyword management method and device for collecting various sentences {KEYWORD MANAGEMENT METHOD AND APPARATUS FOR COLLECTING VARIOUS SENTENCES}

The present invention relates to a keyword management method and apparatus for collecting various sentences.

Recently, more and more companies collect and process large amounts of data on a crowdsourcing basis that engages the general public in some process of corporate activities. In other words, by opening a project and allowing the general public, that is, workers to participate in the project, necessary information is collected through the work results completed by the workers.

And, in order to collect more reliable information, the company assigns the results of work completed by the worker to the inspector to perform the inspection work.

Specifically, when one project is opened, each task is assigned to a plurality of workers. Each worker performs a plurality of tasks assigned to it, and provides the task results.

Thereafter, a plurality of inspection tasks are assigned to each of the plurality of inspectors, and each inspector performs the assigned plurality of inspection tasks.

Meanwhile, there are various types of jobs provided to workers, and among these jobs, some jobs require diversity of data. An example of such work is the work of generating a sentence that a specific person utters according to a specific scenario or a specific subject.

However, there is a limit to the words that can be used by individuals, and there are many difficulties in carrying out such a project that requires collecting various sentences.

Unexamined Patent Publication No. 10-2014-0095956, 2014.08.04.

The problem to be solved by the present invention is to provide a keyword management method and apparatus for collecting various sentences.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

A keyword management method for collecting various sentences according to an aspect of the present invention for solving the above problem is the step of assigning a crowdsourcing-based sentence generation task for one project to a plurality of workers and requesting a task, the plurality of workers Providing a recommended keyword for inputting a sentence to the user, receiving a sentence as a result of a sentence generation operation from the plurality of workers, extracting a first keyword from the input sentence, and controlling the extracted first keyword And determining whether to approve the result of the sentence generation operation by comparing with, and requesting the inspection by allocating the approved sentence generation operation result to a plurality of inspectors.

A computer program according to another aspect of the present invention for solving the above problem is combined with a computer that is hardware to execute the keyword management method for collecting various sentences, and is stored in a computer-readable recording medium.

A keyword management device for collecting various sentences according to another aspect of the present invention for solving the above problem transmits a crowdsourcing-based sentence generation task for one project to a plurality of workers, and generates sentences from the plurality of workers. A communication module for receiving the work result, transmitting the sentence generation work result input from the plurality of workers to a plurality of inspectors, and receiving the sentence generation inspection result from the plurality of inspectors, for managing keywords in the sentence collection process As the program is stored and the program stored in the memory is executed, recommended keywords for sentence input are provided to the plurality of workers, and a first keyword is selected from the sentences input as a result of the sentence generation operation from the plurality of workers. And a processor for extracting and comparing the extracted first keyword with a control keyword to determine whether to approve a work result, assigning the approved sentence generation work result to a plurality of inspectors and requesting an inspection.

Other specific details of the present invention are included in the detailed description and drawings.

According to any one of the solutions to the problem of the present invention described above, words used by various workers are collected as keywords and used as recommended keywords when writing sentences, so that more diverse sentences can be collected from workers.

This has the advantage of improving the speed of writing sentences by workers, and as a result, improving the overall speed of the project.

In addition, in order to prevent a specific word from being used intensively, when a word is used more than a certain number of times, the word is registered as a control keyword so that it is no longer used, thereby inducing various sentences to be collected.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a conceptual diagram of a crowdsourcing service according to an embodiment of the present invention.
2 is a flowchart illustrating a crowdsourcing-based project progress process according to an embodiment of the present invention.
3 is a flowchart of a keyword management method for collecting various sentences according to an embodiment of the present invention.
4 is a diagram illustrating a timeline in which keywords are provided and updated according to the degree of project progress.
5 is a diagram illustrating an example in which a recommended keyword and a control keyword are provided on a work screen.
6 is a diagram illustrating a process of updating a recommended keyword and a control keyword in an embodiment of the present invention.
7 is a diagram illustrating a keyword management device for collecting various sentences according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments are intended to complete the disclosure of the present invention, It is provided to fully inform the technician of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and “and/or” includes each and all combinations of one or more of the mentioned elements. Although "first", "second", and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram of a crowdsourcing service according to an embodiment of the present invention.

Referring to FIG. 1, a crowdsourcing service is performed by being composed of a client 10, a service provider 20, and a public 30.

The sponsor 10 refers to a company or individual who requests a crowdsourcing-based project (hereinafter, referred to as a project).

The requester 10 requests a project for the purpose of collecting source data or annotating data. Collection of source data refers to collecting raw data such as recorded voice collection and photo collection. Data annotation refers to inputting relevant annotation data into source data such as text, photos, and videos. For example, the data annotation may include finding an entity in a given fingerprint or finding a similar sentence, but is not limited thereto. Meanwhile, the types of the above-described projects are only one embodiment, and various projects may be handled in the present invention according to the design of the client.

The service provider 20 refers to a company that provides crowdsourcing services.

When the service provider 20 receives a request for a project for a product or service from the client 10, the service provider 20 allocates a task for the project to the general public 30 and receives the work result from the public 30. Thereafter, the final product extracted based on the work result is provided to the client 10.

At this time, the service provider 20 provides a crowdsourcing service to the client 10 and the public 30 through a crowdsourcing platform (hereinafter, the platform). That is, when the service provider 20 receives a request for a project from the client 10, the service provider 20 opens the project on the platform. Thereafter, when the work result for the open project is received from the public 30, the project is terminated on the platform, and the final product may be extracted and provided to the client 10.

The public 30 refers to the general public participating in the project open on the platform. Here, the public 30 may participate in a project opened on the platform through an application or website provided by the service provider 20.

The public 30 is composed of a worker 32 and an inspector 34.

The worker 32 decides to participate in a specific project among a plurality of projects open on the platform. Thereafter, the operator 32 performs an operation such as collection of source data or data annotation, and transmits it to the platform.

The inspector 34 decides to participate in a specific project among a plurality of projects open on the platform. Thereafter, the inspector 34 inspects the result of the work performed by the operator 32. As a result of the inspection, the inspector 34 can complete or reject the inspection, and can input the reason for rejection when rejecting the job.

2 is a flowchart illustrating a crowdsourcing-based project progress process according to an embodiment of the present invention.

First, the requester 10 requests one or more projects to the service provider 20 (S11).

Thereafter, the service provider 20 opens the requested project on the platform (S12). At this time, the service provider 20 may determine the grade in consideration of the difficulty of the project before opening the project. That is, depending on the difficulty level, it is possible to determine which level or higher public 30 to expose the project. Accordingly, it is possible to increase the reliability of the work result of the project.

Thereafter, the service provider 20 allocates a task to the workers 32 of the corresponding grade or higher according to the grade of the project (S13).

Thereafter, the worker 32 performs the assigned task (S14). In this case, the worker 32 may input the reason for the inability to work without performing the work for a work in which the work itself is impossible for some reason.

Thereafter, the service provider 20 receives the work result from the worker 32 (S15), and assigns the inspection work on the work result to the inspector 34 (S16).

Thereafter, the inspector 34 performs the assigned inspection (S17). At this time, the inspector 34 determines the completion of the inspection when it is determined that the work has been properly performed, and determines the rejection process when it is determined that the inspection operation is wrong. When deciding to reject, the inspector 34 inputs the rejection reason for what reason the task was judged to be wrong.

Thereafter, the service provider 20 receives the inspection result from the inspector 34 (S18).

When the inspection result is the inspection completion, the service provider 20 uses the work result as valid data, and based on this, extracts the final product at the end of the project.

If the result of the inspection is rejection processing, the service provider 20 may internally perform the inspection again or assign the work to the worker 32 again to perform the work.

Thereafter, when the project period ends or sufficient valid data is secured, the service provider 20 terminates the project (S19), calculates a final result based on the secured valid data, and provides it to the client 10 ( S20).

At this time, before the end of the project, the service provider 20 evaluates the performance results of the worker 32 and the inspector 34, calculates the work cost and the inspection cost according to the evaluation, to the worker 32 and the inspector 34. give.

Meanwhile, in FIGS. 1 and 2, the client 10, the service provider 20, the worker 32, and the inspector 34 are simply expressed, but these are smartphones, tablets, PDAs, and laptops operated by each participant. It means a computer device or a telecommunication device such as a desktop or a server.

3 is a flowchart of a keyword management method for collecting various sentences according to an embodiment of the present invention. 4 is a diagram illustrating a timeline in which keywords are provided and updated according to the degree of project progress. 5 is a diagram illustrating an example in which a recommended keyword and a control keyword are provided on a work screen.

Meanwhile, the steps shown in FIG. 3 may be understood as being performed by a platform server (hereinafter, referred to as a server) operated by the service provider 20, but are not limited thereto.

Referring to FIG. 3, the server allocates a crowdsourcing-based sentence generation task for one project to a plurality of workers 32 and requests a task (S110). A project in an embodiment of the present invention is a project for collecting various sentences, and a task belonging to the project may be a task of generating a sentence according to a specific scenario or a specific topic.

Here, one project may contain multiple jobs, and the plurality of jobs may be grouped into a plurality of tasks according to their type, or each job may be a different type of job or the same job. , But is not limited thereto.

In addition, a plurality of workers 32 or a plurality of inspectors 34 perform work using a predetermined terminal device. The terminal device of the worker 32 may be a computer device or a telecommunication device such as a smart phone, tablet, PDA, laptop, desktop, etc., but is not limited thereto.

Next, the server provides recommended keywords for inputting sentences to the plurality of workers 32 (S120).

Referring to FIG. 4, recommended keywords provided to a plurality of workers 32 may be provided after a project progress rate exceeds a preset reference rate.

This is because, at the beginning of the project, the number of keywords used for sentence generation is small and basic keywords will be used, and if the basic keywords are exposed as recommended keywords, the worker 32 may be an obstacle in using various keywords.

Accordingly, according to an embodiment of the present invention, recommended keywords may be provided to the workers 32 after the progress rate of the project exceeds a preset reference rate, for example, 10%.

In addition, one embodiment of the present invention may provide recommended keywords to all workers 32 in a batch according to the progress of the project, but unlike this, a worker who has completed writing more than the standard number of sentences among the plurality of workers 32 You can make sure that the keyword is only available to you.

In this case, the predetermined number may be preset based on the average number of sentences generated per worker before the project progress rate reaches a preset reference rate.

For example, even if the project progress rate exceeds 10%, only those who have completed writing sentences of 50% or more of the average number of sentences generated per worker before the project progress rate reaches 10% may be provided with the keyword recommendation. have.

According to the above example, if the average number of sentences generated per worker is 10, there is no exposure of the recommended keyword when entering the sentence up to the fifth sentence, and the recommended keyword may be exposed and provided from the time of entering the sentence from the sixth sentence.

By limiting the provision of recommended keywords for each time point and for each individual as described above, it is possible to prevent the workers 32 from using the recommended keywords that have already been used, and secure more various keywords at the beginning of the project, so that the workers 32 You can have them write sentences that contain more diverse keywords.

Referring to FIG. 3 again, when the sentence creation of the worker 32 is completed as recommended keywords are provided to the plurality of workers 32, the server receives the written sentence as a result of the sentence generation operation (S130).

Next, the server extracts the first keyword from the input sentence (S140), compares the extracted first keyword with the control keyword to determine whether to approve the sentence generation task result (S150), and the approved sentence generation task result Is assigned to a plurality of inspectors 34 to request inspection (S160).

Referring to FIG. 5, when the word'notebook' is set as the control keyword, when the worker 32 inputs a sentence including the word'notebook' in the sentence generation process, the server selects the keyword'notebook' from the input sentence. After extraction, since the corresponding keyword corresponds to the control keyword, the work result input by the operator 32 is not approved.

Herein, the term "non-approval" of the result of the sentence generation operation means rejection processing at the server level in which the result of the sentence generation operation is transmitted to the inspector 34 or the inspection is not performed. That is, as will be described later, only when the result of the sentence generation operation is approved, the next step of the operation, the inspection, can be performed.

For example, the server may extract the first keyword through morpheme analysis of the input sentence.

In addition, the comparison with the control keyword can be performed by extracting and comparing the first keyword as the sentence input by the operator 32 is completed, as well as the first keyword in real time even when the operator 32 is writing the sentence. Of course, it can be extracted and compared.

For example, when an operator enters the completed sentence'I wish to purchase an expensive laptop', it notifies that the control keyword has been used, or even if only'expensive laptop' is entered, the server is It is possible to inform the operator 32 that it is a control keyword.

As an example, the use of the control keyword may be notified in the form of displaying a pop-up window, but is not limited thereto. Various methods can be used as a method of notifying that the word corresponding to the control keyword is not available.

In addition to the exposure of the control keywords, recommended keywords may also be displayed on the same work screen.

A plurality of recommended keywords may be exposed and provided in a certain area on the work screen, and in some cases, each time the worker 32 clicks the recommended keyword exposure button, it is exposed, or the worker 32 manually exposes the recommended keyword. It can be exposed on the work screen in various ways, such as exposure when activated.

In addition, in an embodiment of the present invention, the recommended keyword and the control keyword may be continuously added or deleted in the course of the project according to the number of keywords stored in the keyword database.

In addition, according to an embodiment of the present invention, diversity of collected sentences may be secured by adjusting the quantity to which the recommended keyword and the control keyword are applied.

For example, the number of recommended keywords displayed on the work screen may be adjusted, or the number of recommended keywords applicable to one sentence may be adjusted. Even if a plurality of recommended keywords are stored in the keyword database, for example, three recommended keywords may be randomly determined and exposed on a work screen. Alternatively, the recommended keyword displayed on the work screen may be updated whenever one sentence is generated.

As described above, according to an embodiment of the present invention, it is possible to collect more various sentences through the recommended keyword and at the same time, the work of the worker 32 may be smoothly performed. In addition, it is possible to naturally induce a variety of sentences to be collected by preventing the distraction of a specific keyword through the control keyword.

6 is a diagram illustrating a process of updating a recommended keyword and a control keyword in an embodiment of the present invention.

First, as the approved sentence generation work result is allocated to the examiner 34, the inspection proceeds by the examiner 34, and the result of the completion of the sentence generation is stored (S210).

Next, a second keyword is extracted from the stored sentence generation task result (S220). In step S140, the first keyword is extracted from the sentence input by the operator 32, and in this step, the second keyword is inputted by the operator 32 and passed the inspection by the inspector 34, that is, a sentence generation task It is extracted from the results. In an embodiment, keyword extraction may be performed through morpheme analysis.

Next, the number of uses of each extracted second keyword is analyzed (S230), and the recommended keyword and the control keyword are updated based on the analyzed number of uses (S240).

In an embodiment, a second keyword used less than a preset maximum usable number of times as a recommended keyword may be updated, and a second keyword used more than the maximum usable number of times as a control keyword may be updated.

The recommended keyword may be a keyword used by two or more of the plurality of workers 32 at least once, targeting the second keyword used in the reviewed sentence.

In this case, if a keyword used only by one worker is used as a recommended keyword, it may be difficult for other workers to use it because the keyword may have specificity. Accordingly, in an embodiment of the present invention, in order to increase the usability of the keyword, only the keywords used by two or more workers 32 at least once can be presented as recommended keywords.

In addition, in an embodiment of the present invention, the maximum number of available times may be calculated by dividing the maximum keyword predicted quantity by the total number of keywords used in the result of generating a sentence that has been reviewed.

Here, the maximum keyword predicted quantity may be calculated by multiplying the sentence generation target quantity of the project and the average keyword predicted quantity per sentence.

In this case, according to an embodiment of the present invention, an average keyword prediction quantity per sentence may be obtained through a pilot project. A pilot project refers to a small, experimental project in advance with managers, selected workers or inspectors before officially opening the project to the public.

Alternatively, a value estimated from a similar project may be used as the average keyword predicted quantity per sentence.

An example of calculating the maximum number of times a keyword can be used is as follows.

The target number of sentences for the project is 10,000, and if the average number of predicted keywords per sentence is 5, the maximum number of predicted keywords is 50,000.

Here, if the total number of keywords used in the result of the reviewed sentence generation task is 400, the maximum number of possible uses per keyword is 125 times.

If the total number of keywords used in the reviewed sentence generation task result is 100, the maximum number of possible uses is 500, and if the total number of keywords is 500, the maximum number of possible uses is 100.

As described above, as the total number of keywords used in the result of creating a sentence that has been inspected is larger, the maximum number of keywords that can be used decreases, and if a specific keyword is used several times, the control keyword is easily reached and the sentence becomes more diverse. Conversely, the smaller the total number of keywords, the greater the maximum number of times a keyword can be used.

Since the total number of keywords used in the results of the reviewed sentence generation work increases with the progress of the project, the maximum number of keywords that can be used varies according to the time of the project.

Referring to FIG. 4, since the total number of keywords used in the result of the sentence generation task completed by the inspector 34 is varied according to the time of the project, the maximum number of times available for use is also varied according to the time of the project.

In this case, the update of the recommended keyword and the control keyword itself or the update of the maximum available number of times may be updated in real time or at preset cycles according to the project progress time point.

On the other hand, in an embodiment of the present invention, a recommended keyword and a control keyword may be set and applied based on the second keyword extracted from the sentence input by the operator 32. You can also save it and use it as a suggested keyword and a control keyword.

Alternatively, when a project similar to the corresponding project has been previously performed, the recommended keyword or control keyword used in the previous project when the project is opened may be used as a default value.

That is, the default value of the keyword may include a recommended keyword or a control keyword of another project having a predetermined similarity to the project.

Accordingly, the server may provide the recommended keywords or control keywords to the workers 32 at the beginning of the project, even before the sentences are sufficiently inputted by the workers 32.

For example, when another project is specified by the client 10 or the manager, the specified other project may be treated as having a predetermined similarity with the corresponding project.

Meanwhile, in the above description, steps S11 to S240 may be further divided into additional steps or may be combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be changed. In addition, even if other contents are omitted, the contents of FIG. 7 to be described later may be applied to the keyword management method for collecting various sentences of FIGS. 1 to 6.

Hereinafter, a keyword management apparatus 100 for collecting various sentences according to an embodiment of the present invention will be described with reference to FIG. 7.

7 is a diagram for explaining a keyword management apparatus 100 for collecting various sentences according to an embodiment of the present invention.

Referring to FIG. 7, the keyword management apparatus 100 for collecting various sentences includes a communication module 110, a memory 120, and a processor 130.

The communication module 110 transmits a crowdsourcing-based sentence generation task for one project to a plurality of workers 32, and receives a sentence generation task result from the plurality of workers 32. In addition, the sentence generation work results inputted from the plurality of workers 32 are transmitted to the plurality of inspectors 34, and the sentence generation inspection results are received from the plurality of inspectors 34.

The memory 120 stores a program for managing keywords in the sentence collection process described above.

The processor 130 executes a program stored in the memory 120. As the processor 130 executes the program stored in the memory 120, it provides recommended keywords for inputting sentences to the plurality of workers 32, and the sentences inputted from the plurality of workers 32 as a result of the sentence generation task The first keyword is extracted from Then, the extracted first keyword is compared with the control keyword to determine whether to approve the result of the sentence generation operation, and the approved result of the sentence generation operation is assigned to the plurality of inspectors 34 to request the inspection.

The keyword management apparatus 100 described with reference to FIG. 7 may be provided as a component of the above-described server.

The keyword management method according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium.

The above-described program includes C, C++, JAVA, Ruby, which can be read by a processor (CPU) of the computer through the device interface of the computer, in order for the computer to read the program and execute the methods implemented as a program. It may include a code (Code) coded in a computer language such as machine language. Such code may include a functional code related to a function that defines necessary functions for executing the methods, and includes a control code related to an execution procedure necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, these codes may further include additional information required for the processor of the computer to execute the functions, or code related to a memory reference to which location (address address) of the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server in the remote in order to execute the functions, the code uses the communication module of the computer to determine how It may further include a communication-related code for whether to communicate, what kind of information or media should be transmitted and received during communication.

The stored medium is not a medium that stores data for a short moment, such as a register, cache, memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. That is, the program may be stored in various recording media on various servers to which the computer can access, or on various recording media on the user's computer. In addition, the medium may be distributed over a computer system connected through a network, and computer-readable codes may be stored in a distributed manner.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains can understand that it is possible to easily transform it into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

10: Client
20: service provider
30: public
32: worker
34: inspector
100: keyword management device
110: communication module
120: memory
130: processor

Claims (12)

As a method performed by a computer,
Allocating a crowdsourcing-based sentence generation task for one project to a plurality of workers and requesting a task;
Providing recommended keywords for inputting sentences to the plurality of workers;
Receiving a sentence as a result of the sentence generation operation from the plurality of workers;
Extracting a first keyword from the input sentence;
Comparing the extracted first keyword with a control keyword to determine whether to approve the result of the sentence generation operation; And
The step of requesting inspection by allocating the result of the approved sentence generation work to a plurality of inspectors
Storing the result of the sentence generation operation completed by the inspector;
Extracting a second keyword from the stored sentence generating operation result;
Analyzing the number of uses for each of the extracted second keywords; And
And updating the recommended keyword and control keyword based on the analyzed number of uses,
Updating the recommended keyword and the control keyword based on the analyzed number of uses,
As a result of analyzing the number of uses for each second keyword, a second keyword used by two or more of the plurality of workers at least once is updated as the recommended keyword, and a second keyword used in excess of a preset maximum available number of times To update to the control keyword,
Keyword management method for collecting various sentences. The method of claim 1,
Providing the plurality of workers with recommended keywords for sentence input,
Keyword management method for collecting various sentences by providing recommended keywords for sentence input to the plurality of workers after the project progress rate exceeds a preset reference ratio. The method of claim 2,
Providing the plurality of workers with recommended keywords for sentence input,
Keyword management method for collecting a variety of sentences to provide a recommended keyword for the sentence input only to the worker who has completed writing more than a predetermined reference number of sentences among the plurality of workers. The method of claim 3,
The reference number is set based on the average number of sentences generated per worker before the project progress rate reaches a preset reference rate. delete delete delete The method of claim 1,
The maximum usable number of times is (a) the maximum keyword predicted quantity calculated based on the project's sentence generation target quantity and the average keyword predicted quantity per sentence, and (b) the sentence generation task result reviewed by the inspector. Keyword management method for collecting various sentences that is calculated based on the total number of keywords used. The method of claim 8,
The keyword management method for collecting various sentences, wherein the average keyword predicted quantity per sentence is obtained through a pilot project. The method of claim 8,
The maximum available number of times is a keyword management method for collecting various sentences that are variable according to the time of progress of the project. A computer program stored in a computer-readable recording medium in order to execute the keyword management method for collecting various sentences of any one of claims 1 to 4 and 8 to 10, combined with a computer as hardware. Sending crowdsourcing-based sentence generation work for one project to a plurality of workers, receiving sentence generation work results from the plurality of workers, and sending the sentence generation work results input from the plurality of workers to a plurality of inspectors And, a communication module for receiving the sentence generation inspection result from the plurality of inspectors,
A memory in which a program for managing keywords in the sentence collection process is stored; And
As the program stored in the memory is executed, recommended keywords for sentence input are provided to the plurality of workers, and a first keyword is extracted from the sentences input as a result of the sentence generation operation from the plurality of workers, and the extraction The first keyword is compared with the control keyword to determine whether to approve the result of the sentence generation operation, assigns the approved sentence generation operation result to a plurality of inspectors to request inspection, and stores the result of the sentence generation operation completed by the inspector. And a processor for extracting a second keyword from the stored sentence generation task result, analyzing the number of uses for each of the extracted second keywords, and updating the recommended keyword and the control keyword based on the analyzed number of uses, ,
Updating the recommended keyword and control keyword based on the analyzed number of uses,
As a result of analyzing the number of uses for each second keyword, a second keyword used by two or more of the plurality of workers at least once is updated as the recommended keyword, and a second keyword used in excess of a preset maximum available number of times To update to the control keyword,
Keyword management device for collecting various sentences.

Images