KR20240062065A - Method, apparatus and computer program for inspecting work quality of gig workers qa platform worker and calculating rewards - Google Patents

Abstract

A method for inspecting work quality and calculating rewards for gig worker QA platform workers according to various embodiments of the present invention is disclosed. The method includes: determining a quality level of at least one work product received from at least one worker among a plurality of workers pre-registered on the gig worker QA platform; And when the quality level is higher than a preset level value, providing a reward corresponding to the quality level to the at least one worker.

Description

Gig worker QA platform workers' work quality inspection and reward calculation methods, devices and computer programs {METHOD, APPARATUS AND COMPUTER PROGRAM FOR INSPECTING WORK QUALITY OF GIG WORKERS QA PLATFORM WORKER AND CALCULATING REWARDS}

The present invention relates to a method for inspecting the work quality of gig worker QA platform workers and calculating rewards. Specifically, the present invention relates to a method for inspecting the quality of results processed by a plurality of workers registered on the gig worker QA platform and providing rewards related thereto. It's about method.

Quality Assurance (QA) refers to performing various tests and inspections before release to ensure that games and software have a certain level of quality. This quality assurance continues throughout the entire life cycle, starting from the initial planning stage for creating games and software and including development, operation, and maintenance.

Even one fatal error hidden behind errors and various conditions that are unintentionally created during the game and software development process can change the success or failure of a project. Accordingly, quality assurance work is a very important task in the supply of games and software, and the number of companies specializing in this work is increasing.

On the other hand, due to the nature of quality assurance work, many tasks require simple testing and simple repetition, so it may be inefficient to deploy specialized personnel for all quality assurance work.

Therefore, there is a demand in the industry for a platform that can efficiently handle quality assurance tasks.

In relation to quality assurance work methods for games and software, Publication Patent Publication No. 10-2019-0019009 discloses an apparatus and method for automating game testing.

The present invention was developed in response to the above-described background technology, and is intended to provide a method for inspecting the work quality of gig worker QA platform workers and calculating rewards.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

According to an embodiment of the present invention for solving the problems described above, a method for inspecting the work quality of a gig worker QA platform worker and calculating a reward is disclosed. The method includes: determining a quality level of at least one work product received from at least one worker among a plurality of workers pre-registered on the gig worker QA platform; And if the quality level is higher than a preset level value, providing a reward corresponding to the quality level to the at least one worker; including, at least one of a plurality of workers pre-registered on the gig worker QA platform Determining a quality level of at least one work product received from a worker may include obtaining a first level associated with the at least one work product using a pre-trained quality level determination model; Assigning an inspection task to another worker to obtain a second level associated with the at least one work product; and determining the quality level based on at least one of the first level and the second level, wherein if the quality level is less than the preset level value, the at least one work result may be rejected. there is.

In an alternative embodiment, obtaining a first level associated with the at least one work product using the pre-trained quality level determination model includes inputting the work product into the quality level determination model that is a multiple classification model. , obtaining a score value for each of a plurality of classes; Recognizing a specific class with the highest score value; and determining a first level corresponding to the specific class.

In an alternative embodiment, assigning an inspection task to another worker to obtain a second level related to the at least one work product includes assigning the same test case type as the at least one work product. Assigning an inspection task for the at least one work product to another worker; and obtaining a second level for the at least one work product from the other worker.

In an alternative embodiment, determining the quality level based on at least one of the first level and the second level may include determining the quality level when a difference between the level values of the first level and the second level is 1 or less. The quality level is determined as the higher level value of the 1st level and the second level, and when the difference between the level values of the first level and the second level is greater than 1, the first level and the second level are respectively Re-acquiring; and when the difference between the level values of the first level and the second level is greater than 1, the quality level can be determined again using the re-acquired first level and the second level. there is.

In an alternative embodiment, the method further includes, if the difference between the level values of the first level and the second level obtained again is greater than 1, to another worker assigned the same test case type as the at least one work product. assigning an inspection task to the at least one work product; and obtaining a third level for the at least one work product from the other worker, and determining an average value of the first level, the second level, and the third level as the quality level. can do.

In an alternative embodiment, when the quality level is greater than or equal to a preset level value, providing a reward corresponding to the quality level to the at least one worker may include providing the reward corresponding to the quality level to the at least one worker when the quality level is greater than or equal to a preset level value. determining that the work product is complete, and determining that the at least one work product is incomplete if the quality level is less than a preset level value; and when it is determined that the at least one work product is incomplete, At least one worker may be requested to reprocess the at least one task, and the at least one task may be assigned to another worker who is assigned the same test case type as the at least one task result.

In an alternative embodiment, when the quality level is higher than a preset level value, providing a reward corresponding to the quality level to the at least one worker may include providing the reward corresponding to the quality level to the at least one worker. It may include providing a weight corresponding to the grade.

In an alternative embodiment, the worker's grade is set to an initial value of intermediate grade when performing work for the first time through the gig worker QA platform, and the number of times the work has been processed as completed is more than a preset number, or the result is If the number of times the quality level of is determined to be the highest level is more than a preset number of times, the worker's grade is raised, the number of times the work is treated as incomplete and reprocessed is more than the preset number of times, or the number of times the assigned work is rejected is more than the preset number of times. If it exceeds a preset number of times, the worker's grade may be lowered.

In an alternative embodiment, the step of providing a reward corresponding to the quality level with a weight corresponding to the grade of the at least one worker includes assigning a weight corresponding to the grade of the worker to the reward value corresponding to the quality level. and providing a reward of a size multiplied by to the at least one worker, wherein a weight corresponding to the worker's grade is preset and displayed on the gig worker QA platform.

In an alternative embodiment, the method includes receiving work results for the first work from a specific worker assigned to the first work and the second work among the plurality of workers pre-registered in the gig worker QA platform. , providing only half of the rewards associated with the first work to the specific worker; and when work results for a second work are received from the specific worker after providing only half of the reward for the first work to the specific worker, the remaining half of the reward related to the first work and the It may further include providing a reward related to the second work to the specific worker.

In an alternative embodiment, the method includes outputting the processing status of a plurality of divided works assigned to each of the plurality of workers, the processing speed ranking of each of the plurality of workers, and the quality level ranking to a dashboard; And for workers included in a preset ranking, additionally providing a reward for the ranking may be further included.

According to an embodiment of the present invention for solving the above-described problem, there is provided a memory that stores one or more instructions; and a processor that executes the one or more instructions stored in the memory, wherein the processor executes the one or more instructions, thereby performing the above-described methods.

According to one embodiment of the present invention for solving the above-described problem, a computer program is disclosed that is combined with a computer as hardware and stored in a computer-readable recording medium to perform the above-described methods.

Other specific details of the invention are included in the detailed description and drawings.

The method of inspecting the work quality of gig worker QA platform workers and calculating rewards of the present invention can support efficient quality assurance work.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram illustrating a system according to an embodiment of the present invention.
Figure 2 is a hardware configuration diagram of a server according to an embodiment of the present invention.
Figure 3 is a schematic diagram showing a network function according to an embodiment of the present invention.
Figures 4, 5, 6, and 7 are diagrams to explain an example of a QA task allocation and monitoring method for gig worker QA platform workers.
Figures 8 and 9 are diagrams to explain an example of a method for inspecting work quality and calculating compensation for gig worker QA platform workers.
Figures 10 and 11 are diagrams to explain an example of a method for assigning QA tasks and collecting work results for gig worker QA platform workers.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless clearly specifically defined.

As used in the specification, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and the “unit” or “module” performs certain roles. However, “part” or “module” is not limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium and may be configured to run on one or more processors. Thus, as an example, a “part” or “module” refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within components and “parts” or “modules” can be combined into smaller components and “parts” or “modules” or into additional components and “parts” or “modules”. Could be further separated.

In this specification, a computer refers to all types of hardware devices including at least one processor, and depending on the embodiment, it may be understood as encompassing software configurations that operate on the hardware device. For example, a computer can be understood to include, but is not limited to, a smartphone, tablet PC, desktop, laptop, and user clients and applications running on each device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and depending on the embodiment, at least part of each step may be performed in a different device.

1 is a diagram illustrating a system according to an embodiment of the present invention.

Referring to FIG. 1, a system according to an embodiment of the present invention may include a computing device 100, an operator terminal 200, and a requestor server 300.

Here, the system shown in FIG. 1 is according to one embodiment, and its components are not limited to the embodiment shown in FIG. 1, and may be added, changed, or deleted as necessary.

In one embodiment, computing device 100 may provide gig worker QA platform services. Here, a gig worker refers to a worker who provides ultra-short-term labor, such as signing a short-term contract and taking on one-time work through the gig worker QA platform.

The terms "quality assurance" and "QA (Quality Assurance)" used in the following description are used interchangeably considering only the ease of preparing specifications and do not have distinct meanings or roles.

The gig worker QA platform service of the present invention may include a service that allocates part of the quality assurance work to the gig worker, collects the tasks processed by the gig worker, and provides the results to the requester.

In one embodiment, computing device 100 may assign QA tasks for gig worker QA platform workers.

Specifically, when assigning a QA task, the computing device 100 may divide a quality assurance target work (eg, a game or software) and assign the divided work to each of a plurality of workers.

More specifically, the computing device 100 may use a pre-trained work division model when dividing work. Here, the task division model may be pre-trained with learning data including a learning work and the type of test case labeled in the learning work, but is not limited to this. Hereinafter, a description of the network functions constituting the work division model will be described later with reference to FIG. 3.

In one embodiment, computing device 100 may monitor the processing status of work assigned to a gig worker QA platform worker. Additionally, the computing device 100 may output a monitoring report to the dashboard of the gig worker QA platform. Here, the monitoring report may include a summary area including progress, special matters, confirmation requests, and QA opinions, and a detailed area containing detailed information on the progress.

In one embodiment, the computing device 100 may inspect the work quality of gig worker QA platform workers.

Specifically, when inspecting the quality of a worker's work, the computing device 100 may determine the quality of the worker's work using a level output from a pre-trained quality level determination model and a level processed by another worker. Here, the quality level determination model may be a multi-classification model that outputs score values for each of a plurality of classes, but is not limited to this. Hereinafter, a description of the network function constituting the quality level determination model will be described later with reference to FIG. 3.

In one embodiment, computing device 100 may calculate compensation for work output and provide compensation to the worker. Specifically, the computing device 100 may provide a reward in consideration of the quality level of the work processed by the worker and the worker's grade.

In one embodiment, the computing device 100 may collect work results processed by each of a plurality of workers. Here, each of the plurality of workers may be a worker holding a device in a different environment.

Specifically, the computing device 100 may collect work results processed in different device environments into one result. For example, when a specific case is performed in each of the first device environment and the second device environment for a specific divided work, the computing device 100 attaches a first tag to the work product performed in the first device environment. In addition, a second tag can be added to the work result performed in the second device environment. Additionally, the computing device 100 may collect work results to which each of the first and second tags have been added and generate the work results of a specific division work.

Hereinafter, various methods performed by the computing device 100 through the gig worker QA platform will be described with reference to FIGS. 4 to 11.

In one embodiment, the computing device 100 may be connected to the worker terminal 200 through the network 400 and may provide a gig worker QA platform for providing QA work to the worker terminal 200, Work results can be received from the worker terminal 200 through the gig worker QA platform, or rewards can be provided to the worker.

In various embodiments, the computing device 100 may provide web- or application-based services. However, it is not limited to this.

Specifically, the computing device 100 may provide a gig worker QA platform web or a gig worker QA platform application accessible from the worker terminal 200. Here, the worker terminal 200 may refer to various terminal devices such as, for example, a computer, a smartphone, a tablet PC, a desktop, or a laptop.

The worker terminal 200 includes a display in at least a portion of the worker terminal 200 and may include an operating system for running an application or extension program-based service provided by the computing device 100. For example, the worker terminal 200 may be a smartphone (Smart-phone), but is not limited to this. The worker terminal 200 is a wireless communication device that guarantees portability and mobility, and may be used for navigation, personal communication (PCS), etc. System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)- 2000, all types of handheld-based wireless communication devices such as W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartpads, tablet PCs, etc. It can be included.

The network 400 may refer to a connection structure that allows information exchange between nodes, such as a plurality of terminals and servers. For example, the network 400 includes a local area network (LAN), a wide area network (WAN), the World Wide Web (WWW), a wired and wireless data communication network, a telephone network, and a wired and wireless television communication network. do.

Wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, and Internet. (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC (Near- Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc., but is not limited thereto.

In one embodiment, the requester server 300 may be connected to the computing device 100 through the network 400, and the computing device 100 may transmit and receive various information/data necessary to provide a quality assurance service. and can store and manage various information/data generated as the computing device 100 provides services.

For example, the requestor server 300 may be a server of a game developer or software developer. That is, the requester server 300 is a server of a company that requests the quality assurance processing service of the present invention, and can provide work subject to quality assurance to the gig worker QA platform.

Hereinafter, the hardware configuration of the computing device 100 will be described with reference to FIG. 2.

Figure 2 is a hardware configuration diagram of a server according to an embodiment of the present invention.

Referring to FIG. 2, the computing device 100 according to an embodiment of the present invention includes one or more processors 110 and a memory 120 that loads a computer program 151 executed by the processor 110. , it may include a bus 130, a communication interface 140, and a storage 150 that stores a computer program 151. Here, only components related to the embodiment of the present invention are shown in Figure 2. Accordingly, anyone skilled in the art to which the present invention pertains will know that other general-purpose components may be included in addition to the components shown in FIG. 2.

The processor 110 controls the overall operation of each component of the computing device 100. The processor 110 includes a Central Processing Unit (CPU), Micro Processor Unit (MPU), Micro Controller Unit (MCU), Graphic Processing Unit (GPU), or any other type of processor well known in the art of the present invention. It can be.

Additionally, the processor 110 may perform operations on at least one application or program for executing methods according to embodiments of the present invention, and the computing device 100 may include one or more processors.

In various embodiments, the processor 110 includes random access memory (RAM) (not shown) and read memory (ROM) that temporarily and/or permanently store signals (or data) processed within the processor 110. -Only Memory, not shown) may be further included. Additionally, the processor 110 may be implemented in the form of a system on chip (SoC) that includes at least one of a graphics processing unit, RAM, and ROM.

Memory 120 stores various data, commands and/or information. Memory 120 may load a computer program 151 from storage 150 to execute methods/operations according to various embodiments of the present invention. When the computer program 151 is loaded into the memory 120, the processor 110 can perform the method/operation by executing one or more instructions constituting the computer program 151. The memory 120 may be implemented as a volatile memory such as RAM, but the technical scope of the present invention is not limited thereto.

Bus 130 provides communication functionality between components of computing device 100. The bus 130 may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

The communication interface 140 supports wired and wireless Internet communication of the computing device 100. Additionally, the communication interface 140 may support various communication methods other than Internet communication. To this end, the communication interface 140 may be configured to include a communication module well known in the technical field of the present invention. In some embodiments, communication interface 140 may be omitted.

Storage 150 may store the computer program 151 non-temporarily. When performing a precision diagnosis module proposal process through the computing device 100, the storage 150 may store various information necessary to perform analysis according to the disclosed embodiment.

The storage 150 is a non-volatile memory such as Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), flash memory, a hard disk, a removable disk, or a device well known in the art to which the present invention pertains. It may be configured to include any known type of computer-readable recording medium.

The computer program 151, when loaded into the memory 120, may include one or more instructions that cause the processor 110 to perform methods/operations according to various embodiments of the present invention. That is, the processor 110 can perform the method/operation according to various embodiments of the present invention by executing the one or more instructions.

In one embodiment, computer program 151 may include one or more instructions to perform various methods related to the gig worker QA platform service.

The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. The software module may be RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

The components of the present invention may be implemented as a program (or application) and stored in a medium in order to be executed in conjunction with a hardware computer. Components of the invention may be implemented as software programming or software elements, and similarly, embodiments may include various algorithms implemented as combinations of data structures, processes, routines or other programming constructs, such as C, C++, , may be implemented in a programming or scripting language such as Java, assembler, etc. Functional aspects may be implemented as algorithms running on one or more processors.

Below, various embodiments according to the present invention will be described.

Figure 3 is a schematic diagram showing a network function according to an embodiment of the present invention.

Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. A neural network can generally consist of a set of interconnected computational units, which can be referred to as nodes. These nodes may also be referred to as neurons. A neural network consists of at least one node. Nodes (or neurons) that make up neural networks may be interconnected by one or more links.

Within a neural network, one or more nodes connected through a link may form a relative input node and output node relationship. The concepts of input node and output node are relative, and any node in an output node relationship with one node may be in an input node relationship with another node, and vice versa. As described above, input node to output node relationships can be created around links. One or more output nodes can be connected to one input node through a link, and vice versa.

In a relationship between an input node and an output node connected through one link, the value of the data of the output node may be determined based on the data input to the input node. Here, the link connecting the input node and the output node may have a weight. Weights may be variable and may be varied by the user or algorithm in order for the neural network to perform the desired function. For example, when one or more input nodes are connected to one output node by respective links, the output node is set to the values input to the input nodes connected to the output node and the links corresponding to each input node. The output node value can be determined based on the weight.

As described above, in a neural network, one or more nodes are interconnected through one or more links to form an input node and output node relationship within the neural network. The characteristics of the neural network may be determined according to the number of nodes and links within the neural network, the correlation between the nodes and links, and the value of the weight assigned to each link. For example, if the same number of nodes and links exist and two neural networks with different weight values of the links exist, the two neural networks may be recognized as different from each other.

A neural network may consist of a set of one or more nodes. A subset of nodes that make up a neural network can form a layer. Some of the nodes constituting the neural network may form one layer based on the distances from the first input node. For example, a set of nodes with a distance n from the initial input node may constitute n layers. The distance from the initial input node can be defined by the minimum number of links that must be passed to reach the node from the initial input node. However, this definition of a layer is arbitrary for explanation purposes, and the order of a layer within a neural network may be defined in a different way than described above. For example, a layer of nodes may be defined by distance from the final output node.

The initial input node may refer to one or more nodes in the neural network through which data is directly input without going through links in relationships with other nodes. Alternatively, in the relationship between nodes based on links within a neural network, it may refer to nodes that do not have other input nodes connected by links. Similarly, the final output node may refer to one or more nodes that do not have an output node in their relationship with other nodes among the nodes in the neural network. Additionally, hidden nodes may refer to nodes constituting a neural network other than the first input node and the last output node.

The neural network according to an embodiment of the present invention is a neural network in which the number of nodes in the input layer may be the same as the number of nodes in the output layer, and the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. You can. In addition, the neural network according to another embodiment of the present invention may be a neural network in which the number of nodes in the input layer may be less than the number of nodes in the output layer, and the number of nodes decreases as it progresses from the input layer to the hidden layer. there is. In addition, the neural network according to another embodiment of the present invention is a neural network in which the number of nodes in the input layer may be greater than the number of nodes in the output layer, and the number of nodes increases as it progresses from the input layer to the hidden layer. You can. The neural network according to another embodiment of the present invention may be a neural network that is a combination of the above-described neural networks.

A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to the input layer and output layer. Deep neural networks allow you to identify latent structures in data. In other words, it is possible to identify the potential structure of a photo, text, video, voice, or music (e.g., what object is in the photo, what the content and emotion of the text are, what the content and emotion of the voice are, etc.) . Deep neural networks include convolutional neural networks (CNN), recurrent neural networks (RNN), auto encoders, generative adversarial networks (GAN), and restricted Boltzmann machines (RBM). machine), deep belief network (DBN), Q network, U network, Siamese network, Generative Adversarial Network (GAN), etc. The description of the deep neural network described above is only an example and the present invention is not limited thereto.

In one embodiment of the present invention, the network function may include an autoencoder. An autoencoder may be a type of artificial neural network to output output data similar to input data. The autoencoder may include at least one hidden layer, and an odd number of hidden layers may be placed between input and output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically and reduced from the bottleneck layer to the output layer (symmetrical to the input layer). Autoencoders can perform nonlinear dimensionality reduction. The number of input layers and output layers can be corresponded to the dimension after preprocessing of the input data. In an auto-encoder structure, the number of nodes in the hidden layer included in the encoder may have a structure that decreases as the distance from the input layer increases. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and decoder) is too small, not enough information may be conveyed, so if it exceeds a certain number (e.g., more than half of the input layers, etc.) ) may be maintained.

A neural network may be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Learning of a neural network may be a process of applying knowledge for the neural network to perform a specific operation to the neural network.

Neural networks can be trained to minimize output errors. In neural network learning, learning data is repeatedly input into the neural network, the output of the neural network and the error of the target for the learning data are calculated, and the error of the neural network is transferred from the output layer of the neural network to the input layer in the direction of reducing the error. This is the process of updating the weight of each node in the neural network through backpropagation. In the case of teacher learning, learning data in which the correct answer is labeled in each learning data is used (i.e., labeled learning data), and in the case of non-teacher learning, the correct answer may not be labeled in each learning data. That is, for example, in the case of teacher learning regarding data classification, the learning data may be data in which each learning data is labeled with a category. Labeled training data is input to the neural network, and the error can be calculated by comparing the output (category) of the neural network with the label of the training data. As another example, in the case of non-teachable learning for data classification, the error can be calculated by comparing the input training data with the neural network output. The calculated error is backpropagated in the reverse direction (i.e., from the output layer to the input layer) in the neural network, and the connection weight of each node in each layer of the neural network can be updated according to backpropagation. The amount of change in the connection weight of each updated node may be determined according to the learning rate. The neural network's calculation of input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently depending on the number of repetitions of the learning cycle of the neural network. For example, in the early stages of neural network training, a high learning rate can be used to increase efficiency by allowing the neural network to quickly achieve a certain level of performance, and in the later stages of training, a low learning rate can be used to increase accuracy.

In the learning of neural networks, the training data can generally be a subset of real data (i.e., the data to be processed using the learned neural network), and thus the error for the training data is reduced, but the error for the real data is reduced. There may be an incremental learning cycle. Overfitting is a phenomenon in which errors in actual data increase due to excessive learning on training data. For example, a phenomenon in which a neural network that learned a cat by showing a yellow cat fails to recognize that it is a cat when it sees a non-yellow cat may be a type of overfitting. Overfitting can cause errors in machine learning algorithms to increase. To prevent such overfitting, various optimization methods can be used. To prevent overfitting, methods such as increasing the learning data, regularization, dropout to disable some of the network nodes during the learning process, and use of a batch normalization layer can be applied. You can.

Figures 4, 5, 6, and 7 are diagrams to explain an example of a QA task allocation and monitoring method for gig worker QA platform workers.

According to one embodiment of the present invention, the processor 110 of the computing device 100 may allocate QA tasks for gig worker QA platform workers and monitor the workers.

Referring to FIG. 4, the computing device 100 may divide work subject to quality assurance and assign it to a plurality of workers pre-registered in the gig worker QA platform (S110). Here, a plurality of workers pre-registered in the gig worker QA platform may be users who are assigned QA tasks from the platform, perform QA tasks on their own devices, and wish to receive certain rewards.

Specifically, referring to FIG. 5 , the computing device 100 may classify works subject to quality assurance into professional works and simple works (S111). Here, a professional work may refer to work to be processed by a QA expert, and a simple work may refer to work to be assigned to a plurality of workers pre-registered on the gig worker QA platform.

For example, the computing device 100 uses a work function level table in which each test case related to a work subject to quality assurance is pre-classified, and determines which section each of the detailed works included in the work subject to quality assurance is located. It can be classified into professional work and simple work depending on whether it corresponds to or not. Here, a test case refers to a test procedure to ensure the quality of a game or software.

For example, referring to Figure 6, the business function level table 10 is divided into business function level 1 (11), business function level 2 (12), business function level 3 (13), and business function level 4 (14). It may be a table classifying multiple test cases.

The computing device 100 uses the business function level table 10 to perform simple tasks on works corresponding to business function level 1 (11) and business function level 2 (12) among test cases related to works subject to quality assurance. It can be classified as work (15), and work corresponding to work function level 3 (13) and work function level 4 (14) can be classified as professional work (16).

Referring again to FIG. 5 , as another example, the computing device 100 may generate a test case based on a work subject to quality assurance. In addition, the computing device 100 classifies the test case as a professional work if the execution time of the test case exceeds a preset time or exceeds the preset number of steps, and if the execution time of the test case is less than or equal to the preset time or the number of preset steps If it is less than a number, it can be classified as a simple work.

For example, if the execution time of a test case exceeds 30 minutes or exceeds 30 steps, the computing device 100 may classify the work of the test case as a professional work. Meanwhile, the computing device 100 may classify the work of the test case as a simple work if the execution time of the test case is 30 minutes or less or 30 steps or less.

In an additional embodiment, the computing device 100 may classify work related to a test case that includes a specific keyword among sentences for describing the test case as specialized work.

Here, the specific keyword may be a word included in a sentence more than a preset number of times among words included in a test case of a rejected work that was assigned to the worker for a preset period of time and then rejected by the worker. For example, a specific keyword may be a word that has been included more than 30 times in test cases of rejected work in the past month.

When the work subject to quality assurance is classified into a professional work and a simple work, the computing device 100 may divide the simple work to obtain a plurality of divided works (S112).

Specifically, the computing device 100 may input a simple work into a pre-learned work division model and obtain a plurality of divided works divided by type of test case. Here, the task division model may be pre-trained with training data including the training work and the type of test case labeled in the learning work.

The type of test case may include at least one of a functional test type, a local test type, a compatibility test type, a verification and featured test type, a loading time test type, and a heating test type.

Additionally, the computing device 100 may allocate a plurality of divided works to each of the plurality of workers based on at least one of their workload, preferred tasks, and skilled tasks (S113).

Specifically, the computing device 100 may recognize the current workload of each of a plurality of workers. Additionally, the computing device 100 may allocate a plurality of divided works to each of the plurality of workers based on the current workload of each of the plurality of workers and the work processing history of each of the plurality of workers.

For example, the computing device 100 may recognize the first worker among the plurality of workers to whom work can be assigned based on the current workload of each of the plurality of workers. Additionally, the computing device 100 may recognize the first type of test case with the highest verification result score in the first worker's work processing history. Additionally, the computing device 100 may allocate a first type of divided work among the plurality of divided works to the first worker.

For another example, the computing device 100 may recognize a second worker among the plurality of workers to whom work can be assigned based on the current workload of each of the plurality of workers. Additionally, the computing device 100 may recognize a second type of test case corresponding to at least one of the second worker's preferred task and skilled task. Additionally, the computing device 100 may allocate a second type of divided work among the plurality of divided works to a second worker. Here, information about the second worker's preferred tasks and skilled tasks may be information entered by the second worker when registering on the gig worker QA platform.

That is, the computing device 100 of the present invention can increase the efficiency of QA work by allocating simple work that does not require QA expertise to gig worker QA platform workers.

Referring again to FIG. 4, the computing device 100 may monitor the processing status of a plurality of divided works assigned to each of a plurality of workers and output a monitoring report on the dashboard (S120). Additionally, when the computing device 100 obtains a plurality of work results for a plurality of divided works from each of the plurality of workers, the computing device 100 may update the report (S130).

Specifically, the computing device 100 may obtain information about the current processing status by transmitting a processing status confirmation request message to each of a plurality of worker terminals at preset time periods. Additionally, the computing device 100 may receive an individual task completion message from each of the plurality of worker terminals as individual tasks of the plurality of divided works are completed in each of the plurality of worker terminals. Also, when the computing device 100 obtains information about the current processing status or receives an individual task completion message, it can update the report and output it on the dashboard.

Here, the dashboard may include a test result report output function provided to the client who requested the work subject to quality assurance. Additionally, the monitoring report may include a summary area including progress, special matters, confirmation requests, and QA opinions, and a detailed area containing detailed information on the progress.

For example, referring to Figure 7, an example screen 20 of a dashboard is shown. The dashboard screen 20 may include a monitoring report, and the monitoring report may include a summary area 21 and a detailed area 22.

In the summary area 21, progress, special matters, confirmation requests, QA opinions, etc. may be displayed in text, but are not limited to this.

The detail area 22 may display a graphical image of the test progress, progress rate, success rate, test progress details table, etc., but is not limited thereto.

Quick view buttons 23 for each test category may be displayed on the dashboard screen 20. When users select the test they want to check, they can check the summary and details about the test.

The dashboard screen 20 may include a button 24 for printing a test result report. When the user clicks the button 24, a test result report file may be downloaded to the user terminal.

As shown in FIG. 7, the dashboard screen 20 may include various functions in addition to the functions described above.

That is, the computing device 100 of the present invention can provide a dashboard that can intuitively check the work status of gig worker QA platform workers, providing convenience to each of the QA workers, QA managers, and QA clients.

In various embodiments, when obtaining a plurality of work results for a plurality of divided works from each of a plurality of workers, the computing device 100 may determine whether each of the plurality of work results corresponds to an issue. Additionally, when the computing device 100 determines that one of the plurality of work results corresponds to an issue, the computing device 100 may upload the issue to external software for tracking and correcting the issue.

That is, the computing device 100 of the present invention can cause a flexible and quick response to the issue by sharing it with external software for tracking and correcting the issue when an issue occurs during QA work.

In various embodiments, the computing device 100 may assign a plurality of divided works to each of a plurality of workers and then recognize at least one rejected work for which allocation has been rejected among the plurality of divided works. The computing device 100 may obtain a reason for rejection from the rejecting worker who rejected allocation of the divided work. Additionally, the computing device 100 may assign at least one rejected work to another worker based on the reason for rejection.

Specifically, when assigning at least one rejected work to another worker, if the reason for rejection is simple rejection, the computing device 100 may assign the work to the other worker without additional reward. Additionally, the computing device 100 may impose a penalty on the rejecting worker in which work assignment is limited to a preset number of times. Additionally, if the reason for rejection is simple rejection, the computing device 100 may assign the work to another worker without additional reward.

Meanwhile, when assigning at least one rejected work to another worker, the computing device 100 may determine the task difficulty level for the at least one rejected work if the reason for rejection is not simple rejection. Additionally, the computing device 100 may grant an additional reward corresponding to the difficulty of the task to at least one rejected work. Additionally, the computing device 100 may notify the gig worker QA platform of at least one rejected work for which an additional reward has been granted, and assign the work to another worker who wishes to process the at least one rejected work.

That is, even if the divided work is rejected, the computing device 100 of the present invention can process QA work by assigning the rejected work to another worker through various methods (e.g., reassignment or providing additional rewards). , QA work continuity can be guaranteed.

In a further embodiment, the computing device 100 may extract one or more words included in a sentence of a test case related to at least one rejected work when the reason for the rejection is not a simple rejection. Additionally, the computing device 100 may determine words that appear more than a preset number of times among words included in sentences included in test cases of rejected works during a preset period as keywords for classifying them as professional works.

For example, when classifying works subject to quality assurance into professional works and simple works, the computing device 100 specializes in works related to test cases containing specific keywords among sentences for describing test cases. It can be classified as work.

Therefore, the computing device 100 of the present invention diversifies the way to classify simple work and professional work, and excludes work from test cases with a high rejection frequency from simple work to more smoothly allocate QA work. can do.

Figures 8 and 9 are diagrams to explain an example of a method for inspecting work quality and calculating compensation for gig worker QA platform workers.

According to one embodiment of the present invention, the processor 110 of the computing device 100 may inspect the work quality of the gig worker QA platform worker and calculate the worker's reward.

Referring to FIG. 8, the computing device 100 may determine the quality level of at least one work product received from at least one worker among a plurality of workers pre-registered in the gig worker QA platform (S210). Here, the quality level is a quantification of the completeness of the work product and can be used to provide rewards to workers or determine whether the work product is complete.

Specifically, referring to FIG. 9 , the computing device 100 may obtain a first level related to at least one work product using a pre-trained quality level determination model (S211). Additionally, the computing device 100 may assign an inspection task to another worker to obtain a second level related to at least one work product (S212). And, the computing device 100 may determine the quality level based on at least one of the first level and the second level (S213).

More specifically, when obtaining the first level, the computing device 100 may obtain a score value for each of a plurality of classes by inputting the work product into a quality level determination model, which is a multi-classification model. The computing device 100 may recognize a specific class with the highest score value. And, the computing device 100 may determine the first level corresponding to a specific class.

For example, the quality level determination model may output a score for the first class as 0.2, a score for the second class as 0.1, and a score for the third class as 0.8. In this case, the computing device 100 may determine the level corresponding to the third class as the first level.

When obtaining the second level, the computing device 100 may assign an inspection task for at least one work product to another worker who is assigned the same test case type as the at least one work product. Additionally, the computing device 100 may obtain a second level of at least one work product from another worker.

That is, the computing device 100 can improve the efficiency of the inspection task by assigning the inspection task to another worker who performed the same test.

When the computing device 100 obtains the first level and the second level, the computing device 100 may determine the quality level based on at least one of the first level and the second level.

For example, when the difference between the level values of the first level and the second level is 1 or less, the computing device 100 may determine the quality level as the higher level value of the first level and the second level. Here, the higher the level value, the better the quality. For example, if the first level is 7 and the second level is 8, the computing device 100 may determine the quality level to be 8.

Meanwhile, when the difference between the level values of the first level and the second level is greater than 1, the computing device 100 may reacquire the first level and the second level, respectively. For example, when the first level is 5 and the second level is 8, the computing device 100 may reacquire each of the first level and the second level.

The computing device 100 may determine the quality level in the same manner as above using the re-obtained first level and second level.

In various embodiments, the computing device 100, when the difference between the level values of the first level and the second level obtained again is greater than 1, provides at least one work product and at least another worker assigned the same test case type. You can assign inspection tasks to one work product. Additionally, the computing device 100 may obtain a third level for at least one work product from another worker, and determine the average value of the first level, second level, and third level as the quality level.

That is, the computing device 100 of the present invention determines the quality level of the work product using the inspection results of another worker performing the same test as the level obtained from the quality level determination model, enabling double check of quality. , more accurate quality inspection can be performed.

Referring again to FIG. 8, when the quality level is higher than a preset level value, the computing device 100 may provide a reward corresponding to the quality level to at least one worker (S220). Here, if the quality level is less than a preset level value, at least one work result may be rejected.

Specifically, the computing device 100 may determine that at least one work product is completed when the quality level is higher than a preset level value, and determine that at least one work product is incomplete when the quality level is less than a preset level value. .

If the computing device 100 determines that at least one work product is incomplete, it may request reprocessing of the at least one work from at least one worker. Additionally, the computing device 100 may assign at least one task to another worker who is assigned the same test case type as the at least one task product.

That is, when the processing by the first worker is insufficient, the computing device 100 requests reprocessing from the first worker and simultaneously allocates the same work to the second worker, thereby preventing a decrease in the quality of the work product. .

In various embodiments, the computing device 100 may provide rewards by assigning weights according to the worker's grade.

Specifically, when the quality level is higher than a preset level value, the computing device 100 provides a reward corresponding to the quality level to at least one worker, and the reward corresponding to the quality level corresponds to the grade of the at least one worker. It can be provided by assigning weights such as

More specifically, a reward of a size obtained by multiplying the reward value corresponding to the quality level by a weight corresponding to the worker's grade may be provided to at least one worker. Here, the higher the weight given to the reward according to the grade, the higher the weight may be given, and the lower the weight, the lower the weight may be given. Additionally, weights corresponding to the worker's grade may be preset and displayed on the gig worker QA platform. Additionally, the worker's grade can be determined based on the work processed through the gig worker QA platform.

The initial value of the worker's grade may be applied to the intermediate grade when performing work for the first time through the gig worker QA platform. Additionally, the worker's grade may be increased if the number of times the work has been processed as completed is more than a preset number of times, or the number of times the quality level of the result is determined to be the highest level is more than a preset number of times. In addition, the worker's grade may be lowered if the number of times the work has been treated as incomplete and reprocessed is more than a preset number, or the number of times the assigned work has been rejected is more than a preset number.

In other words, the computing device 100 of the present invention can grant larger rewards as the worker's level increases, thereby increasing the worker's morale and improving the quality of the results. In this case, the productivity of QA work can increase.

In various embodiments, when two or more tasks are assigned to a worker, the computing device 100 may provide a reward for the tasks when the two or more tasks are all completed.

Specifically, when the computing device 100 receives a work product for the first work from a specific worker assigned to the first work and the second work among a plurality of workers pre-registered in the gig worker QA platform, the computing device 100 1 Only half of the rewards associated with the work can be provided to the specific worker above. And, when the computing device 100 receives a work product for a second work from a specific worker after providing only half of the reward for the first work to a specific worker, the remainder of the reward related to the first work Rewards related to half and a second piece of work can be offered to specific workers.

That is, the computing device 100 of the present invention provides a reward when all work is completed, thereby ensuring work continuity and preventing a situation in which work is not completed.

In various embodiments, the computing device 100 may display workers' work processing rankings on a dashboard and additionally provide rewards according to the rankings.

Specifically, the computing device 100 may output the processing status of a plurality of divided tasks assigned to each of the plurality of workers and the processing speed ranking and quality level ranking of each of the plurality of workers on the dashboard. Additionally, the computing device 100 may additionally provide a reward for the ranking to a worker included in a preset ranking.

In other words, the computing device 100 of the present invention can improve worker morale and increase productivity of QA work by combining entertainment elements with QA work.

In various embodiments, when it is determined that an issue with software has occurred as the worker performs QA work, the computing device 100 may provide compensation to the worker according to the occurrence of the issue. At this time, if the QA task performed by the worker is a QA task that performs testing according to a test case created by the worker, that is, the computing device 100 tests the test object according to a test case directly created by the worker, thereby If an issue occurs, additional compensation may be paid to the worker in addition to compensation for the issue.

Figures 10 and 11 are diagrams to explain an example of a method for assigning QA tasks and collecting work results for gig worker QA platform workers.

According to one embodiment of the present invention, the processor 110 of the computing device 100 may assign tasks to gig worker QA platform workers and collect the workers' work results.

Referring to FIG. 10, the computing device 100 may classify a plurality of workers pre-registered in the gig worker QA platform by device environment (S310). Here, the gig worker QA platform can obtain information about the environment of the device owned by each of the plurality of workers when each of the plurality of workers registers with the platform.

Information about the device environment includes the type of device, the manufacturer of the device, the type of OS installed on the device, the version of the OS installed on the device, the CPU of the device, the free storage capacity of the device, the capacity of RAM of the device, and It may include at least one of the networks supported by the device, but is not limited thereto.

The computing device 100 may allocate a plurality of divided pieces of work subject to quality assurance to a plurality of workers for each device environment (S320).

Specifically, the computing device 100 may allocate divided work to a plurality of workers for each device environment so that tests are performed in each device environment owned by the workers.

Meanwhile, the computing device 100 may classify work subject to quality assurance into professional work and simple work before assigning the divided work to a plurality of workers for each device environment.

For example, the computing device 100 may generate a test case based on a work subject to quality assurance. In addition, the computing device 100 classifies the test case as a professional work if the execution time of the test case exceeds a preset time or exceeds the preset number of steps, and if the execution time of the test case is less than or equal to the preset time or the number of preset steps If it is less than a number, it can be classified as a simple work.

Hereinafter, for a more detailed explanation of how to classify works subject to quality assurance into professional works and simple works, refer to the descriptions of FIGS. 4 and 5 above.

When the work subject to quality assurance is classified into a professional work and a simple work, the computing device 100 may divide the simple work to obtain a plurality of divided works.

For example, the computing device 100 may input the simple work into a pre-learned work division model to obtain the plurality of divided works divided by type of test case, but is not limited to this.

When the computing device 100 acquires a plurality of divided works, it may allocate the plurality of divided works to each worker holding devices in different environments.

Specifically, the computing device 100 may recognize the number of classified device environments in the step of classifying a plurality of workers pre-registered in the gig worker QA platform by device environment. Additionally, the computing device 100 may copy the same work as many as the number of device environments so that each of the plurality of divided works is performed in different device environments. Additionally, the computing device 100 may assign the same work to each worker holding devices in different environments.

For example, assuming that there is a first worker with a device in the first environment and a second worker with a device in the second environment, the same divided work that performs the test case of logging into the software subject to quality assurance is sent to the first worker. It can be assigned to each worker and second worker. In this case, QA work can be performed on whether login to the software subject to quality assurance is normally performed on each device in the first environment and the device in the second environment.

That is, the computing device 100 of the present invention can expand the scope of application of QA by assigning tasks to perform QA in various device environments. Additionally, the quality of quality assurance items can be guaranteed through QA in various environments.

When the computing device 100 obtains a plurality of work results for a plurality of divided works from a plurality of workers for each device environment, the computing device 100 may collect the work results processed in different device environments into one result (S330).

Specifically, referring to FIG. 11, when a specific test case is performed in each of the first device environment and the second device environment for a specific divided work, the computing device 100 may display the results of the work performed in the first device environment. A first tag may be added, and a second tag may be added to the work result performed in the second device environment (S331). Additionally, the computing device 100 may collect work results to which each of the first and second tags have been added and generate the work results of a specific division work (332).

Referring again to FIG. 10, in step S330, for example, the computing device 100 may collect work results performed with the same test case in different device environments and collect them as a unit of the test case. .

As another example, the computing device 100 may collect work results processed in different device environments into units of pre-classified test types. Here, the pre-classified test type may include at least one of a functional test type, a local test type, a compatibility test type, an inspection and featured test type, a loading time test type, and a heat test type.

In various embodiments, the computing device 100 may assign a plurality of divided works to a plurality of workers for each device environment and then recognize at least one rejected work for which allocation has been rejected among the plurality of divided works. Additionally, the computing device 100 may obtain a reason for rejection from the rejecting worker who rejected allocation of the divided work. Additionally, the computing device 100 may assign at least one rejected work to another worker based on the reason for rejection.

For example, if the reason for rejection is simple rejection, the computing device 100 may assign it to another worker without additional reward. Additionally, the computing device 100 may impose a penalty on the rejecting worker in which work assignment is limited to a preset number of times.

As another example, the computing device 100 may determine the task difficulty level for at least one rejected work when the reason for rejection is not simple rejection. Additionally, the computing device 100 may grant an additional reward corresponding to the difficulty of the task to at least one rejected work. Additionally, the computing device 100 may notify the gig worker QA platform of at least one rejected work for which an additional reward has been granted, and assign the work to another worker who wishes to process the at least one rejected work.

In a further embodiment, the computing device 100 may extract one or more words included in a sentence of a test case related to at least one rejected work when the reason for the rejection is not a simple rejection. Additionally, the computing device 100 may determine words that appear more than a preset number of times among words included in sentences included in test cases of rejected works during a preset period as keywords for classifying them as professional works.

For example, when classifying works subject to quality assurance into professional works and simple works, the computing device 100 specializes in works related to test cases containing specific keywords among sentences for describing test cases. It can be classified as work. Here, the specific keyword may be a word included in a sentence more than a preset number of times among words included in a test case of a rejected work that was assigned to the worker for a preset period of time and then rejected by the worker. For example, a specific keyword may be a word that has been included more than 30 times in test cases of rejected work in the past month.

Above, embodiments of the present invention have been described with reference to the attached drawings, but those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

Claims (13)

In a method of inspecting the work quality of a gig worker QA platform worker and calculating a reward, which is performed by a computing device including at least one processor,
determining a quality level of at least one work product received from at least one worker among a plurality of workers pre-registered in the gig worker QA platform; and
When the quality level is higher than a preset level value, providing a reward corresponding to the quality level to the at least one worker;
Including,
The step of determining the quality level of at least one work product received from at least one worker among the plurality of workers pre-registered in the gig worker QA platform,
Obtaining a first level associated with the at least one work product using a pre-trained quality level determination model;
Assigning an inspection task to another worker to obtain a second level associated with the at least one work product; and
determining the quality level based on at least one of the first level and the second level;
Including,
If the quality level is less than the preset level value, rejecting the at least one work result,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 1,
Obtaining a first level associated with the at least one work product using the pre-trained quality level determination model includes:
Inputting the work result into the quality level determination model, which is a multi-classification model, to obtain a score value for each of a plurality of classes;
Recognizing a specific class with the highest score value; and
determining a first level corresponding to the specific class;
Including,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 1,
Assigning an inspection task to another worker to obtain a second level related to the at least one work product,
Assigning an inspection task for the at least one work product to another worker assigned the same test case type as the at least one work product; and
obtaining a second level of the at least one work product from the other worker;
Including,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 1,
Determining the quality level based on at least one of the first level and the second level includes:
If the difference between the level values of the first level and the second level is 1 or less, the quality level is determined as the higher level value of the first level and the second level, and the level of the first level and the second level is determined. If the difference in values is greater than 1, reacquiring each of the first level and the second level;
Including,
When the difference between the level values of the first level and the second level is greater than 1, determining the quality level again using the re-obtained first level and the second level,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 4,
If the difference between the re-obtained level values of the first level and the second level is greater than 1, an inspection task for the at least one work product to another worker assigned to the same test case type as the at least one work product. allocating; and
Obtaining a third level for the at least one work product from the another worker, and determining an average value of the first level, the second level, and the third level as the quality level;
Containing more,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 1,
When the quality level is higher than a preset level value, providing a reward corresponding to the quality level to the at least one worker includes:
determining that the at least one work product is completed when the quality level is higher than a preset level value, and determining that the at least one work product is incomplete when the quality level is less than a preset level value;
Including,
If it is determined that the at least one work product is incomplete, requesting the at least one worker to reprocess the at least one work,
Assigning the at least one task to another worker assigned the same test case type as the at least one work product,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 1,
When the quality level is higher than a preset level value, providing a reward corresponding to the quality level to the at least one worker includes:
providing a reward corresponding to the quality level by giving a weight corresponding to the grade of the at least one worker;
Including,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 7,
The worker's grade is:
When performing work for the first time through the gig worker QA platform, the initial value is applied to a medium grade,
If the number of times a work has been processed as completed is more than a preset number of times, or the number of times the quality level of the result has been determined to be the highest level is more than a preset number of times, the worker's grade is raised,
If the number of times a work is treated as incomplete and reprocessed is more than a preset number, or the number of times assigned work is rejected is more than a preset number, the worker's grade is lowered.
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 7,
The step of providing a reward corresponding to the quality level by giving a weight corresponding to the grade of the at least one worker,
Providing a reward of a size obtained by multiplying the reward value corresponding to the quality level by a weight corresponding to the worker's grade to the at least one worker;
Including,
The weight corresponding to the worker's grade is preset and displayed on the gig worker QA platform,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 1,
When a work result for the first work is received from a specific worker assigned to the first work and the second work among a plurality of workers pre-registered on the gig worker QA platform, the reward related to the first work is received. providing only half to the specific worker; and
If work results for a second work are received from the specific worker after providing only half of the reward for the first work to the specific worker, the remaining half of the reward associated with the first work and the second work product are received from the specific worker. 2 providing a reward related to the work to the specific worker;
Containing more,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
According to claim 1,
outputting the processing status of the plurality of divided works assigned to each of the plurality of workers, the processing speed ranking of each of the plurality of workers, and the quality level ranking on a dashboard; and
For workers included in a preset ranking, additionally providing a reward for the ranking;
Containing more,
Method of inspecting the work quality of gig worker QA platform workers and calculating rewards.
A memory that stores one or more instructions; and
a processor executing the one or more instructions stored in the memory;
Including,
The processor executes the one or more instructions,
An apparatus for performing the method of claim 1.
A computer program combined with a computer as hardware and stored on a computer-readable recording medium to perform the method of claim 1.