KR102505411B1 - Method, apparatus and computer program for task assignment and monitoring for gig workers qa platform worker - Google Patents

Abstract

According to various embodiments of the present invention, disclosed is a QA task assigning and monitoring method for a gig worker QA platform worker. The method comprises the following steps of: dividing a quality assurance target work to allocate them to a plurality of workers pre-registered in a gig worker QA platform; monitoring a processing state of a plurality of divided workpieces assigned to each of the plurality of workers and outputting a monitoring report to a dashboard; and updating the report when obtaining a plurality of work results for the plurality of divided workpieces from each of the plurality of workers.

Description

QA task assignment and monitoring method, device and computer program for gig worker QA platform workers

The present invention relates to a QA task assignment and monitoring method for gig worker QA platform workers, and specifically, to a method of dividing and assigning quality assurance tasks to each of a plurality of workers registered in the gig worker QA platform, and monitoring them will be.

Quality Assurance (QA) refers to performing various tests and inspections prior to release so that games and software can have a certain level of quality. This quality assurance is continuously carried out throughout the entire life cycle, starting from the planning stage for making early games and software, and including development, operation, and maintenance.

In the course of game and software development, errors that are unintentionally created and fatal failures hidden behind various conditions can change the success or failure of a project. Accordingly, quality assurance work is a very important task in the supply of games and software, and an increasing number of companies specialize in this task.

On the other hand, due to the nature of the work, quality assurance work involves many tasks that require simple tests and simple repetitions, so it may be inefficient to use specialized personnel for all quality assurance work.

Therefore, there is a demand for a platform capable of efficiently processing quality assurance tasks in the art.

Regarding a method for quality assurance of games and software, Patent Publication No. 10-2019-0019009 discloses a game test automation device and method.

The present invention has been made in response to the above background art, to provide a QA task allocation and monitoring method for gig worker QA platform workers.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to one embodiment of the present invention for solving the above problems, a QA task allocation and monitoring method for gig worker QA platform workers is disclosed. The method includes: dividing a quality assurance target work and allocating it to a plurality of workers pre-registered in the gig worker QA platform; monitoring the processing status of a plurality of divided works assigned to each of the plurality of workers, and outputting a monitoring report to a dashboard; And when obtaining a plurality of work results for the plurality of divided work from each of the plurality of workers, updating the report; including, dividing the work to be guaranteed for quality to the gig worker QA platform Allocating to a plurality of pre-registered workers may include classifying the quality assurance target work into a specialized work and a simple work; Dividing the simple workpiece to obtain the plurality of divided workpieces; and allocating the plurality of divided work pieces to each of the plurality of workers based on at least one of a work load, a preferred task, and a skilled task of each of the plurality of workers.

In an alternative embodiment, the step of classifying the quality assurance target work into a specialized work and a simple work may include generating a test case based on the quality assurance target work; and when the execution time of the test case exceeds a preset time or exceeds a preset number of steps, it is classified as the specialized work, and the execution time of the test case is equal to or less than the preset time or equal to or less than the preset number of steps. In this case, classifying the simple work item; may include.

In an alternative embodiment, the step of dividing the simple workpiece to obtain the plurality of divided workpieces may include inputting the simple workpiece to a pre-learned task division model, and dividing the plurality of divided workpieces by test case type. Acquiring a division workpiece; and the task division model may be pre-trained with learning data including a learning workpiece and a test case type labeled in the learning workpiece.

In an alternative embodiment, the step of allocating the plurality of divided work to each of the plurality of workers based on at least one of a workload, a preferred task, and a skilled task of each of the plurality of workers, each of the plurality of workers recognizing the current workload of; and allocating the plurality of divided work pieces to each of the plurality of workers based on the current workload of each of the plurality of workers and the work processing history of each of the plurality of workers.

In an alternative embodiment, based on the current workload of each of the plurality of workers and the work processing history of each of the plurality of workers, the step of assigning the plurality of divided work to each of the plurality of workers, the plurality of workers Based on each current workload, recognizing a first worker capable of assigning tasks among the plurality of workers; Recognizing a first type of test case having the highest verification result score in the work process history of the first worker; and allocating the first type of divided work among the plurality of divided work to the first operator.

In an alternative embodiment, the step of allocating the plurality of divided work to each of the plurality of workers based on at least one of a workload, a preferred task, and a skilled task of each of the plurality of workers, each of the plurality of workers Recognizing a second worker from among the plurality of workers to whom work can be assigned based on the current workload of the worker; recognizing a second type of test case corresponding to at least one of a preferred task and a skilled task of the second worker; And allocating the second type of divided work among the plurality of divided work to the second worker, wherein the gig worker QA platform, when each of the plurality of workers registers on the platform, Information on the preferred task and the skilled task may be acquired.

In an alternative embodiment, the method may include, after allocating the plurality of divided workpieces to each of the plurality of workers, recognizing at least one rejected workpiece of which assignment is rejected among the plurality of divided workpieces; Obtaining a reason for refusal from a refusal worker who rejected the assignment of the division work; and allocating the at least one rejected work to another worker based on the reason for rejection.

In an alternative embodiment, the assigning of the at least one rejected work to another worker based on the reason for rejection includes, when the reason for rejection is a simple rejection, the at least one rejected work is transferred to another worker without additional reward. assigning to workers; and granting the rejected worker a penalty that is limited to a predetermined number of work assignments.

In an alternative embodiment, the assigning of the at least one rejected work to another worker based on the reason for rejection may include, when the reason for rejection is not a simple rejection, the task difficulty for the at least one rejected work determining; granting an additional reward corresponding to the difficulty level of the at least one rejected work; and notifying the gig worker QA platform of the at least one rejected work to which the additional reward has been granted, and allocating the at least one rejected work to another worker desiring to process the at least one rejected work.

In an alternative embodiment, the step of monitoring the processing status of the work assigned to each of the plurality of workers and outputting the monitoring report to the dashboard includes a processing status confirmation request message to each of the plurality of worker terminals at a predetermined time period. obtaining information on a current processing state by transmitting a; Receiving an individual job completion message from each of the plurality of worker terminals as the individual jobs of the plurality of divided work pieces are completed in each of the plurality of worker terminals; and updating the report and displaying the updated report on the dashboard when obtaining information on the current processing state or receiving the individual task completion message.

In an alternative embodiment, the dashboard includes a test result report output function provided to a client who has requested the work subject to quality assurance, and the monitoring report includes progress, unusual matters, confirmation requests, and QA opinions. It may include a summary area including , and a detailed area including details of the progress.

In an alternative embodiment, the method may include, when obtaining a plurality of work products for the plurality of divided work products from each of the plurality of workers, determining whether each of the plurality of work products corresponds to an issue; and when it is determined that any one of the plurality of work products corresponds to the issue, uploading the issue to external software for tracking and correcting the issue.

According to one embodiment of the present invention for solving the above problems, a memory for storing one or more instructions; and a processor that executes the one or more instructions stored in the memory, wherein the processor executes the one or more instructions, thereby performing the methods described above.

According to one embodiment of the present invention for solving the above problems, a computer program stored in a computer-readable recording medium to perform the above methods by being combined with a computer as hardware is disclosed.

Other specific details of the invention are included in the detailed description and drawings.

The QA task allocation and monitoring method for gig worker QA platform workers of the present invention can support efficient quality assurance tasks.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram illustrating a system according to an embodiment of the present invention.
2 is a hardware configuration diagram of a server according to an embodiment of the present invention.
3 is a schematic diagram showing a network function according to an embodiment of the present invention.
4, 5, 6 and 7 are diagrams for explaining an example of a QA task allocation and monitoring method for a gig worker QA platform worker.
8 and 9 are diagrams for explaining an example of a method for checking the work quality of a gig worker QA platform worker and calculating compensation.
10 and 11 are diagrams for explaining an example of a method of assigning QA tasks and collecting work results for gig worker QA platform workers.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only these embodiments are intended to complete the disclosure of the present invention, and are common in the art to which the present invention belongs. It is provided to fully inform the person skilled in the art of the scope of the invention, and the invention is only defined by the scope of the claims.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Like reference numerals throughout the specification refer to like elements, and “and/or” includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

The term "unit" or "module" used in the specification means a hardware component such as software, FPGA or ASIC, and "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, a “unit” or “module” may refer to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and "units" or "modules" may be combined into smaller numbers of components and "units" or "modules" or may be combined into additional components and "units" or "modules". can be further separated.

In this specification, a computer means any kind of hardware device including at least one processor, and may be understood as encompassing a software configuration operating in a corresponding hardware device according to an embodiment. For example, a computer may be understood as including a smartphone, a tablet PC, a desktop computer, a laptop computer, and user clients and applications running on each device, but is not limited thereto.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Although each step described in this specification is described as being performed by a computer, the subject of each step is not limited thereto, and at least a part of each step may be performed in different devices according to embodiments.

1 is a diagram illustrating a system according to an embodiment of the present invention.

Referring to FIG. 1 , a system according to an embodiment of the present invention may include a computing device 100 , a worker terminal 200 and a requester server 300 .

Here, the system shown in FIG. 1 is according to an embodiment, and its components are not limited to the embodiment shown in FIG. 1, and may be added, changed, or deleted as necessary.

In one embodiment, the computing device 100 may provide a gig worker QA platform service. Here, the gig worker means a worker who provides ultra-short-term labor, such as signing a short-term contract through the gig worker QA platform and taking on one-off work.

The terms "quality assurance" and "QA (Quality Assurance)" used in the following description are used interchangeably in consideration of only the ease of writing the specification, and do not have a meaning or role that is distinct from each other by itself.

The gig worker QA platform service of the present invention may include a service that allocates a part of the quality assurance task to a gig worker, collects tasks processed by the gig worker, and provides the result to the requester.

In one embodiment, computing device 100 may assign QA tasks for gig worker QA platform workers.

Specifically, when assigning QA tasks, the computing device 100 may divide quality assurance target work (eg, games or software) and allocate the divided work to each of a plurality of workers.

More specifically, the computing device 100 may use a pre-learned task division model when dividing work. Here, the task division model may be pre-learned with training data including a learning workpiece and a type of test case labeled in the learning workpiece, but is not limited thereto. Hereinafter, a description of the network function constituting the task division model will be described with reference to FIG. 3 .

In one embodiment, the computing device 100 may monitor the processing status of the work assigned to the gig worker QA platform worker. Then, the computing device 100 may output the monitoring report to the dashboard of the gig worker QA platform. Here, the monitoring report may include a summary area including progress, unusual items, confirmation request items, and QA opinions, and a detailed area including detailed contents of the progress.

In one embodiment, the computing device 100 may inspect the work quality of the gig worker QA platform worker.

Specifically, when inspecting the worker's work quality, the computing device 100 may determine the worker's work quality using a level output from a pre-learned quality level determination model and a level processed by another worker. Here, the quality level determination model may be a multi-classification model outputting score values for each of a plurality of classes, but is not limited thereto. Hereinafter, a description of the network function constituting the quality level determination model will be given with reference to FIG. 3 .

In one embodiment, the computing device 100 may calculate compensation for work results and provide compensation to workers. Specifically, the computing device 100 may provide a reward in consideration of the quality level of the work processed by the worker and the grade of the worker.

In one embodiment, the computing device 100 may collect work results processed by each of a plurality of workers. Here, each of the plurality of workers may be a worker having a device in a different environment.

Specifically, the computing device 100 may combine work results processed in different device environments into one result. For example, when a specific case is performed in each of the first device environment and the second device environment for a specific divided work, the computing device 100 assigns a first tag to a work result performed in the first device environment. In addition, a second tag may be added to a work result performed in the second device environment. Also, the computing device 100 may collect work products to which each of the first tag and the second tag are added and generate a work product of a specific division work product.

Hereinafter, descriptions of various methods performed by the computing device 100 through the gig worker QA platform will be described later with reference to FIGS. 4 to 11 .

In one embodiment, the computing device 100 may be connected to the worker terminal 200 through the network 400, and may provide a gig worker QA platform for providing QA tasks to the worker terminal 200, wherein the A work result may be received from the worker terminal 200 through the gig worker QA platform, or a reward may be provided to the worker.

In various embodiments, the computing device 100 may provide a web or application based service. However, it is not limited thereto.

Specifically, the computing device 100 may provide a gig worker QA platform web or a gig worker QA platform application accessible from the worker terminal 200 . Here, the worker terminal 200 may refer to various terminal devices such as, for example, a computer, a smart phone, a tablet PC, a desktop, and a laptop computer.

The worker terminal 200 includes a display on at least a portion of the worker terminal 200 and may include an operating system for driving services based on applications or extension programs provided from the computing device 100 . For example, the worker terminal 200 may be a smartphone (Smart-phone), but is not limited thereto, and the worker terminal 200 is a wireless communication device that ensures portability and mobility, and includes navigation, personal communication (PCS) System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)- 2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smart pad (Smartpad), tablet PC (Tablet PC), and all kinds of handheld-based wireless communication devices can include

The network 400 may refer to a connection structure capable of exchanging information between nodes such as a plurality of terminals and servers. For example, the network 400 includes a local area network (LAN), a wide area network (WAN), a world wide web (WWW), a wired and wireless data communication network, a telephone network, a wired and wireless television communication network, and the like. do.

Wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC (Near- Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. are included, but are not limited thereto.

In one embodiment, the requester server 300 may be connected to the computing device 100 through the network 400, and the computing device 100 may transmit/receive various information/data required to provide a quality assurance agency service. and store and manage various types of information/data generated as the computing device 100 provides services.

For example, the requester server 300 may be a server of a game developer or a software developer. That is, the requester server 300 is a server of a company requesting the quality assurance processing service of the present invention, and can provide a work subject to quality assurance to the gig worker QA platform.

Hereinafter, with reference to FIG. 2 , the hardware configuration of the computing device 100 will be described.

2 is a hardware configuration diagram of a server according to an embodiment of the present invention.

Referring to FIG. 2, a computing device 100 according to an embodiment of the present invention includes one or more processors 110 and a memory 120 that loads a computer program 151 executed by the processor 110. , a bus 130, a communication interface 140, and a storage 150 for storing the computer program 151. Here, in FIG. 2, only components related to the embodiment of the present invention are shown. Therefore, those skilled in the art to which the present invention pertains can know that other general-purpose components may be further included in addition to the components shown in FIG. 2 .

The processor 110 controls the overall operation of each component of the computing device 100 . The processor 110 includes a Central Processing Unit (CPU), a Micro Processor Unit (MPU), a Micro Controller Unit (MCU), a Graphic Processing Unit (GPU), or any type of processor well known in the art of the present invention. It can be.

Also, the processor 110 may perform an operation for at least one application or program for executing a method according to embodiments of the present invention, and the computing device 100 may include one or more processors.

In various embodiments, the processor 110 may temporarily and/or permanently store signals (or data) processed in the processor 110 (RAM: Random Access Memory, not shown) and ROM (ROM: Read -Only Memory, not shown) may be further included. In addition, the processor 110 may be implemented in the form of a system on chip (SoC) including at least one of a graphics processing unit, RAM, and ROM.

Memory 120 stores various data, commands and/or information. Memory 120 may load computer program 151 from storage 150 to execute methods/operations according to various embodiments of the present invention. When the computer program 151 is loaded into the memory 120, the processor 110 may perform the method/operation by executing one or more instructions constituting the computer program 151. The memory 120 may be implemented as a volatile memory such as RAM, but the technical scope of the present invention is not limited thereto.

The bus 130 provides a communication function between components of the computing device 100 . The bus 130 may be implemented in various types of buses such as an address bus, a data bus, and a control bus.

The communication interface 140 supports wired and wireless Internet communication of the computing device 100 . Also, the communication interface 140 may support various communication methods other than internet communication. To this end, the communication interface 140 may include a communication module well known in the art. In some embodiments, communication interface 140 may be omitted.

The storage 150 may non-temporarily store the computer program 151 . When the detailed diagnosis module proposal process is performed through the computing device 100, the storage 150 may store various types of information required to perform analysis according to the disclosed embodiment.

The storage 150 may be a non-volatile memory such as read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or the like, a hard disk, a removable disk, or a device well known in the art. It may be configured to include any known type of computer-readable recording medium.

Computer program 151 may include one or more instructions that when loaded into memory 120 cause processor 110 to perform methods/operations in accordance with various embodiments of the invention. That is, the processor 110 may perform the method/operation according to various embodiments of the present disclosure by executing the one or more instructions.

In one embodiment, computer program 151 may include one or more instructions to perform various methods related to gig worker QA platform service.

Steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented in a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art to which the present invention pertains.

Components of the present invention may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present invention may be implemented as software programming or software elements, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, such as C, C++ , Java (Java), can be implemented in a programming or scripting language such as assembler (assembler). Functional aspects may be implemented in an algorithm running on one or more processors.

Hereinafter, various embodiments according to the present invention will be described respectively.

3 is a schematic diagram showing a network function according to an embodiment of the present invention.

Throughout this specification, the terms computational model, neural network, network function, and neural network may be used interchangeably. A neural network may consist of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network includes one or more nodes. Nodes (or neurons) constituting neural networks may be interconnected by one or more links.

In a neural network, one or more nodes connected through a link may form a relative relationship of an input node and an output node. The concept of an input node and an output node is relative, and any node in an output node relationship with one node may have an input node relationship with another node, and vice versa. As described above, an input node to output node relationship may be created around a link. More than one output node can be connected to one input node through a link, and vice versa.

In a relationship between an input node and an output node connected through one link, the value of data of the output node may be determined based on data input to the input node. Here, a link interconnecting an input node and an output node may have a weight. The weight may be variable, and may be changed by a user or an algorithm in order to perform a function desired by the neural network. For example, when one or more input nodes are interconnected by respective links to one output node, the output node is set to a link corresponding to values input to input nodes connected to the output node and respective input nodes. An output node value may be determined based on the weight.

As described above, in the neural network, one or more nodes are interconnected through one or more links to form an input node and output node relationship in the neural network. Characteristics of the neural network may be determined according to the number of nodes and links in the neural network, an association between the nodes and links, and a weight value assigned to each link. For example, when there are two neural networks having the same number of nodes and links and different weight values of the links, the two neural networks may be recognized as different from each other.

A neural network may be composed of a set of one or more nodes. A subset of nodes constituting a neural network may constitute a layer. Some of the nodes constituting the neural network may form one layer based on distances from the first input node. For example, a set of nodes having a distance of n from the first input node may constitute n layers. The distance from the first input node may be defined by the minimum number of links that must be passed through to reach the corresponding node from the first input node. However, the definition of such a layer is arbitrary for explanation, and the order of a layer in a neural network may be defined in a method different from the above. For example, a layer of nodes may be defined by a distance from a final output node.

An initial input node may refer to one or more nodes to which data is directly input without going through a link in relation to other nodes among nodes in the neural network. Alternatively, in a relationship between nodes based on a link in a neural network, it may mean nodes that do not have other input nodes connected by a link. Similarly, the final output node may refer to one or more nodes that do not have an output node in relation to other nodes among nodes in the neural network. Also, the hidden node may refer to nodes constituting the neural network other than the first input node and the last output node.

In the neural network according to an embodiment of the present invention, the number of nodes in the input layer may be the same as the number of nodes in the output layer, and the number of nodes decreases and then increases again as the number of nodes progresses from the input layer to the hidden layer. can In addition, the neural network according to another embodiment of the present invention may be a neural network in which the number of nodes of the input layer may be less than the number of nodes of the output layer, and the number of nodes decreases as the number of nodes increases from the input layer to the hidden layer. there is. In addition, the neural network according to another embodiment of the present invention is a type of neural network in which the number of nodes in the input layer may be greater than the number of nodes in the output layer, and the number of nodes increases as the number of nodes progresses from the input layer to the hidden layer. can A neural network according to another embodiment of the present invention may be a neural network in the form of a combination of the aforementioned neural networks.

A deep neural network (DNN) may refer to a neural network including a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks can reveal latent structures in data. In other words, it can identify the latent structure of a photo, text, video, sound, or music (e.g., what objects are in the photo, what the content and emotion of the text are, what the content and emotion of the audio are, etc.). . Deep neural networks include convolutional neural networks (CNNs), recurrent neural networks (RNNs), auto encoders, generative adversarial networks (GANs), and restricted boltzmann machines (RBMs). machine), a deep belief network (DBN), a Q network, a U network, a Siamese network, a Generative Adversarial Network (GAN), and the like. The description of the deep neural network described above is only an example, and the present invention is not limited thereto.

In one embodiment of the present invention, the network function may include an autoencoder. An autoencoder may be a type of artificial neural network for outputting output data similar to input data. An auto-encoder may include at least one hidden layer, and an odd number of hidden layers may be disposed between input and output layers. The number of nodes of each layer may be reduced from the number of nodes of the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically with the reduction from the bottleneck layer to the output layer (symmetrical to the input layer). Autoencoders can perform non-linear dimensionality reduction. The number of input layers and output layers may correspond to dimensions after preprocessing of input data. In the auto-encoder structure, the number of hidden layer nodes included in the encoder may decrease as the distance from the input layer increases. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and decoder) is too small, a sufficient amount of information may not be conveyed, so more than a certain number (e.g., more than half of the input layer, etc.) ) may be maintained.

The neural network may be trained using at least one of supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. Learning of the neural network may be a process of applying knowledge for the neural network to perform a specific operation to the neural network.

A neural network can be trained in a way that minimizes output errors. In the learning of the neural network, the learning data is repeatedly input into the neural network, the output of the neural network for the training data and the error of the target are calculated, and the error of the neural network is transferred from the output layer of the neural network to the input layer in the direction of reducing the error. It is a process of updating the weight of each node of the neural network by backpropagating in the same direction. In the case of teacher learning, the learning data in which the correct answer is labeled is used for each learning data (ie, the labeled learning data), and in the case of comparative teacher learning, the correct answer may not be labeled in each learning data. That is, for example, learning data in the case of teacher learning regarding data classification may be data in which each learning data is labeled with a category. Labeled training data is input to a neural network, and an error may be calculated by comparing an output (category) of the neural network and a label of the training data. As another example, in the case of comparative history learning for data classification, an error may be calculated by comparing input learning data with a neural network output. The calculated error is back-propagated in a reverse direction (ie, from the output layer to the input layer) in the neural network, and the connection weight of each node of each layer of the neural network may be updated according to the back-propagation. The amount of change in the connection weight of each updated node may be determined according to a learning rate. The neural network's computation of input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently according to the number of iterations of the learning cycle of the neural network. For example, a high learning rate may be used in the early stage of neural network training to increase efficiency by allowing the neural network to quickly obtain a certain level of performance, and a low learning rate may be used in the late stage to increase accuracy.

In neural network learning, generally, training data can be a subset of real data (ie, data to be processed using the trained neural network), and therefore, errors for training data are reduced, but errors for real data are reduced. There may be incremental learning cycles. Overfitting is a phenomenon in which errors for actual data increase due to excessive learning on training data. For example, a phenomenon in which a neural network that has learned a cat by showing a yellow cat does not recognize that it is a cat when it sees a cat other than yellow may be a type of overfitting. Overfitting can act as a cause of increasing the error of machine learning algorithms. Various optimization methods can be used to prevent such overfitting. To prevent overfitting, methods such as increasing the training data, regularization, inactivating some nodes in the network during learning, and using a batch normalization layer should be applied. can

4, 5, 6 and 7 are diagrams for explaining an example of a QA task allocation and monitoring method for a gig worker QA platform worker.

According to one embodiment of the present invention, the processor 110 of the computing device 100 may allocate QA tasks for gig worker QA platform workers and monitor the workers.

Referring to FIG. 4 , the computing device 100 may divide the quality assurance target work and allocate it to a plurality of workers pre-registered in the gig worker QA platform (S110). Here, a plurality of workers pre-registered in the gig worker QA platform may be users who want to be assigned QA tasks from the platform, perform QA tasks on their own devices, and receive a certain reward.

Specifically, referring to FIG. 5 , the computing device 100 may classify the quality assurance target work into a specialized work and a simple work (S111). Here, the specialized work may refer to a work to be processed by a QA expert, and the simple work may refer to a work to be assigned to a plurality of workers pre-registered in the gig worker QA platform.

For example, the computing device 100 uses a work function level table in which each test case related to the work subject to quality assurance is pre-classified, and each detailed work included in the work subject to quality assurance is determined in which section. Depending on whether or not it falls under, it can be classified as professional work and simple work. Here, the test case means a test procedure for guaranteeing the quality of a game or software.

For example, referring to FIG. 6, the business function level table 10 is divided into business function level 1 (11), business function level 2 (12), business function level 3 (13), and business function level 4 (14). It may be a table in which a plurality of test cases are classified.

The computing device 100 uses the business function level table 10 to perform simple tasks on tasks corresponding to the first business function level 11 and the second business function level 12 among test cases related to the work subject to quality assurance. It is classified as water (15), and the work corresponding to the business function level 3 (13) and the business function level 4 (14) can be classified as professional work (16).

Referring back to FIG. 5 , as another example, the computing device 100 may generate a test case based on a quality assurance target work. In addition, the computing device 100 classifies the test case as a specialized work when the execution time of the test case exceeds the preset time or exceeds the preset number of steps, and the test case execution time is less than the preset time or the preset number of steps. If it is less than a number, it can be classified as a simple work.

For example, if the execution time of the test case exceeds 30 minutes or exceeds 30 steps, the computing device 100 may classify the work of the corresponding test case as a specialized work. Meanwhile, when the execution time of the test case is 30 minutes or less or 30 steps or less, the computing device 100 may classify the work of the corresponding test case as a simple work.

In an additional embodiment, the computing device 100 may classify a work related to a test case including a specific keyword among sentences for describing the test case as a specialized work.

Here, the specific keyword may be a word included a predetermined number of times or more among words included in a sentence included in a test case of a rejected work that was assigned to a worker for a predetermined period of time and then rejected by the worker. For example, a specific keyword may be a word included more than 30 times in test cases of rejected work in the last month.

When the quality assurance target work is classified into a specialized work and a simple work, the computing device 100 may acquire a plurality of divided work by dividing the simple work (S112).

Specifically, the computing device 100 may obtain a plurality of divisional workpieces divided according to test case types by inputting a simple workpiece to a pre-learned task division model. Here, the task division model may be pre-trained with training data including a learning work and a test case type labeled in the learning work.

The test case type may include at least one of a functional test type, a local test type, a compatibility test type, an inspection and featured test type, a loading time test type, and an overheating test type.

In addition, the computing device 100 may allocate a plurality of divided work pieces to each of the plurality of workers based on at least one of a workload, a preferred task, and a skilled task of each of the plurality of workers (S113).

Specifically, the computing device 100 may recognize the current workload of each of a plurality of workers. Then, the computing device 100 may allocate a plurality of divided work to each of the plurality of workers based on the current workload of each of the plurality of workers and the work processing history of each of the plurality of workers.

For example, the computing device 100 may recognize a first worker to whom work can be assigned from among a plurality of workers based on the current workload of each of the plurality of workers. In addition, the computing device 100 may recognize the first type of test case having the highest verification result score in the work process history of the first worker. And, the computing device 100 may allocate a first type of division work among a plurality of division work to the first worker.

For another example, the computing device 100 may recognize a second worker to whom work can be assigned from among the plurality of workers based on the current workload of each of the plurality of workers. In addition, the computing device 100 may recognize a second type of test case corresponding to at least one of a preferred task and a skilled task of the second worker. Also, the computing device 100 may allocate a second type of division work among a plurality of division work to a second worker. Here, the information about the second worker's preferred task and skilled task may be information entered when the second worker registers with the gig worker QA platform.

That is, the computing device 100 of the present invention can increase the efficiency of QA work by allocating simple work that does not require QA expertise to gig worker QA platform workers.

Referring back to FIG. 4 , the computing device 100 may monitor a processing state of a plurality of divided work items assigned to each of a plurality of workers, and output a monitoring report to the dashboard (S120). In addition, the computing device 100 may update a report when obtaining a plurality of work results for a plurality of divided workpieces from each of a plurality of workers (S130).

Specifically, the computing device 100 may obtain information on the current processing state by transmitting a processing state check request message to each of a plurality of worker terminals at predetermined time intervals. In addition, the computing device 100 may receive an individual job completion message from each of a plurality of worker terminals as individual jobs of a plurality of divided work pieces are completed in each of a plurality of worker terminals. In addition, the computing device 100 may obtain information on a current processing state or update a report and output it on a dashboard when receiving an individual task completion message.

Here, the dashboard may include a test result report output function provided to a requester who has requested a work subject to quality assurance. In addition, the monitoring report may include a summary area including progress, unusual matters, confirmation request items, and QA opinions, and a detailed area including detailed contents of the progress.

For example, referring to FIG. 7 , an example screen 20 of a dashboard is shown. The dashboard screen 20 may include a monitoring report, and the monitoring report may include a summary area 21 and a detailed area 22 .

In the summary area 21, progress, unusual matters, confirmation requested items, QA opinions, etc. may be displayed as text, but is not limited thereto.

In the detailed area 22, an image representing a test progress as a graph, a progress rate, a success rate, a test progress detailed table, etc. may be displayed, but is not limited thereto.

Quick view buttons 23 for each test category may be displayed on the dashboard screen 20 . When a user selects a test to check, the user can check the summary and details of the test.

The dashboard screen 20 may include a button 24 for outputting a test result report. When the user clicks the button 24, the test result report file may be downloaded to the user terminal.

As shown in FIG. 7 , the dashboard screen 20 may include various functions in addition to the functions described above.

That is, the computing device 100 of the present invention provides a dashboard that can intuitively check the work status of gig worker QA platform workers, providing convenience to each QA worker, QA manager, and QA requestor.

In various embodiments, the computing device 100 may determine whether each of the plurality of work products corresponds to an issue when obtaining a plurality of work products for a plurality of divided work products from each of a plurality of workers. And, when it is determined that any one of the plurality of work products corresponds to an issue, the computing device 100 may upload the issue to external software for tracking and correcting the issue.

That is, when an issue occurs during QA work, the computing device 100 of the present invention can cause a flexible and quick response to the issue by sharing it with external software for tracking and correcting the issue.

In various embodiments, the computing device 100 may allocate a plurality of divided work to each of a plurality of workers, and then recognize at least one rejected work from among the plurality of divided work. The computing device 100 may obtain a reason for refusal from the refusal worker who has rejected the allocation of the division work. In addition, the computing device 100 may allocate at least one rejected work to another worker based on the reason for rejection.

Specifically, when allocating at least one rejected work to another worker, the computing device 100 may allocate the at least one rejected work to the other worker without an additional reward when the reason for the rejection is simple rejection. In addition, the computing device 100 may impose a penalty limiting the number of work assignments to the rejected worker by a preset number of times. In addition, when the reason for rejection is simple rejection, the computing device 100 may allocate the worker to another worker without an additional reward.

Meanwhile, when allocating at least one rejected work to another worker, the computing device 100 may determine the difficulty level of the at least one rejected work when the reason for rejection is not simple rejection. In addition, the computing device 100 may grant an additional reward corresponding to the difficulty level of the at least one rejected job. Then, the computing device 100 may notify the gig worker QA platform of at least one rejected work to which an additional reward has been granted, and assign it to another worker who wishes to process the at least one rejected work.

That is, even if the divided work is rejected, the computing device 100 of the present invention can process the QA task by assigning the rejected work to another worker through various methods (eg, reassignment or providing additional rewards) , QA business continuity can be guaranteed.

In an additional embodiment, the computing device 100 may extract one or more words included in a sentence of a test case related to at least one rejected work when the reason for rejection is not simple rejection. In addition, the computing device 100 may determine a word that appears more than a predetermined number of times among words included in sentences included in test cases of rejected work during a predetermined period as a keyword for classifying as a specialized work.

For example, when the computing device 100 classifies the quality assurance target work into a specialized work and a simple work, the work related to the test case including a specific keyword among sentences for describing the test case is specialized. It can be classified as work.

Therefore, the computing device 100 of the present invention diversifies the method of classifying simple work and specialized work, and excludes the work of test cases with high rejection frequency from the simple work, and allocates QA tasks more smoothly. can do.

8 and 9 are diagrams for explaining an example of a method for checking the work quality of a gig worker QA platform worker and calculating compensation.

According to one embodiment of the present invention, the processor 110 of the computing device 100 may inspect the work quality of the gig worker QA platform worker, and calculate the worker's reward.

Referring to FIG. 8 , the computing device 100 may determine a quality level of at least one work result received from at least one worker among a plurality of workers pre-registered in the gig worker QA platform (S210). Here, the quality level is a digitization of the degree of completion of the work product, and can be used to provide a reward to the worker or determine whether the work product is completed.

Specifically, referring to FIG. 9 , the computing device 100 may obtain a first level related to at least one work result by using a pre-learned quality level determination model (S211). In addition, the computing device 100 may obtain a second level related to at least one work result by assigning an inspection task to another worker (S212). And, the computing device 100 may determine the quality level based on at least one of the first level and the second level (S213).

More specifically, when obtaining the first level, the computing device 100 may acquire score values for each of a plurality of classes by inputting a work result to a quality level determination model that is a multi-classification model. The computing device 100 may recognize a specific class having the highest score value. Also, the computing device 100 may determine a first level corresponding to a specific class.

For example, the quality level determination model may output a score of 0.2 for the first class, a score of 0.1 for the second class, and a score of 0.8 for the third class. In this case, the computing device 100 may determine a level corresponding to the third class as the first level.

When obtaining the second level, the computing device 100 may assign an inspection task for at least one work product to another worker assigned the same test case type as the at least one work product. Also, the computing device 100 may obtain a second level for at least one work product from another worker.

That is, the computing device 100 may improve the efficiency of the inspection task by allocating the inspection task to other workers who have performed the same test.

When the first level and the second level are obtained, the computing device 100 may determine the quality level based on at least one of the first level and the second level.

For example, the computing device 100 may determine the quality level as a higher level value of the first level and the second level when the difference between the level values of the first level and the second level is 1 or less. Here, the higher the level value, the better the quality. For example, when the first level is 7 and the second level is 8, the computing device 100 may determine the quality level as 8.

Meanwhile, the computing device 100 may reacquire each of the first level and the second level when the difference between the level values of the first level and the second level is greater than 1. For example, when the first level is 5 and the second level is 8, the computing device 100 may acquire each of the first level and the second level again.

The computing device 100 may determine the quality level using the first level and the second level obtained again in the same manner as above.

In various embodiments, when the difference between the level values of the first level and the second level obtained again is greater than 1, the computing device 100 assigns at least one test case type identical to at least one work product to another worker. You can assign an inspection task for one work product. Also, the computing device 100 may obtain a third level for at least one work result from another worker, and determine an average value of the first level, the second level, and the third level as the quality level.

That is, the computing device 100 of the present invention determines the quality level of the work result using the inspection result of another worker who performs the same test as the level obtained in the quality level determination model, so that the quality can be double checked. , more accurate quality inspection can be performed.

Referring back to FIG. 8 , the computing device 100 may provide a reward corresponding to the quality level to at least one worker when the quality level is greater than or equal to a preset level value (S220). Here, if the quality level is less than a preset level value, at least one work result may be rejected.

Specifically, the computing device 100 may determine that at least one work result is completed when the quality level is equal to or greater than a preset level value, and determine that at least one work result is incomplete when the quality level is less than the preset level value. .

When the computing device 100 determines that at least one work result is incomplete, it may request reprocessing of at least one work to at least one worker. Additionally, the computing device 100 may assign at least one task to another worker assigned with the same test case type as the at least one task product.

That is, when the first worker's processing is insufficient, the computing device 100 requests reprocessing from the first worker and simultaneously allocates the same workpiece to the second worker, thereby preventing quality deterioration of the work result. .

In various embodiments, the computing device 100 may provide a reward by assigning a weight according to a worker's grade.

Specifically, when the quality level is greater than or equal to a preset level value, the computing device 100 responds to the reward corresponding to the quality level to the grade of the at least one worker when providing a reward corresponding to the quality level to at least one worker. It can be provided by assigning a weight to

More specifically, a reward having a size obtained by multiplying a reward value corresponding to a quality level by a weight corresponding to the grade of the worker may be provided to at least one worker. Here, the higher the weight assigned to the reward according to the grade, the higher the weight is assigned, and the lower the weight is assigned, the lower the weight may be assigned. In addition, the weight corresponding to the worker's grade may be preset and displayed on the gig worker QA platform. And, the worker's grade may be determined according to the work processed through the gig worker QA platform.

The initial value may be applied to the worker's grade as an intermediate grade when performing work for the first time through the gig worker QA platform. In addition, the worker's grade may be raised when the number of times that the work is completed is equal to or greater than a preset number of times, or when the number of times that the quality level of the result is determined to be the highest level is equal to or greater than a preset number of times. In addition, the worker's grade may be lowered when the number of reprocessing by processing the work as incomplete is equal to or greater than a preset number of times, or when the number of times of rejecting an assigned task is equal to or greater than a preset number of times.

That is, the computing device 100 of the present invention can increase the morale of the worker and improve the quality of the result by granting a larger reward as the level of the worker increases. In this case, the productivity of QA work can be increased.

In various embodiments, when two or more work tasks are assigned to a worker, the computing device 100 may provide a reward for the work at the time when all of the two or more work tasks are completed.

Specifically, when the computing device 100 receives a work result for the first work from a specific worker assigned to the first work and the second work among a plurality of workers pre-registered in the gig worker QA platform, 1 Only half of the reward related to the work can be provided to the specific worker. And, when the computing device 100 receives a work result for a second work from a specific worker after providing only half of the reward for the first work to the specific worker, the remainder of the reward related to the first work A reward related to the half and the second work may be provided to a specific worker.

That is, the computing device 100 of the present invention provides a reward at the time when all work tasks are completed, thereby guaranteeing the continuity of tasks and preventing a situation in which tasks are not completed.

In various embodiments, the computing device 100 may display job processing rankings of workers on a dashboard and additionally provide rewards according to the rankings.

Specifically, the computing device 100 may output processing states of a plurality of divided work assigned to each of the plurality of workers, and processing speed rankings and quality level rankings of each of the plurality of workers on the dashboard. In addition, the computing device 100 may additionally provide a ranking reward for a worker included in a preset ranking order.

That is, the computing device 100 of the present invention can improve the morale of workers and increase the productivity of QA tasks by combining QA tasks with entertainment elements.

In various embodiments, the computing device 100 may provide compensation according to the occurrence of the issue to the worker when it is determined that an issue with the software occurs as the worker performs the QA task. At this time, the computing device 100, when the QA task performed by the operator is a QA task of performing a test according to the test case prepared by the operator, that is, as the test target is tested according to the test case directly created by the operator, the software If an issue is generated, additional compensation can be provided to the worker in addition to the compensation for the occurrence of the issue.

10 and 11 are diagrams for explaining an example of a method of assigning QA tasks and collecting work results for gig worker QA platform workers.

According to one embodiment of the present invention, the processor 110 of the computing device 100 may allocate jobs of gig worker QA platform workers and collect work results of the workers.

Referring to FIG. 10 , the computing device 100 may classify a plurality of workers pre-registered in the gig worker QA platform for each device environment (S310). Here, the gig worker QA platform may obtain information about the environment of a device possessed by each of a plurality of workers when each of a plurality of workers registers on the platform.

Information on the environment of the device is the type of device, the manufacturer of the device, the type of OS installed on the device, the version of the OS installed on the device, the device's CPU, the device's free storage capacity, the device's RAM capacity, and It may include at least one of networks supported by the device, but is not limited thereto.

The computing device 100 may allocate a plurality of divided work pieces obtained by dividing the quality assurance target work to a plurality of workers for each device environment (S320).

Specifically, the computing device 100 may allocate the divided work to a plurality of workers for each device environment so that the test is performed in each device environment owned by the workers.

Meanwhile, the computing device 100 may classify the quality assurance target work into specialized work and simple work before allocating the divided work to a plurality of workers for each device environment.

For example, the computing device 100 may generate a test case based on a quality assurance target work. In addition, the computing device 100 classifies the test case as a specialized work when the execution time of the test case exceeds the preset time or exceeds the preset number of steps, and the test case execution time is less than the preset time or the preset number of steps. If it is less than a number, it can be classified as a simple work.

Hereinafter, reference is made to the descriptions of FIGS. 4 and 5 described above for a detailed description of a method of classifying work to be guaranteed as a specialized work and a simple work.

When the quality assurance target work is classified into a specialized work and a simple work, the computing device 100 may acquire a plurality of divided work by dividing the simple work.

For example, the computing device 100 may input the simple workpiece into a pre-learned task division model to obtain the plurality of divided workpieces divided according to test case types, but is not limited thereto.

When obtaining a plurality of divided work pieces, the computing device 100 may allocate the plurality of divided work pieces to each worker having a device in a different environment.

Specifically, the computing device 100 may recognize the number of classified device environments in the step of classifying a plurality of workers pre-registered in the gig worker QA platform for each device environment. Also, the computing device 100 may copy the same work as many times as the number of device environments so that each of the plurality of divided work is performed in different device environments. Also, the computing device 100 may allocate the same work to each worker having a device in a different environment.

For example, assuming that a first worker holding a device of the first environment and a second worker holding a device of the second environment exist, the same division work that performs a test case of logging in to the software subject to quality assurance is performed by the first worker. It can be assigned to each of the worker and the second worker. In this case, a QA task may be performed on whether or not login is normally performed in the software subject to quality assurance in each of the device of the first environment and the device of the second environment.

That is, the computing device 100 of the present invention can expand the application range of QA by allocating tasks to perform QA in a device environment of various environments. In addition, the quality of the quality assurance object can be guaranteed through QA in various environments.

When obtaining a plurality of work results for a plurality of divided workpieces from a plurality of workers for each device environment, the computing device 100 may combine work results processed in different device environments into one result (S330).

Specifically, referring to FIG. 11 , when a specific test case is performed in each of the first device environment and the second device environment for a specific division work, the computing device 100 determines the work result performed in the first device environment. A first tag may be added, and a second tag may be added to a work result performed in the second device environment (S331). Then, the computing device 100 may collect work products to which the first tag and the second tag are respectively added and generate a work product of a specific division work (332).

Referring back to FIG. 10 , in step S330, for example, the computing device 100 may collect work results performed with the same test case in different device environments and collect them as a unit of the test case. .

As another example, the computing device 100 may collect work results processed in different device environments in units of pre-classified test types. Here, the pre-classified test types may include at least one of a functional test type, a local test type, a compatibility test type, an inspection and featured test type, a loading time test type, and an exothermic test type.

In various embodiments, the computing device 100 may allocate a plurality of divided work pieces to a plurality of workers for each device environment, and then recognize at least one rejected work from among the plurality of divided work pieces. Also, the computing device 100 may obtain a reason for refusal from the refusal worker who rejected the allocation of the division work. In addition, the computing device 100 may allocate at least one rejected work to another worker based on the reason for rejection.

For example, when the reason for rejection is simple rejection, the computing device 100 may allocate the worker to another worker without an additional reward. In addition, the computing device 100 may impose a penalty that is limited to a predetermined number of work assignments to the rejected worker.

As another example, if the reason for rejection is not simple rejection, the computing device 100 may determine the difficulty level of at least one rejected work. In addition, the computing device 100 may grant an additional reward corresponding to the difficulty level of the at least one rejected job. Then, the computing device 100 may notify the gig worker QA platform of at least one rejected work to which an additional reward has been granted, and assign it to another worker who wishes to process the at least one rejected work.

In an additional embodiment, the computing device 100 may extract one or more words included in a sentence of a test case related to at least one rejected work when the reason for rejection is not simple rejection. In addition, the computing device 100 may determine a word that appears more than a predetermined number of times among words included in sentences included in test cases of rejected work during a predetermined period as a keyword for classifying as a specialized work.

For example, when the computing device 100 classifies the quality assurance target work into a specialized work and a simple work, the work related to the test case including a specific keyword among sentences for describing the test case is specialized. It can be classified as work. Here, the specific keyword may be a word included a predetermined number of times or more among words included in a sentence included in a test case of a rejected work that was assigned to a worker for a predetermined period of time and then rejected by the worker. For example, a specific keyword may be a word included more than 30 times in test cases of rejected work in the last month.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (14)

In the QA task assignment and monitoring method for gig worker QA platform workers performed by a computing device including at least one processor,
Dividing the quality assurance target work and allocating it to a plurality of workers pre-registered in the gig worker QA platform;
monitoring the processing status of a plurality of divided works assigned to each of the plurality of workers, and outputting a monitoring report to a dashboard; and
Updating the report when obtaining a plurality of work results for the plurality of divided workpieces from each of the plurality of workers;
including,
The step of dividing the quality assurance target work and allocating it to a plurality of workers pre-registered in the gig worker QA platform,
classifying the work to be guaranteed as a specialized work and a simple work;
Dividing the simple workpiece to obtain the plurality of divided workpieces; and
allocating the plurality of divided work pieces to each of the plurality of workers based on at least one of a work load, a preferred task, and a skilled task of each of the plurality of workers;
including,
After allocating the plurality of divided work to each of the plurality of workers, recognizing at least one rejected work from among the plurality of divided work;
Obtaining a reason for refusal from a refusal worker who rejected the assignment of the division work; and
allocating the at least one rejected job to another worker based on the rejection reason;
Including more,
Allocating the at least one rejected work to another worker based on the reason for rejection includes:
allocating the at least one rejected work to another worker without an additional reward when the reason for the rejection is simple rejection; and
assigning a penalty limited to a predetermined number of work assignments to the rejected worker;
including,
Allocating the at least one rejected work to another worker based on the reason for rejection includes:
determining a difficulty level of the at least one rejected work when the reason for rejection is not a simple rejection;
granting an additional reward corresponding to the difficulty level of the at least one rejected work; and
notifying the gig worker QA platform of the at least one rejected work to which the additional reward has been granted, and allocating the at least one rejected work to another worker desiring to process the at least one rejected work;
including,
When obtaining a plurality of work results for the plurality of divided works from each of the plurality of workers, the step of updating the report,
determining a quality level of the first work product; and
providing a reward corresponding to the quality level to a first worker who has processed the first work product when the quality level is greater than or equal to a preset level value; Including more,
Determining the quality level of the first work result,
obtaining a first level related to the first work product by using a pre-learned quality level determination model;
Acquiring a second level related to the first work result by assigning an inspection task to a second worker; and
determining the quality level based on at least one of the first level and the second level;
including,
When the quality level is less than the preset level value, the first work result is rejected;
Determining the quality level based on at least one of the first level and the second level,
When the difference between the level values of the first level and the second level is 1 or less, the quality level is determined as a higher level value of the first level and the second level, and the level of the first level and the second level acquiring the first level and the second level again when the difference between the values is greater than 1;
including,
When the difference between the level values of the first level and the second level obtained again is greater than 1, a third worker assigned the same test case type as the first work product is inspected for the first work product. allocating; and
obtaining a third level for the first work result from the third worker, and determining an average value of the first level, the second level, and the third level as the quality level;
Including more,
The step of dividing the quality assurance target work and allocating it to a plurality of workers pre-registered in the gig worker QA platform,
A device environment for a plurality of workers pre-registered in the gig worker QA platform - the device environment includes at least one of the type of device used by the worker, the manufacturer of the device, the OS installed on the device, the hardware performance of the device, and the network network supported by the device including one - classify by star; and
Allocating a plurality of divided work pieces obtained by dividing the quality assurance target work to a plurality of workers for each device environment; including,
The step of allocating the plurality of divided work to the plurality of workers for each device environment,
Recognizing the number of device environments classified in the step of classifying a plurality of workers pre-registered in the gig worker QA platform for each device environment;
copying the same work as many times as the number of device environments so that each of the plurality of divided work is performed in different device environments; and
allocating the same workpiece to each worker having the device in the different environment;
including,
How to assign and monitor QA tasks for gig worker QA platform workers.
According to claim 1,
The step of classifying the quality assurance target work into a professional work and a simple work,
Generating a test case based on the quality assurance target work; and
When the execution time of the test case exceeds the preset time or the preset number of steps, it is classified as the specialized work, and the test case execution time is less than the preset time or the preset number of steps. classifying the simple work;
including,
How to assign and monitor QA tasks for gig worker QA platform workers.
According to claim 1,
Dividing the simple workpiece to obtain the plurality of divided workpieces,
inputting the simple workpiece into a pre-learned task division model and obtaining the plurality of divided workpieces divided according to test case types;
including,
The work division model,
Pre-learning with training data including a learning workpiece and a type of test case labeled in the learning workpiece,
How to assign and monitor QA tasks for gig worker QA platform workers.
According to claim 1,
The step of allocating the plurality of divided work to each of the plurality of workers based on at least one of the workload, preferred work, and skilled work of each of the plurality of workers,
Recognizing the current workload of each of the plurality of workers; and
Allocating the plurality of divided workpieces to each of the plurality of workers based on the current workload of each of the plurality of workers and the work processing history of each of the plurality of workers;
including,
How to assign and monitor QA tasks for gig worker QA platform workers.
According to claim 4,
Allocating the plurality of divided work to each of the plurality of workers based on the current workload of each of the plurality of workers and the work processing history of each of the plurality of workers,
Based on the current workload of each of the plurality of workers, recognizing a fourth worker capable of assigning tasks among the plurality of workers;
recognizing a first type of test case having the highest verification result score in the work process history of the fourth worker; and
allocating the first type of division work from among the plurality of division works to the fourth worker;
including,
How to assign and monitor QA tasks for gig worker QA platform workers.
According to claim 1,
The step of allocating the plurality of divided work to each of the plurality of workers based on at least one of the workload, preferred work, and skilled work of each of the plurality of workers,
Based on the current workload of each of the plurality of workers, recognizing a fifth worker capable of assigning tasks among the plurality of workers;
recognizing a second type of test case corresponding to at least one of a preferred task and a skilled task of the fifth worker; and
allocating the second type of divided work from among the plurality of divided work to the fifth operator;
including,
The gig worker QA platform,
Obtaining information about the preferred task and the skilled task when each of the plurality of workers registers on the platform,
How to assign and monitor QA tasks for gig worker QA platform workers.
delete delete delete According to claim 1,
The step of monitoring the processing status of the work assigned to each of the plurality of workers and outputting a monitoring report to the dashboard,
Transmitting a processing state confirmation request message to each of the plurality of worker terminals for each predetermined time period, obtaining information on the current processing state;
Receiving an individual job completion message from each of the plurality of worker terminals as the individual jobs of the plurality of divided work pieces are completed in each of the plurality of worker terminals; and
obtaining information on the current processing status or updating the report and displaying the updated report on the dashboard when the individual task completion message is received;
including,
How to assign and monitor QA tasks for gig worker QA platform workers.
According to claim 1,
The dashboard is
Including a test result report output function provided to a requester who requested the work subject to quality assurance,
The monitoring report,
Including a summary area including progress, unusual matters, confirmation requests, and QA opinions, and a detailed area including details of the progress,
How to assign and monitor QA tasks for gig worker QA platform workers.
According to claim 1,
determining whether each of the plurality of work products corresponds to an issue when obtaining a plurality of work products for the plurality of divided works from each of the plurality of workers; and
uploading the issue to external software for tracking and correcting issues when it is determined that any one of the plurality of work products corresponds to the issue;
Including more,
How to assign and monitor QA tasks for gig worker QA platform workers.
a memory that stores one or more instructions; and
a processor that executes the one or more instructions stored in the memory;
including,
By executing the one or more instructions, the processor:
An apparatus that performs the method of claim 1 .
A computer program stored in a computer-readable recording medium to be combined with a computer, which is hardware, to perform the method of claim 1.

Images