KR102660320B1 - A sever for providing optimal source code based cloud service and a method for providing source code - Google Patents

Abstract

The source code providing server according to an embodiment of the present invention is a source code providing server that recommends source code that executes a process to achieve a goal, and receives request information reflecting the target purpose that the user wants to achieve. module; A storage module in which code information, which is information about source code, is stored; And using a combination model learned through machine learning, a recommendation code that is run on the user's computing device based on the code information and the request information and is a combination of the source code and the type of source code to achieve the target purpose is determined. It may include a recommendation module.

Description

Cloud-based optimal source code provision server and source code provision method {A SEVER FOR PROVIDING OPTIMAL SOURCE CODE BASED CLOUD SERVICE AND A METHOD FOR PROVIDING SOURCE CODE}

The present invention relates to a source code provision server and source code provision method that recommends and provides the optimal source code combination to users based on cloud services.

As Internet network technology increases and space restrictions for computer work, such as working from home, disappear, the cloud environment has recently grown rapidly. In the past, people only received storage space through cloud services, but as cloud technology improves, the number of people running programs through cloud services is increasing. Moreover, recently, platforms have been developed and released that allow program development through the cloud to maximize geographical and work efficiency for developers. Through these platforms, developers can receive various development tools and development sources anytime, anywhere.

However, existing cloud-based development platforms simply provide execution results of development software or simple information sharing of source codes. As a result, when data provided in the cloud contains errors, developers are exposed to various problems by using the erroneous data as is. Additionally, since developers spend a lot of time actually combining and operating source code, there is a problem that does not increase the actual work efficiency of developers.

The present invention is intended to solve the above-described problems, and seeks to provide a source code provision server and a source code provision method that can maximize the safety of computing resources and work efficiency.

The source code providing server according to an embodiment of the present invention is a source code providing server that recommends source code that executes a process to achieve a goal, and receives request information reflecting the target purpose that the user wants to achieve. module; A storage module in which code information, which is information about source code, is stored; And using a combination model learned through machine learning, a recommendation code that is run on the user's computing device based on the code information and the request information and is a combination of the source code and the type of source code to achieve the target purpose is determined. It may include a recommendation module.

In addition, an inspection module that checks whether an error occurs when executing the process according to the recommendation code; And when the inspection module determines that an error occurs when operating as the recommendation code, an explanation module that identifies the cause of the error using explainable artificial intelligence.

In addition, it may further include an interface module that calculates an interface that displays information about the recommendation code or error to the user.

In addition, the request information consists of purpose information, which is information about the target purpose, and usage environment information, which is information about the operation of the user's computing device, and the recommendation module determines the recommendation code in consideration of the usage environment information. You can.

Additionally, the recommendation module may determine the recommendation code in which different source codes are executed in time series, taking into account the load generated in the user's computing device.

In addition, in order to learn the combination model, it further includes a labeling module that performs labeling on the source code, wherein the labeling module includes an automatic labeling unit that automatically labels the source code using the labeling model. can do.

In addition, the labeling module further includes an inspection unit that requests pre-registered inspection personnel to check the validity of labeling performed by the automatic labeling unit, and the automatic labeling unit bases the evaluation results of the pre-registered inspection personnel on the basis of the evaluation results of the pre-registered inspection personnel. You can retrain the labeling model.

In addition, the labeling module may further include a schedule unit that manages manual labeling work by a plurality of workers when a predetermined start condition is satisfied based on the evaluation results of the pre-registered inspection personnel.

In addition, the labeling module further includes a human resources department that evaluates the work competency of the workforce and a difficulty calculation unit that evaluates the difficulty of labeling, and the schedule unit is configured to perform the task based on the difficulty of labeling and the work competency of the workforce. Labeling tasks can be assigned to personnel.

The method of providing source code according to an embodiment of the present invention is implemented by a source code providing server, and is a method of recommending a process to achieve the purpose. A step of receiving request information reflecting the target purpose; And by the recommendation module, a combination model learned through machine learning is used and driven on the user's computing device based on the code information stored in the storage module and the request information to achieve the type of source code to achieve the target purpose. It may include a step of determining a recommendation code that is a combination of and source code.

The source code provision server and source code provision method according to the present invention can increase the user's work convenience.

Additionally, errors can be minimized.

Additionally, the reliability of cloud services can be maximized.

Additionally, the usability of cloud services can be increased.

However, the effects of the present invention are not limited to the effects described above, and effects not mentioned can be clearly understood by those skilled in the art from this specification and the attached drawings.

1 is a relationship diagram of a source code providing server according to an embodiment of the present invention.
Figure 2 is a configuration diagram of a source code providing server according to an embodiment of the present invention.
Figure 3 is a flowchart of a method for providing source code according to an embodiment of the present invention.
4 and 5 are diagrams showing examples of user interfaces provided by a source code providing server according to an embodiment of the present invention.
Figure 6 is a diagram for explaining the labeling process by the labeling module provided by the source code providing server according to an embodiment of the present invention.
Figure 7 is a diagram illustrating a manual labeling process executed by a labeling module provided by a source code providing server according to an embodiment of the present invention.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the presented embodiments, and a person skilled in the art who understands the spirit of the present invention may add, change, or delete other components within the scope of the same spirit, thereby creating other degenerative inventions or the present invention. Other embodiments that are included within the scope of the invention can be easily proposed, but this will also be said to be included within the scope of the invention of the present application.

In addition, components having the same function within the scope of the same idea shown in the drawings of each embodiment will be described using the same reference numerals.

Figure 1 is a relationship diagram of a source code providing server according to an embodiment of the present invention.

Referring to FIG. 1, the source code providing server 100 according to an embodiment of the present invention provides the user's computing device 10, the external server 20, the inspection personnel's computing device 30, and/or the inspection personnel's computing device 100. It can be networked with computing devices wired or wirelessly.

The online network referred to in the present invention may be a core network integrated with a wired public network, a wireless mobile communication network, or a mobile Internet, and may include TCP/IP protocols and various services existing at the upper layer thereof, such as HTTP (Hyper Text Transfer Protocol). , can refer to a worldwide open computer network structure that provides HTTPS (Hyper Text Transfer Protocol Secure), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), and SMTP (Simple Mail Transfer Protocol). It is not limited to examples, but comprehensively refers to a data communication network that can transmit and receive data in various forms.

The cloud service provided by the system referred to in the present invention is a service that integrates the resources of devices existing in different physical locations through virtualization technology and provides an environment in which IT resources such as hardware and software can be borrowed and used as needed when needed. As a service, it may be comprised of an Infrastructure as a Services (IaaS) service, a Platform as a Services (PaaS) service, and/or a Software as a Services (SaaS) service.

The user 10 may refer to a person who wants to execute a process to achieve a certain purpose.

The computing device referred to in the present invention may mean a device capable of processing information processing operations.

For example, computing devices may include desktop computers, laptops, smartphones, Personal Digital Assistants (PDAs), Portable Multimedia Players (PMPs), mobile terminals including portable terminals, and/or smart TVs.

Hereinafter, user and user computing device may be used with the same meaning.

The external server 20 may refer to a server accessible through a public network.

For example, the external server 20 may refer to a server that can store and provide source code.

The server referred to in the present invention may include other components for performing the server environment. A server can include any type of device.

For example, a server may be a digital device, such as a laptop computer, notebook computer, desktop computer, web pad, or mobile phone, equipped with a processor and equipped with memory and computing power.

In one example, the server may be a web server. However, it is not limited to this, and the type of server can be changed in various ways at a level that is obvious to a person skilled in the art.

The user can request the source code needed for his or her work (process) from the source code providing server 100, and the source code providing server 100 can recommend or provide the optimal source code combination suitable for the user's process. .

Below, the source code provision server will be described in detail.

Figure 2 is a configuration diagram of a source code providing server according to an embodiment of the present invention.

The source code providing server 100 according to an embodiment of the present invention recommends source code for executing a process to achieve a goal, and reflects the target purpose, which is the goal the user wants to achieve. Using a receiving module 110 that receives request information, a storage module 120 that stores code information, which is information about source code, and a combination model learned through machine learning, based on the code information and the request information It may include a recommendation module 130 that runs on the user's computing device and determines a recommendation code that is a combination of the source code and the type of source code to achieve the target purpose.

In addition, the source code providing server 100 has an inspection module 140 that checks whether an error occurs when executing a process according to the recommendation code, and an error error occurs when the inspection module 140 is operated as the recommendation code. If it is determined that occurs, an explanation module 150 that identifies the cause of the error using explainable artificial intelligence may be further included.

In addition, the source code providing server 100 performs labeling on the source code in order to learn the combination model and the interface module 160 that calculates an interface that displays information about the recommendation code or error to the user. It may further include a labeling module 170.

The reception module 110, recommendation module 130, explanation module 150, collection module, storage module 120, inspection module 140, interface module 160, and/or labeling module 170 are wired/wireless. They can be connected to each other and communicate information with each other.

The receiving module 110 can receive request information reflecting the target purpose that the user wants to achieve.

For example, the target purpose may mean what the user wants to achieve through the program.

For example, the target purpose may be the purpose of performing a classification task of error data, a purpose of performing an image editing task, a classification process, or an automatic infrastructure construction task, etc.

However, it is not limited to this, and the above object can be modified in various ways at a level that is obvious to those skilled in the art.

Request information may consist of purpose information, which is information about the target purpose, specification information, which is information about the specifications of the user's computing device, and usage environment information, which is information about the operation of the user's computing device.

This may be because the recommendation code is executed on the user's computing device.

For example, specification information may include information about hardware such as CPU, RAM, storage, network, GPU, etc. of the user's computing device, and information about operating/application software that is installed and executed.

Additionally, the specification information may further include information about the load generated while the user's computing device operates, and metric/log information (usage environment information).

However, it is not limited to this, and the specific types of specification information can be modified in various ways at a level that is obvious to a person skilled in the art.

The receiving module 110 may collect information about the usage environment from the user computing device among the specification environments.

The storage module 120 may store all information necessary while implementing the source code provision method.

Code information may be stored in the storage module 120.

For example, code information includes source code, the function of the process executed by the source code, information about the stability of the source code, information about the efficiency of the source code, and information about the computing device suitable for the process by the source code to run without error. May contain information about the environment.

Code information may be stored in a database in the storage module 120.

Since the database technology is a known technology, detailed description thereof may be omitted to the extent that it overlaps with the known technology.

The storage module 120 can access and download external servers to collect code information.

The recommendation module 130 uses a combination model learned through machine learning and runs on the user's computing device based on the code information and the request information to combine the type of source code and source code to achieve the target purpose. You can decide on the referral code.

The recommendation module 130 can learn a combination model using machine learning.

As a specific example, the recommendation module 130 performs machine learning/deep learning based on past learning data, which is information about past user request information and past recommendation codes provided according to the request information, and generated learning data described later. You can learn a combination model.

Here, the implementation algorithm for machine learning/deep learning utilizes known technologies, and detailed description thereof may be omitted.

The combination model can determine and recommend a recommendation code that runs without error on the user's computing device by considering the specifications and usage environment information of the user's computing device.

The recommendation module 130 may determine the recommendation code by considering the usage environment information.

The recommendation code is a preliminary recommendation code that means a combination of source code that can operate without error on the user's computing device while achieving the target purpose, a final recommendation code selected from among the preliminary recommendation codes in consideration of the usage environment of the user's computing device, and a designated time Accordingly, the final recommendation code may be changed and consist of a combination recommendation code that is a combination of at least one final recommendation code configured to be executed.

For example, the recommendation module 130 may use a combination model to calculate a preliminary recommendation code that is a combination of the source code and the type of source code that can achieve the target purpose by considering the specifications of the user's computing device.

Additionally, the recommendation module 130 may select the final recommendation code from among the preliminary recommendation codes in consideration of the usage environment of the user's computing device.

Additionally, the recommendation module 130 may calculate a combination recommendation code that is executed by changing the final recommendation code in time series according to the usage environment of the user's computing device.

The recommendation module 130 can classify recommendation codes into recommendation codes that emphasize stability and recommendation codes that emphasize efficiency.

To this end, the recommendation module 130 can machine learn a classification model, which is a model that classifies the recommendation code into a stability-oriented code or an efficiency-oriented code based on the stability/efficiency classification information of past recommendation codes.

Here, the implementation algorithm for machine learning/deep learning utilizes known technologies, and detailed description thereof may be omitted.

For example, when the load of the user's computing device is greater than a predetermined standard, a preliminary recommendation code that emphasizes efficiency may be selected as the final recommendation code.

For example, if the load of the user's computing device is less than a predetermined standard, a preliminary recommendation code that emphasizes stability may be selected as the final recommendation code.

Here, the load of the user's computing device can be calculated by the number of programs being executed, CPU operation rate, RAM operation rate, etc.

Different from the above, the classification model can directly execute the recommendation code and classify whether the recommendation code emphasizes stability or efficiency by considering the degree and speed of the generated load.

The recommendation module 130 can recommend a preliminary recommendation code to the user.

Differently, the recommendation module 130 may select and recommend a final recommendation code from among the preliminary recommendation codes in consideration of the usage environment of the user's computing device based on the time the user requests provision of the source code.

Alternatively, the recommendation module 130 may recommend the combination recommendation code in which different source codes are executed in time series, taking into account the load generated in the user's computing device.

For example, the load of the user's computing device may vary over time, and whether the load of the user's computing device is above or below a predetermined standard may also change over time.

The recommendation module 130 can recommend a combination recommendation code implemented to automatically run a final recommendation code that emphasizes stability or a recommendation code that emphasizes efficiency over time.

For example, during times when the load of the user's computing device is more than a predetermined standard, the recommendation module 130 executes the final recommendation code that emphasizes efficiency, and during times when the load of the user's computing device is less than a predetermined standard, the recommendation module 130 executes the final recommendation code. A combination recommendation code can be configured so that the final recommendation code that emphasizes stability is executed.

In one example, the predetermined standard may be 50% of the available load of the user's computing device.

However, it is not limited to this, and the specific values of the predetermined standard can be modified in various ways at a level that is obvious to those skilled in the art.

The recommendation code may include information about the type of source code and the order between source codes.

Here, source code refers to text that describes a computer program in a human-readable programming language, and can be identified by being divided into arbitrary functional units.

The inspection module 140 can check whether an error occurs when executed with a recommendation code (preliminary recommendation code, final recommendation code, or combination recommendation code) recommended by the recommendation module 130.

The inspection module 140 can execute the recommendation code and determine whether an error occurs based on the generated metrics or log data.

To this end, the inspection module 140 generates an error judgment model by deep learning/machine learning based on past recommendation codes, metrics/log data generated from the past recommendation codes, and error information generated from the past recommendation codes. You can.

Here, the implementation algorithm for machine learning/deep learning utilizes known technologies, and detailed description thereof may be omitted.

That is, the inspection module 140 can determine errors in the recommendation code to be recommended to the user using the error determination model.

If the explanation module 150 determines that an error occurs when the inspection module 140 operates as the recommendation code, the explanation module 150 can identify the cause of the error using explainable artificial intelligence (XAI).

To this end, the explanation module 150 can perform machine learning/deep learning on an explanation model that analyzes the cause of the error using the past recommendation code, the cause of the error occurring in the past recommendation code, and an error judgment model.

Here, the implementation algorithm for machine learning/deep learning utilizes known technologies, and detailed description thereof may be omitted.

The interface module 160 may calculate an interface that displays information about the recommendation code or errors in the recommendation code to the user and provide the interface to the user's computing device.

The interface provided by the interface module 160 may display a recommendation code identification image, a link to access the recommendation code, and error information expected to occur in the recommendation code and/or the recommendation code.

However, it is not limited to this, and the types of information displayed by the interface can be modified in various ways at a level that is obvious to those skilled in the art.

The labeling module can perform labeling on source code in order to learn a combination model, and a detailed explanation of this will be provided later.

Hereinafter, the method of providing source code provided by the source code providing server will be described in detail.

Figure 3 is a flowchart of a method for providing source code according to an embodiment of the present invention.

Referring to Figure 3, the source code provision method according to an embodiment of the present invention is implemented by a source code provision server, and is a method of recommending a process to achieve the purpose. In the source code provision method, a receiving module By the step of receiving request information that reflects the target purpose, which is the goal that the user wants to achieve, and by the recommendation module, a combination model learned through machine learning is used, based on the code information stored in the storage module and the request information. It may include a step of determining a type of source code and a recommendation code that is a combination of the source code to achieve the target purpose by running on the user's computing device.

To be more specific, the user can request a recommendation code from the source code providing server in order to proceed with the process to implement the target purpose.

In the step where the request information is received, the interface module may provide the user computing device with an interface through which the request information can be written.

In the step where the recommendation code is determined, the recommendation code may be determined by a combination model based on the code information and request information stored in the storage module.

Errors are checked for the determined recommendation code by the inspection module, and if the error is predicted or confirmed, the cause of the error can be calculated by the explanation module.

Through the interface module, an interface in which information about a recommendation code or error is displayed may be provided to the user.

Figures 4 and 5 are diagrams showing examples of user interfaces provided by a source code providing server according to an embodiment of the present invention.

Figures 4 and 5 show time series images displayed on the user interface when the recommendation module recommends a combination recommendation code.

For example, the time section divided by the characteristics of the recommendation code presented by the recommendation module may be A11 to A15, and from 0:00 to 8:30 section (A11), the load of the user's computing device operates below a predetermined standard. In this case, from 8:30 to 12:30 section (A12), the load of the user computing device operates above a certain standard, and from 12:30 to 13:30 section (A13), the load of the user computing device operates above a certain standard. This is the case where the load of the user computing device is operated above a certain standard from 13:30 to 19:00 section (A14), and from 19:00 to 24:00 section (A14), the load of the user computing device is operated above a certain standard. It can be assumed that the load is operated below a certain standard.

Therefore, the recommendation module can recommend a combination recommendation code in which the final recommendation code that emphasizes stability is operated in sections A11, A13, and A15, and the final recommendation code that emphasizes efficiency is operated in sections A12 and A14.

In the user interface, time-series images may be displayed so that images representing the final recommendation code with the same characteristics (stability/efficiency) have the same color/pattern.

In time-series images, final recommendation codes of different characteristics can be distinguished using a predetermined dividing line (L11).

Referring to Figure 5, the user moves the dividing line (L11) in the time-series image (A10) to change the proportion of the final recommendation code that emphasizes stability and the final recommendation code that emphasizes efficiency for each time interval presented by the recommendation module. You can change it.

To this end, the user can move the dividing line of the time-series image displayed on his or her computing device.

Additionally, when a predetermined condition is satisfied, the interface module may provide a warning indication to the user's computing device to terminate the operation of at least part of the program other than the recommendation code.

Here, the predetermined condition may be a condition in which the rate of change from the recommendation code that emphasizes efficiency to the recommendation code that emphasizes stability may be more than a predetermined rate during the time interval in which the recommendation code that emphasizes efficiency is recommended.

For example, referring to FIG. 5, the user can move the dividing line (L11) dividing the section A12 displayed in the time-series image (A10) to reduce the area of A12.

Due to this, if the rate of change from the recommendation code that emphasizes efficiency to the recommendation code that emphasizes stability is more than a certain percentage during the period of time (A12) during which the recommendation code that emphasizes efficiency is recommended, the interface module is a user computing device and sends a predetermined warning. can be displayed.

This may be to solve problems caused by overload on the computing device due to the user's incorrect judgment.

In one example, the predetermined ratio may be 50%.

However, it is not limited to this, and the specific value of the predetermined ratio can be modified in various ways at a level that is obvious to those skilled in the art.

In addition, the method of providing source code may further include a step of labeling raw data collected from an external server or storage module by a labeling module to generate learning data to be learned in a combination model.

Below, the labeling process will be described in detail.

Figure 6 is a diagram for explaining the labeling process by the labeling module 170 provided by the source code providing server according to an embodiment of the present invention.

Referring to FIG. 6, the labeling module 170 according to an embodiment of the present invention includes a pre-processing unit 110 that pre-processes the collected raw data, and a pre-processing unit 110 that performs virtual labeling on the raw data to calculate virtual label data. It may be provided with an automatic labeling unit 120 and an inspection unit 130 that requests a pre-registered inspection personnel (P10) to check the validity of the virtual label data.

In addition, the labeling unit is a schedule unit 140 that performs and manages the manual labeling task through a plurality of workers when a predetermined start condition is satisfied based on the evaluation results of the pre-registered inspection personnel (P10). , a human resources department 170 that evaluates the work capabilities of the workforce and a difficulty calculation department 150 that evaluates the difficulty of labeling may be further provided.

In addition, the labeling unit includes the preprocessing unit 110, the automatic labeling unit 120, the inspection unit 130, the schedule unit 140, the human resources unit 170, and the difficulty calculation unit 150. Additionally, a label management unit 160 may be provided to transmit the labeled generated learning data to the recommendation module.

The pre-processing unit 110 may perform a data pre-processing process to remove or convert parts unnecessary for labeling from raw data.

Raw data consists of source codes, and the preprocessor 110 can remove unnecessary parts for labeling or divide raw data into purpose units and store them for labeling.

The automatic labeling unit 120 can automatically label source code using a labeling model.

The automatic labeling unit 120 can calculate the virtual label data using a labeling model learned through machine learning.

When preprocessed raw data is input into the labeling model, a virtual label can be attached to the raw data.

A bundle of preprocessed raw data and virtual labels corresponding to the preprocessed raw data can be defined as virtual label data.

Here, the label may consist of information about the purpose of the source code and the specifications of the computing device to be used.

For example, the labeling model can implement virtual labeling through Pseudo-Labeling, one of the unsupervised learning techniques, but the present invention is not limited to this.

The automatic labeling unit 120 may re-learn the labeling model based on the evaluation results of the pre-registered inspection personnel.

The inspection unit 130 may request a pre-registered inspection personnel to check the validity of labeling performed by the automatic labeling unit 120.

The inspection unit 130 may request a pre-registered inspection personnel (P10) to check the validity of the virtual label data.

For example, pre-registered inspection personnel may be employees of a company that operates a source code provision server.

However, without being limited to this, the identity of the pre-registered inspection personnel can be modified in various ways at a level that is obvious to an ordinary technician.

The inspection unit 130 may request a validity check by calculating an interface for requesting a validity check and transmitting it to the computing device of a pre-registered inspection personnel.

The inspection unit 130 receives one of a first response indicating that the labeling is normal, a second response indicating that the labeling process is suspicious, and a third response indicating that the labeling needs to be checked. You can request a validity check from inspection personnel (P10).

The inspection unit 130 may transmit the evaluation results received from pre-registered inspection personnel to the automatic labeling unit 120.

The automatic labeling unit 120 can increase the accuracy of the basic label model by re-learning it by changing the weight value of the basic label model based on the evaluation results.

The human resources department 170 can evaluate and classify the work capabilities of the inspection work force (P20, hereinafter referred to as work force).

The human resources department 170 may store lists of work personnel (P20).

The human resources department 170 can manage the workforce by dividing it into grades based on the evaluation of the work capabilities of the workforce (P20).

For example, the Ministry of Human Resources 170 may divide the workforce into first, second, and third grades, with the first grade being the grade with the best work capability, and the third grade being the grade with the worst work competency. It can be.

However, it is not limited to this, and the grade of the manpower unit 170 can be modified in various ways at a level that is obvious to a person skilled in the art.

The human resources department 170 can evaluate the work force in real time based on whether or not it was evaluated with manual label results that replace virtual label data.

If the work force is evaluated using manual labeling results that replace virtual label data, additional points may be given for work ability.

Conversely, work personnel who are not evaluated using manual labeling results that replace virtual label data may be given reduced points for their work ability.

The difficulty calculation unit 150 may store a difficulty evaluation model that calculates the difficulty level for virtual labeling.

The difficulty calculation unit 150 may calculate a difficulty evaluation model by performing machine learning/deep learning based on information about labeling the preprocessed raw data and information about the corresponding difficulty level.

Since machine learning utilizes known technologies, detailed descriptions thereof may be omitted.

For example, the difficulty calculation unit 150 can classify labeling into a total of four levels of difficulty.

For example, the difficulty calculation unit 150 can classify the difficulty of labeling into special, high, intermediate, and low difficulty levels.

Here, according to the difficulty evaluation model, the labeling difficulty is divided into high, intermediate, and low levels of difficulty, and special level can be specially designated when the conditions described later are satisfied.

The schedule unit 140 may proceed with the manual labeling process when predetermined start conditions are satisfied.

The schedule unit 140 may assign labeling tasks to the workers based on the difficulty of labeling and the work capabilities of the workers.

The predetermined start condition may be a condition in which the evaluation result is an evaluation that the labeling process is suspicious or an evaluation that the labeling process needs to be checked.

To explain specifically, if the third response is the result of the effectiveness review, the predetermined start condition may be satisfied.

In addition, if the effectiveness review result is a second response, and the effectiveness check result is a second response or a third response from an inspection personnel other than the pre-registered inspection personnel who have already checked the validity, the predetermined start condition may be satisfied.

If the result of the initial validity review is a second response, the inspection department 130 may re-request the validity check from an inspection personnel other than the pre-registered inspection personnel who have already checked the validity.

If the re-requested evaluation result is the second or third response, the virtual labeling result is questionable, so the virtual labeling result can be replaced through a manual labeling process.

This may be to prevent the accuracy of the learning model from decreasing due to contamination of the learning data.

The schedule unit 140 may assign labeling tasks to the inspection staff based on the difficulty of labeling and the work capabilities of the inspection staff.

For example, the schedule department 140 provides labeling work to the first-class workforce at a rate of 20% for express labeling work, 30% for high-level labeling work, 30% for intermediate labeling work, and 20% for low-level labeling work, based on the total workload. can be distributed.

For example, the schedule department 140 may distribute labeling work to the second-class workforce at a ratio of 30% for high-level labeling work, 50% for middle-level labeling work, and 20% for low-level labeling work, based on the total workload.

For example, the schedule department 140 may distribute labeling work to third-class workers at a ratio of 20% to high-level labeling work, 30% to middle-level labeling work, and 50% to low-level labeling work.

The schedule unit 140 may request manual labeling of preprocessed raw data that satisfies predetermined start conditions from a plurality of employees.

For example, the schedule unit 140 may request manual labeling from 5 to 10 workers, but the present invention is not limited thereto, and the specific number can be modified in various ways at a level that is obvious to those skilled in the art. .

The schedule unit 140 transmits the manual labeling result, which is the result of manual labeling by business personnel, to the label management unit, and the label management unit 160 provides labeled learning based on the manual labeling result, which is the result of manual labeling by business personnel. Manual label data can be calculated.

The label management unit 160 can transmit generated learning data consisting of automatic label data and manual label data to the recommendation module.

Figure 7 is a diagram illustrating a manual labeling process executed by a labeling module provided by a source code providing server according to an embodiment of the present invention.

Referring to Figures 6 and 7, the label management unit can calculate manual label data based on manual labeling results that match a standard ratio or higher. (S310)

If the predetermined change conditions are not satisfied while there are manual label results that match more than the standard ratio, the label management unit can select the manual label results that match more than the standard ratio and calculate manual label data. (S313)

In addition, if a predetermined change condition is satisfied while there are manual label results that match more than the standard ratio, the label management unit can select the manual label result that satisfies the preset change condition and calculate manual label data. there is. (S312)

For example, the standard ratio may be 70%.

However, it is not limited to this, and the specific value of the standard ratio can be modified in various ways at a level that is obvious to those skilled in the art.

The predetermined change condition is that the first grade ratio of workers who have derived manual labeling results that match more than the standard ratio is less than the first ratio, and the first grade of workers who have derived manual labeling results that match less than the standard ratio are The condition may be that the ratio is greater than or equal to the second ratio. (S311)

Here, the first ratio may be 30% and the second ratio may be 60%.

For example, we can assume a situation where a total of 10 workers performed labeling work. Here, 9 workers (6 people in the 1st level, 2 people in the 2nd level, 1 person in the 3rd level) matched the A label to the raw data, and 1 worker (2nd level) matched the raw data with the A label. It can be assumed that the B labels are matched.

Here, in the presence of manual labeling results (A label, 90%) that match the standard ratio of 70% or more, the first grade among the workers designated as A is 66.67%, which is higher than the first ratio, so a predetermined change is made. Conditions may not be satisfied.

Therefore, the label management unit can select A label, which is a manual label result that matches more than the standard ratio, and calculate manual label data.

In contrast, we can assume a situation where a total of 10 workers performed the labeling task. Here, 7 workers (2 1st grade, 3 2nd grade, 2 3rd grade) matched the A label to the raw data, and 3 workers (all 1st grade) matched the raw data to the A label. It can be assumed that label B has been matched.

Here, in the presence of manual labeling results (A label, 70%) that match the standard ratio of 70% or more, the first grade among the workers designated as A label is 28.57%, which is below the first ratio and is labeled B. Among the designated work force, the first level is 100%, which is higher than the second rate, and the predetermined change conditions can be satisfied.

Therefore, the label management department determines the B label, which is the manual labeling result derived by workers whose first grade ratio is greater than or equal to the second ratio of workers who have derived manual labeling results that match less than the standard ratio and that satisfies the predetermined change conditions. By selecting, you can calculate manual label data.

In other words, the label management department matches the manual labeling results below the standard ratio when the predetermined change conditions are met, even if there are manual labeling results that match more than the standard ratio, and the group has the first grade ratio higher than the second ratio. The derived manual labeling results can be calculated as manual label data.

If there are no manual label results that match more than the standard ratio, the label management unit can determine whether there are manual label results that match the virtual label of the virtual label data. (S321)

The label management department may additionally proceed with the labeling process on the raw data preprocessed using the first verification method when there are no manual label results that match the standard ratio or when there are manual label results that match the virtual label of the virtual label data. You can. (S322)

Here, the first verification method may be a method of setting the labeling difficulty of the currently problematic preprocessed raw data to express and performing manual labeling again later using the schedule module.

This is a situation in which there is a high possibility that an answer exists among the results of the basic label model and the manual labeling results of the work force, and this may be to calculate accurate manual label data by preferentially assigning it to the first class work force.

The label management department performs a labeling process on the raw data preprocessed using the second verification method when there are no manual label results matching the standard ratio or when there are no manual label results matching the virtual label of the virtual label data. You can proceed further. (S323)

Here, the second verification method is to stop labeling the raw data that is currently problematic, save it separately by the label management department, and then restart the process of virtual labeling by the basic label model after a predetermined time has elapsed. You can.

This is a situation in which accurate labeling results cannot be derived at present, and may be re-evaluated after the performance of the basic label model and the work capabilities of the workforce grow to derive accurate label data.

The recommendation module can use the calculated manual label data to learn the basic label model instead of virtual label data.

The schedule department can settle the cost of labeling work at different costs depending on the level of labeling difficulty and provide it to the workforce.

For example, the cost of the labeling task may be lowered in the order of difficulty of the labeling task: special, high, intermediate, and low. In other words, the cost of the express labeling task may be the highest, and the cost of the low-grade labeling task may be the lowest.

The schedule department can set mentor schedules for the first-grade and third-grade workers, taking into account the region, age, gender, etc. of the workers.

For example, the schedule department may match mentor work to third-class workers who reside within a predetermined distance from the residence area of the first-class workers.

For example, the schedule department may set a mentor schedule for first-grade and third-grade workers of the same gender and whose age difference is less than a predetermined age.

The scheduling department can reduce the manual labeling task allocation ratio according to the mentoring schedule of first-level workers.

The schedule department can change the labeling work ratio of special, upper, middle, and lower levels of first-level workers participating in mentoring by reflecting the amount that must be paid for participation in mentoring.

To this end, based on the average number of manual labeling assigned to the first class of work personnel over a predetermined period, the ratio of labeling tasks of special, upper, middle, and lower levels assigned to the first class of work force can be changed.

The schedule department calculates the cost to pay for manual labeling based on the average number of manual labeling assigned to the first class of workers and the ratio of special, high, intermediate, and low level labeling tasks assigned to the first class of workers during a given period. can do.

For example, the schedule department calculates the payment cost for mentoring as the difference between the payment cost to be paid to the first-grade workforce before mentoring progresses and the payment fee to be paid to the first-grade workforce adjusted after the mentoring progresses. To achieve this, the work ratio of the first level workforce participating in mentoring can be adjusted.

Through this, the overall manual labeling work capacity can be increased by having the first-grade workers train the third-grade workers without causing significant disruption to the overall manual labeling work.

This improves the quality of learning data and can have the net function of improving the accuracy of models learned by the recommendation module.

In the attached drawings, in order to more clearly express the technical idea of the present invention, components that are unrelated or less relevant to the technical idea of the present invention are briefly expressed or omitted.

In the above, the configuration and features of the present invention have been described based on the embodiments according to the present invention, but the present invention is not limited thereto, and various changes or modifications may be made within the spirit and scope of the present invention. It is obvious to those skilled in the art, and therefore, it is stated that such changes or modifications fall within the scope of the appended patent claims.

110: Receiving module 120: Storage module
130: Recommended module 140: Inspection module

Claims (10)

In the source code provision server that recommends source code for executing a process to achieve the purpose,
A receiving module that receives request information reflecting the target purpose that the user wants to achieve;
A storage module in which code information, which is information about source code, is stored; and
Using a combination model learned through machine learning, it is run on the user's computing device based on the code information and the request information to determine the type of source code and a recommendation code that is a combination of the source code to achieve the target purpose. Includes a recommended module;
The requested information is,
It consists of purpose information, which is information about the target purpose, specification information, which is information about the specifications of the user's computing device, and usage environment information, which is information about the operation of the user's computing device.
The recommendation code is,
It is equipped with a preliminary recommendation code, which represents a combination of source code that can operate without error on the user's computing device while achieving the target purpose, and a final recommendation code selected from among the preliminary recommendation codes in consideration of the usage environment of the user's computing device.
The recommended module is,
If the load of the user's computing device is greater than a predetermined standard, the preliminary recommendation code that emphasizes efficiency is selected as the final recommendation code, and if the load of the user's computing device is less than a predetermined standard, the preliminary recommendation code that emphasizes stability is selected as the final recommendation code. Selected by the final recommendation code,
Source code provision server.
According to paragraph 1,
An inspection module that checks whether an error occurs when executing the process according to the recommendation code; and
When the inspection module determines that an error occurs when operating as the recommendation code, an explanation module that identifies the cause of the error using explainable artificial intelligence; further comprising,
Source code provision server.
According to paragraph 2,
It further includes an interface module that calculates an interface that displays information about the recommendation code or error to the user.
Source code provision server.
According to paragraph 1,
The recommendation code is,
Further comprising a combination recommendation code that is a combination of at least one final recommendation code configured to change and execute the final recommendation code according to a specified time,
Source code provision server.
According to clause 4,
The recommended module is,
Considering the load generated on the user's computing device, recommending the combination recommendation code in which the final recommendation code that is different in time series is executed,
Source code provision server.
According to paragraph 1,
In order to learn the combination model, it further includes a labeling module that performs labeling on the source code,
The labeling module is,
Equipped with an automatic labeling unit that automatically labels source code using a labeling model,
Source code provision server.
According to clause 6,
The labeling module is,
Further comprising an inspection unit that requests pre-registered inspection personnel to check the validity of the labeling performed by the automatic labeling unit,
The automatic labeling unit,
Re-learning the labeling model based on the evaluation results of the pre-registered inspection personnel,
Source code provision server.
In clause 7,
The labeling module is,
When predetermined start conditions are satisfied based on the evaluation results of the pre-registered inspection personnel, it further includes a schedule unit that manages the manual labeling work by a plurality of workers,
Source code provision server.
According to clause 8,
The labeling module is,
It is further equipped with a human resources department that evaluates the work capabilities of the workforce and a difficulty calculation department that evaluates the difficulty of labeling,
The schedule department,
Assigning labeling tasks to the workers based on the difficulty of labeling and the work capabilities of the workers.
Source code provision server.
In the source code provision method, which is implemented by a source code provision server and is a method of recommending a process to achieve the purpose,
Receiving, by a receiving module, request information reflecting the target purpose that the user wants to achieve; and
By the recommendation module, a combination model learned through machine learning is used and driven on the user's computing device based on the code information stored in the storage module and the request information to determine the type of source code to achieve the target purpose. Including a step of determining a recommendation code that is a combination of source codes,
The requested information is,
It consists of purpose information, which is information about the target purpose, specification information, which is information about the specifications of the user's computing device, and usage environment information, which is information about the operation of the user's computing device.
The recommendation code is,
It is equipped with a preliminary recommendation code, which represents a combination of source code that can operate without error on the user's computing device while achieving the target purpose, and a final recommendation code selected from among the preliminary recommendation codes in consideration of the usage environment of the user's computing device.
The step in which the recommendation code is determined is,
If the load of the user's computing device is greater than a predetermined standard, the preliminary recommendation code that emphasizes efficiency is selected as the final recommendation code, and if the load of the user's computing device is less than a predetermined standard, the preliminary recommendation code that emphasizes stability is selected as the final recommendation code. Selected by recommendation code,
How to provide source code.

Images