WO2019163823A1

WO2019163823A1 - Information processing system

Info

Publication number: WO2019163823A1
Application number: PCT/JP2019/006319
Authority: WO
Inventors: 敏弥河崎
Original assignee: 株式会社Ａｂｅｊａ
Priority date: 2018-02-20
Filing date: 2019-02-20
Publication date: 2019-08-29
Also published as: JPWO2019163823A1; JP6689507B2; JP2020113319A; JP7281427B2

Abstract

The objective of the present invention is to simplify the management of a machine learning model. A learning data input unit 101 receives an input of learning data. A learning method input unit 102 receives an input of a learning method. A learning condition selection unit 103 selects a set comprising a prescribed data and a prescribed learning method. A learning unit 104 uses the data and/or learning method selected by the learning condition selection unit 103 to carry out learning. An adoption determination unit 105 determines whether to adopt or not adopt a learning result output by the learning unit 104. An inference input unit 106 receives an input of an inference method. A model management unit 107 combines an adopted result and an inference method and sets a model version, and prepares one or more model versions for each task managing the same. An execution selection unit 108 linked to the model management unit 107 selects an execution environment and a model version managed by the model management unit 107, and an execution implementation unit 109 executes the model version in the execution environment.

Description

Information processing system

The present invention relates to an information processing system.

Conventionally, machine learning has been used in various fields. Specifically, the machine learning is a method in which a plurality of pieces of data are used for repetitive learning, and a computer can find a pattern hidden in the learning. And the technique using machine learning is proposed (for example, refer to patent documents 1).

Japanese Unexamined Patent Publication No. 2016-192007

However, with only the conventional technology including the technology described in Patent Document 1 described above, it has been difficult to update the model according to the data collection status or to operate in various execution environments.

The present invention has been made in view of such a situation, and an object thereof is to facilitate model management of machine learning.

In order to achieve the above object, an information processing system according to one embodiment of the present invention includes:
Data input means for inputting learning data;
Learning method input means for inputting a learning method;
Learning condition selection means for selecting the predetermined learning data input by the data input means and the predetermined learning method set input by the learning method input means;
Learning means for performing learning using at least one of data including the learning data selected by the learning condition selection means or a learning method;
Adopting decision means for deciding whether to adopt or not adopt the result of learning performed by the learning means;
An inference input means for inputting an inference method;
A task of managing one or more of the model versions by combining the learning result determined by the adoption determination unit and the inference method input by the inference input unit into a model version Model management means prepared for each,
Execution selection means associated with the model management means, execution selection means for selecting a model version and execution environment managed by the model management means;
Execution selection means for executing the model version selected by the execution selection means in the execution environment selected by the execution selection means;
Is provided.

According to the present invention, machine learning model management can be facilitated.

It is a block diagram which shows the hardware constitutions of the machine learning management operation system as one Embodiment of the information processing system of this invention. It is a functional block diagram which shows an example of a functional structure of FIG. It is a figure which shows an example of a learning job progress status confirmation screen. It is a figure which shows the example different from FIG. 3 of a learning job progress status confirmation screen. FIG. 5 is a diagram illustrating an example different from FIGS. 3 and 4 of a learning job progress status confirmation screen. It is a figure which shows an example of the version management screen of a model. It is a figure which shows the example different from FIG. 6 of the version management screen of a model. It is a figure which shows the example different from FIG.6 and FIG.7 of the version management screen of a model. It is a figure which shows the example different from FIG. 6 thru | or FIG. 8 of the model version management screen.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a hardware configuration example of a machine learning management operation system as an embodiment of an information processing system of the present invention.
The machine learning management operation system can take various various embodiments such as an embodiment constituted by a plurality of information devices via a network such as a server and a client. However, for convenience of explanation, in the present embodiment, the machine learning management operation system will be described as being configured by the information processing apparatus shown in FIG.

The information processing apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a GPU (Graphics Processing Unit) 14, a bus 15, and an input / output interface 16. An output unit 17, an input unit 18, a storage unit 19, a communication unit 20, and a drive 21.

The CPU 11 and the GPU 14 execute various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 19 to the RAM 13.
The RAM 13 appropriately stores data necessary for the CPU 11 or the GPU 14 to execute various processes.

The CPU 11, the ROM 12, the RAM 13, and the GPU 14 are connected to each other via the bus 15. An input / output interface 16 is also connected to the bus 15. An output unit 17, an input unit 18, a storage unit 19, a communication unit 20, and a drive 21 are connected to the input / output interface 16.

The output unit 17 is composed of a liquid crystal display or the like and outputs various information.
The input unit 18 is composed of various hardware leads and the like, and inputs various information.
The memory | storage part 19 is comprised by DRAM (Dynamic Random Access Memory) etc., and memorize | stores various data.
The communication unit 20 controls communication performed with other devices via the network N including the Internet.

The drive 21 is provided as necessary. A removable medium 31 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 21. The program read from the removable medium 31 by the drive 21 is installed in the storage unit 19 as necessary. The removable medium 31 can also store various data stored in the storage unit 19 in the same manner as the storage unit 19.

FIG. 2 is a functional block diagram showing an example of a functional configuration of the machine learning management operation system (information processing apparatus 1 in FIG. 1).

2, the learning data input unit 101, the learning method input unit 102, the learning condition selection unit 103, the learning unit 104, the employment determination unit 105, and the inference input unit 106 are included. The model management unit 107, the execution selection unit 108, and the execution execution unit 109 function.
Further, in the storage unit 19 of the learning processing apparatus of FIG. 1, a learning data DB 200 in which learning data is stored, a model DB 300 and a model DB 400 in which each model version is stored, and an execution environment in which execution environment data is stored A data DB 500 is provided.

Learning data input unit 101 inputs data used for learning (hereinafter referred to as “learning data”) such as an image including a learning target as a subject and data including information related to the learning target. The learning data is stored in the learning data DB 200.

The learning method input unit 102 inputs a learning method.

Learning condition selection unit 103 selects a set of predetermined data (including data selected by the user from one or more learning data stored in learning data DB 200) and a learning method.

The learning unit 104 performs learning using at least one of the data selected by the learning condition selection unit 103 and the learning method.

The hiring decision unit 105 decides whether or not to adopt the learning result output by the learning unit 104.

The inference input unit 106 inputs an inference method.

The model management unit 107 sets the learning result determined by the adoption determination unit 105 and the inference method input by the inference input unit 106 as a model version, and manages each of the model versions for one or more tasks. Prepare to.
That is, in the example of FIG. 2, each of the model DB 300 and the model DB 400 stores one or more model versions (all three model versions in the example of FIG. 2) for a predetermined task. Therefore, the model management unit 107 can also manage each model version for each task using the model DB for each task such as the model DB 300 and the model DB 400.

The execution selection unit 108 associated with the model management unit 107 selects a model version and an execution environment managed by the model management unit 107.

The execution execution unit 109 executes the model version in the execution environment.

As described above, the machine learning management operation system to which the present invention is applied can solve the following problems, and as a result, the following effects can be achieved.
That is, there has been a problem that the operation of updating the model according to the data collection status or operating in various execution environments has been difficult in the past.
Since the machine learning management operation system to which the present invention is applied can facilitate the model management, it is possible to achieve an effect that such a problem can be solved.

In other words, the machine learning management operation system to which the present invention is applied can manage both the version of the learning result for each task and the execution environment in a centralized manner on the cloud, and can solve such problems. The effect that it is.

Machine learning is not a profit in itself, but it is important to verify its effectiveness by taking measures at the site of introduction. In other words, the accuracy of machine learning itself is information for reference. Here, by managing the version and the execution environment in pairs, it is linked which model of machine learning actually leads to the customer's value. From the individual effects of the version management mechanism that only knows information of the reference level and the mechanism that selects the execution environment, the above effects cannot be directly inferred. The above effect can be realized only by connecting.

FIG. 3 is a diagram illustrating an example of a learning job progress confirmation screen.
The learning job progress status confirmation screen of the example of FIG. 3 is displayed on the output unit 17 of FIG. This also applies to the example of FIG. 4 and the example of FIG. 5 described later.
By using the learning job progress confirmation screen in the example of FIG. 3, the user assigns each of a plurality of learning jobs to one learning definition (Job Definition) for learning executed by the learning unit 14 (FIG. 2). They can be executed independently of each other.
Here, the learning definition is, for example, Ver. 11 or Ver. The learning definition of a predetermined version corresponds to the above-mentioned “one learning definition”.
The learning job includes data used when learning is executed by the learning unit 104, that is, at least one of data selected by the learning condition selection unit 103 and a learning method.
By using the learning job progress confirmation screen in the example of FIG. 3, the user can select “status”, “comment”, “execution user”, “progress”, “corresponding learning definition” for each of the plurality of jobs. "Version", "Optionally defined items and values", "Elapsed time", "Execution start date", "Job execution log", "Learning status details confirmation" confirmation, and "Learning result download" be able to.

FIG. 4 is a diagram showing an example different from FIG. 3 of the learning job progress status confirmation screen.
The learning job progress confirmation screen in the example of FIG. 4 corresponds to “Main” on the lower side of the learning job progress confirmation screen of the example of FIG. 3, but is different from the example in FIG. The situation where the user is writing comments).
For example, the execution user can leave a comment for each learning job by using the learning job progress confirmation screen in the example of FIG. As a result, the machine learning management and operation system (the information processing apparatus 1 in FIG. 1) can record the background of execution of the learning job and comments on the results. This is useful when confirmation after a certain period of time or when a user other than the executing user investigates a learning job.

FIG. 5 is a diagram showing an example different from FIGS. 3 and 4 of the learning job progress confirmation screen.
The learning job progress confirmation screen in the example of FIG. 5 corresponds to “Main” on the lower side of the learning job progress confirmation screen of the example of FIG. 3, but each of the examples in FIGS. It is in a different situation (a situation where the executing user writes a comment).
By using the learning job progress confirmation screen shown in the example of FIG. 5 (particularly by visually recognizing the portion surrounded by the thick line frame in FIG. 5), the “arbitrarily defined items and numerical values” and “status” are displayed. And a function of selecting or ordering learning jobs that are considered to be most effective from a plurality of learning jobs, and displaying them on the output unit 17 in FIG. The learning management and operation system (the information processing apparatus 1 in FIG. 1) can be exhibited.

FIG. 6 is a diagram illustrating an example of a model version management screen.
The model version management screen of the example of FIG. 6 is displayed on the output unit 17 of FIG. This also applies to the example of FIG. 7 and the example of FIG. 8 described later.
The user uses the model version management screen (stored in the model DB 300, the model DB 400, etc.) managed by the model management unit 107 by using the model version management screen in the example of FIG. Managed source code. The user can record the version number (Version) and the comment at the time of update in each update of the model version. The user can record the history of the update and the memo of the deployment result in the comment.

FIG. 7 is a diagram showing an example different from FIG. 6 of the model version management screen.
The model version confirmation screen of the example of FIG. 7 corresponds to the upper right area of the model version management screen of the example of FIG. 6, but is in a different situation from the example of FIG. In a situation where it is desired to create
The user can create a new model version (new version of the model) by pressing the “Create Version” software button on the model version management screen in the example of FIG.

FIG. 8 is a diagram showing an example different from FIGS. 6 and 7 of the model version management screen.
When the “Create Version” software button is pressed on the model version management screen of the example of FIG. 7 described above, the screen displayed on the output unit 17 of FIG. 1 displays the version of the model of the example of FIG. Transition to the management screen.
The user can create a new model version (a new version of the model) by using the model version management screen in the example of FIG. 8 (instruction operation thereof), and can also use “version number (Version)” and “use”. Execution environment settings and other settings can be specified.

FIG. 9 is a diagram showing an example different from FIGS. 6 to 8 of the model version management screen.
The model version management screen in the example of FIG. 9 corresponds to the lower area of the model version management screen in the example of FIG. In other words, when the user is creating a new model version (a new version of the model) (instructing operation thereof), the user can display the screen displayed on the output unit 17 of FIG. 1 by performing a scroll operation or the like. The model version management screen in the example of FIG. 8 can be changed to the model version management screen of the example in FIG.
The user performs an operation of dragging and dropping the source code (icon) on the gray area below “Source Code” on the version management screen of the model in the example of FIG. An operation to select a file (source code) can be performed.

The machine learning management operation system as an embodiment of the information processing system of the present invention has been described above. The machine learning management and operation system can be applied to an information processing system such as the following (1) or can have various functions such as the following (2) to (11).
(1) AI consulting recommendation system using model learning data in the cloud.
(2) A function for independently managing the source code of the model together with the learning result.
(3) Function of Super Awesome Model Platform (samp).
(4) A billing plan function according to the achievement of model learning.
(5) Blue / Green deployment function that guarantees accuracy improvement.
(6) A function having a confirmation interface for Web API (Application Programming Interface).
(7) A function for managing the same model for each application using the concept of deployment.
(8) A function for automatically saving an inference request (raw data) and an inference response (inference result).
(9) Visualization (monitoring, graphing) and alert functions regarding inference results.
(10) Inference result visualization function.
(11) A function for checking the true value of the inference result.

(1) Features of an AI consulting recommendation system (hereinafter simply referred to as “recommendation system”) using model learning data in the cloud are described below.
Since the recommendation system is completed only in the cloud, the user's action history can be grasped.
For example, if it is strange that only this level of accuracy is available for this amount of data and this category, the recommendation system can recommend the user to consult the consultant. .
By using the recommendation system, the user can understand the network used by analyzing the source code.
The recommendation system has originality compared to a chatbot.
The recommendation system can recommend appropriate consulting from there.
The recommendation system can issue a request to the consultant with understanding the troubled points in advance according to the action history.
The recommendation system can be used in Japan, India, the US and countries around the world.

The recommendation system can solve the problem that without the know-how of machine learning / deep learning, even if learning is executed, accuracy cannot be achieved as expected.
As a result, the recommendation system can produce an effect that it is possible to recommend an appropriate AI consulting at an appropriate timing by analyzing the user behavior history and the source code used for learning the model.

Here, in the past, there was only a solution like a chatbot that provided a channel with consulting. On the other hand, the recommendation system has a mechanism for making a recommendation based on an analysis result of action history and source code stored on the cloud.

The recommendation system has the basicity that the need for efficient consulting is very high because there are few companies with know-how for the demand of machine learning / deep learning.

Also, in the recommendation system, some channels (Web interface, e-mail, etc.) should have become apparent in order to contact consulting.

Next, (2) a function (hereinafter referred to as “function (2)”) that independently manages the source code of the model together with the learning result will be described.

The characteristics of function (2) are described below.
Conventionally, local development requires manual operation such as directory management.
On the other hand, the source code of the model in which the data set, code, parameter, and log used for learning are version-controlled is linked on the cloud by the function (2).
As a result, the learning result and the source code can be managed separately, so that the license problem can be solved.
Furthermore, function (2) can manage software copyright and model rights separately.

The function (2) can solve the problem that there is a possibility that the right relationship between the product (such as network weight) generated as a result of the model learning and the source code for executing the inference may be different.
As a result, the function (2) has the effect that different rights / licenses can be applied to each by managing the product generated as a result of the model learning and the source code for executing the inference as separate components. be able to.

Function (2) has the basicity that it is important for the platform to clarify the right relationship between the source code created by the developer who is the user (user) and the learning result generated on the cloud. .

Function (2) has the obviousness that can be understood by looking at the handling of source code and learning results.

Next, the function of (3) Super Awesome Model Platform (samp) (hereinafter referred to as “function (3)”) will be described.

The characteristics of function (3) are described below.
Function (3) provides preprocessing of input / output data while hiding it from the user (user).
Function (3) is a mechanism for making the user program and the execution environment independent.
The function (3) is HTTP (Hypertext Transfer Protocol).
Function (3) can be event driven on the cloud.
Function (3) can be performed on the edge device.
Function (3) can reduce the man-hours of the developer as one user.

Next, (4) a billing plan function (hereinafter referred to as “function (4)”) according to the degree of achievement of model learning will be described.

The characteristics of function (4) are described below.
Function (4) has a new effect of machine learning cloud only.
The function (4) can reduce the cost by not committing when the learning result is not satisfied.
Function (4) can be extended if learning takes time.
Function (4) can finally be canceled.

Function (4) is that it is difficult to predict in advance how much accuracy will be achieved in “how much time” for model learning in machine learning / deep learning. It is possible to solve the problem of wanting to suppress the problem.

As a result, the function (4) can achieve the following effects.
In other words, the operations “open” and “commit” are defined as follows in the model learning in the cloud.
“Open” represents the start of learning. “Commit” represents that learning is completed and a model is created from the learning result.
In this case, the function (4) charges the model learning at the time of “open” and “commit”, and keeps the price at the time of “open” lower than the price at the time of “commit”. The (user) can achieve an effect that the cost until learning is completed can be reduced.

Here, conventionally, in model learning, it is common to charge based on the performance / usage time of a GPU used as an infrastructure. On the other hand, the function (4) is a new method of defining the degree of achievement of model learning and charging according to it.

Function (4) is that in the process of machine learning / deep learning model learning shifting to the cloud, it is considered that the user's awareness of the GPU 14 itself becomes obsolete, and the charging system corresponding to the GPU 14 is no longer used. Should come.

Function (4) has the obviousness of confirming the billing plan.

Next, (5) Blue / Green deployment function (hereinafter referred to as “function (5)”) that guarantees accuracy improvement will be described.

The features of function (5) are described below.
The function (5) can be switched instantaneously on a UI (User Interface).
Function (5) can compare the accuracy between different versions by sending one or more test data sets as requests to each endpoint.
In function (5), the accuracy comparison is visualized, and the best one can be selected as the primary.

For function (5), when updating to a new version of the API, confirm that the accuracy has been improved over the previous version of the API, and then immediately update (zero downtime release with Blue / Green deployment) Can solve the problem.
As a result, the function (5) can compare the accuracy of the current version and the new version using the same data set when updating the API. While confirming the comparison result, there is an effect that it is possible to determine whether to switch to the new version.

Here, unlike general Web services, the accuracy of the model is important as an index in the machine learning / deep learning API.
Function (5) can use multiple versions of accuracy comparison results under the same conditions for Blue / Green deployment.

Function (5) has the basicity that, as described above, the accuracy of the model is important in the API of machine learning / deep learning, and general Blue / Green deployment is not sufficient.

Function (5) has the obviousness that it can be confirmed by the WebAPI deployment method.

Next, the function (6) having a Web API confirmation interface (hereinafter referred to as “function (6)”) will be described.

The characteristics of function (6) are described below.
The interface of the function (6) maintains the ease of comparing the input image and the output, and makes each drawing area independent in order to increase the visibility.
Function (6) can process the input image.
Function (6) can use a data lake file associated with the annotation service for input.
Function (6) can also be compared with the results.
The function (6) can analyze the output result and superimpose the bounding box on the input image or devise the output.

Function (6) indicates that the input and output results including the teacher data of the machine learning / deep learning model are consistent with the unstructured text data of the inference result and the teacher data. Can solve the problem of wanting to confirm.

As a result, the function (6) can achieve the following effects.
With the function (6), a confirmation interface that can be used in the Web browser can be prepared.
This interface is divided into an input data drawing area and an output data drawing area, and each can be scrolled independently. Thereby, the visibility is improved while maintaining the ease of comparing the input image and the output.
The following data (a) to (c) can be used as input data.
(A) A sample image incorporated in the platform.
(B) Uploaded image.
(C) Images and teacher data (unstructured text data) stored in the data lake.
Here, when there is teacher data of data (c), the unstructured text data is analyzed and superimposed on the image as a color-coded rectangle to improve visibility.
As output data, unstructured text data or an image can be acquired as an inference result.
In that case, the image is drawn as it is. Also, the unstructured text data is analyzed and superimposed on the image as a color-coded rectangle.
Since the function (6) can provide these interfaces, an effect of facilitating comparison between input and output can be achieved.

Function (6) is unique in the interface that automatically analyzes input teacher data / images and unstructured text data output from the API and renders them as image data.

Function (6) has the basicity that it is important to facilitate confirmation of deployed Web API.

Function (6) has the obviousness that it can be confirmed because it is an interface.

Next, (7) a function for managing the same model for each application using the concept of deployment (hereinafter referred to as “function (7)”) will be described.

Function (7) has the feature that it is divided for each store and for each edge.

Function (7) can solve the problem that it is difficult to manage different settings and capacities for each application, although it is desired to reuse the same model for different applications.
As a result, the function (7) can execute a certain model on a plurality of infrastructures, and can achieve the effect that the scale of the infrastructure and the program settings necessary for execution can be managed in units of “deployment”. .

Function (7) has the basicity that it is important because the performance and settings of the same model need to be changed in response to different requests in the SaaS service or the like.

Function (7) has the obviousness of being understood because it is set by the developer.

Next, (8) a function for automatically saving an inference request (raw data) and an inference response (inference result) (hereinafter referred to as “function (8)”) will be described.

Function (8) is not inferred and finished, but later confirms that the inference result is correct. Therefore, as a learning data for improving accuracy, other requests for inference (raw data) and inference The response (inference result) is stored automatically.

Function (8) can solve the following problems.
That is, inference is neither 100% accurate nor 100% correct. Therefore, the correct accuracy cannot be measured unless the inferred data and results are confirmed. Furthermore, the action to improve accuracy is also difficult. A mechanism for checking accuracy later is required.
Since the function (8) can solve such a problem, the accuracy can be easily checked later by automatically storing the inferred result.

Next, (9) Visualization (monitoring, graphing) regarding the inference result and alert function (hereinafter referred to as “function (9)”) will be described.

Function (9) has a feature of graphing whether the accuracy of the inferred result satisfies the intended standard, and sending an alert if the accuracy is not satisfied.

Function (9) can solve the following problems.
That is, there may be a case where the external environment for inference does not satisfy a certain level. For example, when counting the number of people with a camera, changing the proof of the camera installation environment may greatly affect the accuracy and reduce the accuracy. In that case, it is necessary to detect the abnormality and take action.
Since the function (9) can solve such a problem, it is possible to obtain an effect that an abnormality can be detected and dealt with promptly.

Next, (10) the inference result visualization function (hereinafter referred to as “function (10)”) will be described.

Function (10) can display a label indicating what the image is classified as a result of inferring the image and a thumbnail of the image. In addition to classification, segmentation and bounding boxes are also supported.

The function (10) can solve the following problems.
That is, in a case where it is impossible to measure whether the inferred result is correct numerically, it is necessary to check the accuracy by checking with human eyes. It is necessary to save the data and check the data for easy confirmation.
The function (10) can produce an effect that development and workflow for checking data are unnecessary.

Next, (11) the function of checking the true value of the inference result (hereinafter referred to as function (11)) will be described.

The features of function (11) are described below.
That is, in a case where it cannot be measured numerically whether the inferred result is correct, it is necessary to check the accuracy by checking with human eyes. Annotation is manually performed on a part of the data obtained by sampling the inferred result, and the true value is checked.

The function (11) can solve the problem of how to check the accuracy in a case where it is impossible to numerically determine whether the inferred result is correct.
As a result, the function (11) can bring about an effect that it is possible to reduce the assignment and management cost of the person for checking the true value.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, and the like within a scope that can achieve the object of the present invention are included in the present invention. is there.

For example, the series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 2 is merely an example and is not particularly limited.
That is, it is sufficient that the machine learning management operation system has a function capable of executing the above-described series of processes as a whole. The configuration of the machine learning management operation system is not particularly limited to the example of FIG. What functional blocks are used for the implementation is not particularly limited to the example of FIG. Further, the location of the functional block is not particularly limited to that shown in FIG. For example, the machine learning management operation system may include a server and a client. In this case, each functional block of the information processing apparatus 1 in FIG. 1 may be distributed to the server or the client and transferred.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

For example, when a series of processing is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware.
The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or personal computer other than a server.

Further, for example, a recording medium including such a program is not only constituted by a removable medium (not shown) distributed separately from the apparatus main body in order to provide a program to the user, but is also pre-installed in the apparatus main body. And a recording medium provided to the user.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall apparatus configured by a plurality of devices, a plurality of means, and the like.

In other words, an information processing system to which the present invention is applied only needs to have the following configuration, and can take various embodiments.
That is, an information processing system to which the present invention is applied is
Data input means for inputting learning data (for example, the learning data input unit 101 in FIG. 2);
Learning method input means for inputting a learning method (for example, the learning method input unit 102 in FIG. 2);
Learning condition selection means (for example, the learning condition selection unit 103 in FIG. 2) for selecting the predetermined learning data input by the data input means and the predetermined set of learning methods input by the learning method input means; ,
Learning means (for example, the learning unit 104 in FIG. 2) that performs learning using at least one of the data including the learning data selected by the learning condition selection means and the learning method;
Employment determination means (for example, the employment determination unit 105 in FIG. 2) for determining whether to adopt or not adopt the result of learning performed by the learning means;
An inference input means for inputting an inference method (for example, the inference input unit 106 in FIG. 2);
A task of managing one or more of the model versions by combining the learning result determined by the adoption determination unit and the inference method input by the inference input unit into a model version Model management means (for example, model management unit 107 in FIG. 2) prepared for each,
Execution selection means associated with the model management means, an execution selection means for selecting a model version and execution environment managed by the model management means (for example, the execution selection unit 108 in FIG. 2);
Execution selection means (for example, the execution execution unit 109 in FIG. 2) that executes the model version selected by the execution selection means in the execution environment selected by the execution selection means;
Is provided.

DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus, 11 ... CPU, 14 ... GPU, 101 ... Learning data input part, 102 ... Learning method input part, 103 ... Learning condition selection part, 104 ...・ Learning unit, 105... Adoption decision unit, 106 .. inference input unit, 107... Model management unit, 108... Execution selection unit, 109. DB, 300, 400 ... model DB, 500 ... execution environment data DB

Claims

Data input means for inputting learning data;
Learning method input means for inputting a learning method;
Learning condition selection means for selecting the predetermined learning data input by the data input means and the predetermined learning method set input by the learning method input means;
Learning means for performing learning using at least one of data including the learning data selected by the learning condition selection means or a learning method;
Adopting decision means for deciding whether to adopt or not adopt the result of learning performed by the learning means;
An inference input means for inputting an inference method;
A task of managing one or more of the model versions by combining the learning result determined by the adoption determination unit and the inference method input by the inference input unit into a model version Model management means prepared for each,
Execution selection means associated with the model management means, execution selection means for selecting a model version and execution environment managed by the model management means;
Execution selection means for executing the model version selected by the execution selection means in the execution environment selected by the execution selection means;
An information processing system comprising: