JP6781956B2 - Learning result comparison device, learning result comparison method, and its program - Google Patents

Description

The present invention relates to a technique for comparing learning results.

Conventionally, research on artificial intelligence technology such as neural networks (hereinafter referred to as "AI technology") has been widely conducted (see, for example, Patent Document 1). In particular, with the rise of AI technology called deep learning, for example, the recognition rate of object recognition technology using images has improved rapidly in the last few years, and the level of image classification exceeds the human recognition rate. Is reaching. Deep learning technology is not limited to image recognition, but also voice recognition, personal authentication, behavior prediction, sentence summarization, automatic translation, monitoring, automatic driving, failure prediction, sensor data analysis, song genre judgment, content generation, It is expected to be applied to security systems and a wide range of other fields.

Japanese Patent No. 5816771

In machine learning such as deep learning, it is possible to train a machine to acquire a predetermined ability. The ability acquired as a result of learning can be used on another device via a storage medium or communication means by converting it into data as information on the structure and parameters of the learned neural network, for example. ..

In a general-purpose learning device or a system that provides a learning service, many learning results are output. Further, even in a learning device for a specific purpose, if there are many sets of learning data, many learning results will be output as well. Since the difference in the learning result affects the result when the learning result is used, the user of the learning result needs to grasp the difference in the learning result. However, as AI technology becomes more widespread in the future, as a wide variety of machine learning is performed in various places, it may become more difficult to understand how the learning results differ from each other. Expected.

Therefore, an object of the present invention is to provide a technique for comparing and appropriately evaluating learning results obtained by performing predetermined learning by machine learning.

The comparison device according to one aspect of the present invention is connected to a database that stores learning results obtained by performing predetermined learning by machine learning, and extracts a plurality of learning results from the database, and an extraction unit. It is provided with a comparison unit for comparing the abilities acquired by each of the plurality of learning results extracted in step 1, and an output unit for outputting the comparison result by the comparison unit.

According to such a configuration, the learning results obtained by performing a predetermined learning by machine learning can be compared and appropriately evaluated, and the learning results different from the learning results intended by the user are used. It is possible to prevent the problem of not being able to obtain results and utilize appropriate learning results. Further, since the comparison processing is performed on a desired part of the learning results stored in the database, the processing load and the communication load can be suppressed.

Each of the learning results stored in the database is given identification information that can distinguish the learning results from each other, and the comparison unit may perform the comparison based on the identification information given to the learning results. According to such a configuration, since the learning results can be compared based on the identification information given to the learning results, the load of the comparison processing can be suppressed and the comparison processing speed can be improved.

The identification information may be able to mutually identify a plurality of learning results extracted by the extraction unit within a specific identification range, for example, within a specific group. According to such a configuration, by narrowing the identification range, the amount of identification information data required for the comparison processing can be reduced, so that the processing load and the communication load can be suppressed.

The comparison unit performs comparison processing using a comparison program specified based on a plurality of learning results extracted by the extraction unit. According to such an invention, since the comparison program can be autonomously specified and acquired based on the features common to the extracted learning results, the reliability of the comparison process can be improved.

Further, the comparison method according to one aspect of the present invention is a comparison method of learning results executed by a computer provided with a control unit. The computer includes a step of extracting a plurality of learning results from a database, a step of comparing the abilities acquired by each of the extracted learning results, and a step of outputting the comparison result.

The program according to one aspect of the present invention is a means for extracting a plurality of learning results from a database connected to a database that stores learning results obtained by performing predetermined learning by machine learning. It functions as a means for comparing the abilities acquired by each of the plurality of learning results and as a means for outputting the comparison results.

According to the present invention, it is possible to provide a technique for comparing and appropriately evaluating the learning results obtained by performing predetermined learning by machine learning.

It is a figure which shows the outline of the whole system which provides the learning service in this embodiment. It is a figure which conceptually shows the flow of the learning service in this embodiment. It is a block diagram which shows an example of the structure of the training data preparation device. It is a block diagram which shows an example of the structure of the learning requesting apparatus. It is a block diagram which shows an example of the structure of the learning result utilization apparatus. It is a block diagram which shows an example of the structure of a learning result database. It is a figure which shows an example of the basic factor. It is a figure which shows an example of an influence factor. It is a block diagram which shows an example of the structure of the learning request reception device. It is a block diagram which shows an example of the structure of a learning device. It is a block diagram which shows an example of the structure of the learning management apparatus. It is a figure which shows an example of the list of identification ranges. It is a figure which shows an example of the structure of the information about learning managed in a learning result database. It is a flowchart which shows an example of the processing of a learning request system. It is a flowchart which shows an example of the processing of the learning service providing system. It is a flowchart of the process of comparing learning results. It is a block diagram which shows an example of the structure of an inspection machine.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same elements are designated by the same reference numerals, and duplicate description will be omitted. In addition, the following embodiments are examples for explaining the present invention, and the present invention is not intended to be limited only to the embodiments. Furthermore, the present invention can be modified in various ways as long as it does not deviate from the gist thereof.

<1. System overview>
The outline of the system in this embodiment will be described with reference to FIGS. 1 and 2.
FIG. 1 is a diagram showing an outline of the entire system. This system includes a learning service providing system 1 and a learning request system 2, and both are connected to each other via a network N0 such as the Internet. The configuration of this system is not limited to the one shown in the figure, and for example, the learning service providing system 1 and the learning request system 2 may be integrated.

As shown in FIG. 1, the learning service providing system 1 has a learning result database 11, a learning request receiving device 12, one or more learning devices 13, and a learning management system 14, which are local networks N1. Are connected to each other via. Further, the learning request system 2 has a learning data preparation device 21, a learning request device 22, and one or more learning result utilization devices 23, which are connected to each other via a local network N2. Further, a learning data input device 24 is connected to the learning data preparation device 21 and the learning result utilization device 23, respectively.

In the following description, each device included in the learning service providing system 1 and the learning request system 2 will be described as being composed of a computer such as a PC (Personal Processor) or a server device, but the present invention is not limited to this. For example, it may be realized by an arbitrary embedded device using a processor. Further, the functions implemented in each device are not limited to the configuration realized by software. Any function included in each device may be configured to be realized by hardware. For example, the neural networks 132 and 233 described later may be configured by electronic circuits such as a custom LSI (Large-Scale Integration) and an FPGA (Field-Programmable Gate Array). Further, a part of each device may be configured to be realized by a physical key, a biochemical method such as DNA, or an identification means by an optical method such as a hologram.

FIG. 2 is a diagram conceptually showing the flow of the learning service by the system shown in FIG. The learning request system 2 accesses the learning service providing system 1 via the network N0, and requests machine learning of the ability to be added to the learning result utilization device 23. Information necessary for performing machine learning (hereinafter, also referred to as “learning request information”) is provided from the learning request device 22 to the learning request receiving device 12 of the learning service providing system 1 (S1). The learning request information includes, for example, a learning goal, information for identifying a device that uses the learning result, learning data, and the like.

The learning request information is transmitted from the learning request receiving device 12 to the learning device 13 (S2). The learning device 13 performs learning based on the learning request information, and acquires a new ability as a learning result. The learning result is converted into replicable packaged data (S3; hereinafter, also simply referred to as “learning result”). The dataized learning result is transmitted from the learning device 13 to the learning result database 11 and managed in a format that can be searched by various users (S4. Details will be described later). The various users include the creator of the learning program, the provider of the learning result, the user who uses the learning result, and other people who are involved in the learning result. Further, the learning result is transmitted to the learning result utilization device 23 of the learning request system 2, and the learning result is used in the learning result utilization device 23 to exert the ability (S5).

<2. Function>
<2-1. Learning request system 2>
Next, the function of the learning request system 2 will be described with reference to FIGS. 3 to 5. The learning request system 2 is not limited to the configuration described below, and may be configured by, for example, a single device.

First, the learning data input device 24 acquires information about a target to be input data of the neural network 233, which will be described later, of the learning result utilization device 23, and data to be input data of the learning data preparation device 21. For example, as the learning data input device 24, any input device such as a camera, a sensor, a network terminal, or a sensor of a self-propelled robot can be used.

FIG. 3 is a functional block diagram of the learning data preparation device 21 according to the present embodiment. The learning data preparation device 21 has a function of preparing data (learning data) necessary for learning by a learning module (for example, a neural network 233 described later). As shown in FIG. 3, the learning data preparation device 21 has an operation unit 211, a learning data acquisition unit 212, a learning data storage unit 213, and a data acquisition control unit 214 as functional units.

For example, the operation unit 211 receives an operation from a user of the learning request system 2 (hereinafter, also referred to as a “requester”). The learning data acquisition unit 212 acquires data necessary for creating learning data from the learning data input device 24, and stores it in the learning data storage unit 213. The data acquisition control unit 214 controls the operation unit 211, the learning data acquisition unit 212, and the learning data storage unit 213 to prepare data necessary for learning. The communication unit 216 connects to the local network of the learning request system 2 and transmits the data required for learning created by the data acquisition control unit 214 to another device.

The learning data preparation device 21 may be constructed by using the same device as the learning result utilization device 23 described later. In this case, the learning data preparation device 21 can be constructed as an input device externally connected to the learning result utilization device 23.

FIG. 4 is a functional block diagram of the learning request device 22 according to the present embodiment. The learning request device 22 has a function of transmitting learning request information to the learning service providing system 1. As shown in FIG. 4, the learning request device 22 has a learning request unit 221, a learning request content storage unit 222, a learning data storage unit 223, and a communication unit 224.

The learning request unit 221 receives a learning request from the requester, creates learning request information, and transmits the learning request information to the learning service providing system 1 via the communication unit 224. The learning request information transmitted at this time is stored in the learning request content storage unit 222. Further, the learning data transmitted from the learning data preparation device 21 is stored in the learning data storage unit 223.

Further, the learning requesting device 22 may refer to the identification information (details will be described later) given to the learning result and determine whether or not the input learning result is suitable for the purpose of use. When the learning result is suitable for the purpose of use, the learning request device 22 incorporates the learning result into the learning result utilization device 23 and makes the learning result used. On the other hand, when the learning result does not match the purpose of use, the learning requesting device 22 can notify the learning service providing system 1 of the nonconformity. At this time, the learning requesting device 22 may request learning again under different conditions.

FIG. 5 is a functional block diagram of the learning result utilization device 23 in the present embodiment. The learning result utilization device 23 has a function of providing a new ability to the user by utilizing the learning result. As shown in FIG. 5, the learning result utilization device 23 has a learning result input unit 231, a neural network setting unit 232, a neural network 233, a control unit 234, an input unit 235, and a communication unit 236 as functional units. And a data acquisition unit 237 and an output unit 238.

The learning result input unit 231 accepts the input of the learning result. At this time, the neural network setting unit 232 sets the neural network 233 according to the purpose of use. Further, the control unit 234 controls the data acquisition unit 237 and the input unit 235 to input the data necessary for using the learning result into the neural network 233 and use the learning result. The result of using the learning result is output from the output unit 238 via the communication unit 236.

<2-2. Learning service provision system 1>
The function of the learning service providing system 1 will be described with reference to FIGS. 6 to 10. The learning service providing system 1 can be implemented using a data center or a cloud. In this case, each device of the learning service providing system 1 can be constructed by using a PC server or a blade PC. By constructing each device of the learning service providing system 1 with a plurality of PCs, it is possible to shorten the processing time when executing repetitive operations such as deep learning. The learning service providing system 1 may have a configuration constructed by one PC or a configuration implemented by an embedded device.

FIG. 6 is a block diagram showing the configuration of the learning result database 11 in the present embodiment. The learning result database 11 stores various information required for learning. The learning result database 11 has a function of identifying individual learning results, a function of extracting learning results based on predetermined search conditions, and a function of outputting and presenting detailed information about the extracted learning results. You may have.

As shown in FIG. 6, the learning result database 11 has a learning data DB 111, a learning request DB 112, a learning result utilization information DB 113, a version control DB 114, a learning program DB 115, and a factor information DB 116.

The learning data DB 111 stores learning data used for learning. For example, the learning data DB 111 can store the learning data in association with the learning requirements such as the target of the learning data, the breakdown of the learning data, the range of the learning data, and the purpose of learning.

The learning request DB 112 stores the learning request information and the content of learning to be performed when the learning request information is received from the learning request device 22. For example, the learning request DB 112 can store information about a learning requester, a target of learning data, a breakdown of learning data, a range of learning data, a purpose of learning, and other learning request requirements in association with the learning request information.

The learning result usage information DB 113 stores information on the learning result results and usage history. For example, the learning result utilization information DB 113 can store information about the result of classification by using the acquired classification ability when the classification ability is acquired as a learning result. Further, the learning result usage information DB 113 is related to the use of the learning result in relation to the information about the user of the learning result, the target of the learning data, the breakdown of the learning data, the range of the learning data, the purpose of learning, and the like. Information can be stored. Further, the learning result utilization information DB 113 preferably includes usage identification information. The usage identification information is information that can identify the usage and results of the learning result, and includes, for example, information on settings that affect the ID and ability of the device to be used (basic factors and influential factors described later). Since the acquired ability may change depending on the device to be used and the environment in which it is used, the usage history can be recorded in more detail by including the usage identification information in the learning usage history DB 133.

The version control DB 114 stores information related to variations in abilities acquired by learning. For example, the version control DB 114 can store information related to the variation of ability for each learning result. The information stored in the version control DB 114 includes learning results and learning result identification information, which will be described later. That is, the version control DB functions as a storage unit that stores learning results and information that can identify the learning results.

The learning program DB 115 stores a learning program for performing learning. For example, the learning program DB 115 can store a learning program in association with learning requirements such as a learning target, contents of learning data, and learning goals. It is preferable that a large number of learning programs can be registered in the learning program DB 115. In this case, the learning device 13, which will be described later, can identify the learning program from the learning program DB 115 and make it executable by designating the learning requirements. Further, the learning program DB 115 stores a program for comparing the abilities of learning results for each learning program. This comparison program is created in advance for each learning program. The comparison program is not limited to the preparation for each learning program, and the comparison program may be prepared for each learning target.

Here, an example of the learning target will be specifically described. In the present embodiment, the learning target can be set for each learning program. Learning targets include things and things that are targeted by the abilities acquired as a result of learning. For example, if the learning result is the ability to classify agricultural products, then the learning target is agricultural products. At this time, the ability to classify the grades of agricultural products to be learned may differ depending on the learning program and the range of learning data used for learning. If the learning result is the ability to classify the grades of cucumbers and tomatoes, the learning target is cucumbers and tomatoes. In addition, for example, if the learning result is the ability to create a summary of a news article, the learning target is a news article, and if the learning result is the control of an assembly device for a specific customer, the learning target is specified. Control of product assembly equipment.

The factor information DB 116 stores information on factors that influence the learning results and abilities obtained as a result of learning. The information stored in the factor information DB 116 is managed hierarchically and has a tree-like structure. In this embodiment, the factors are roughly classified into basic factors and influential factors. The basic factor is a factor used to identify the learning result in the upper hierarchy. On the other hand, influential factors are factors other than the basic factors that affect the learning result. In the factor information DB 116, ID information for identification is given to each stored factor, and it is possible to identify each other.

An example of information regarding the factor stored in the factor information DB 116 will be described with reference to FIGS. 7A and 7B. In the present embodiment, an example in which factors are classified into basic factors and influential factors to manage identification information will be described, but the method of classifying factors is not limited to this, and even if the factors are classified into three or more. It may be managed as one.

FIG. 7A is a table showing an example of the correspondence between the basic factor and the type of ID assigned to the basic factor. The basic factor is a factor that allows the learning results to be grouped.

Since learning is performed to acquire a predetermined ability, it can be identified by the acquired ability. Therefore, an ID is assigned to the type of acquired ability and managed. In addition, learning is performed based on the request of the learning requester, and different learning is performed for each learning request. Therefore, the learning result can also be identified by the learning requester (requester ID). Furthermore, all learning is performed based on the learning data, and it is possible to identify the learning result by the target of the learning data. Since the target of the learning data can be specified by the acquisition source of the input data and the acquisition period, IDs (input data ID, acquisition period ID) are also assigned to these factors and managed as basic factors.

Further, even if the acquired abilities are the same (the acquired ability ID is the same), the learning result may change due to the influence of the specifications of the device in which the abilities are used, the usage environment, and the like. Therefore, the device to be used and the target to be used are also given the device ID and the target ID, and are managed as basic factors. The basic factors are not limited to those shown in FIG. 7A, and some of the basic factors in FIG. 7A may be selectively used, or other factors may be included as long as the learning results can be grouped. ..

FIG. 7B is a diagram showing an example of the interrelationship of influential factors. Influencing factors are additionally registered in the factor information DB 116 based on set parameters, conditions, etc., for example, in past learning. As shown in FIG. 7B, the influencing factors preferably have a tree-structured interrelationship. Although not shown in FIG. 7B, an identifiable ID is assigned to each influencing factor. Further, it is preferable that the ID assigned to each influencing factor is assigned so that the hierarchical structure can be grasped.

As shown in FIG. 7B, the highest hierarchy of influencing factors consists, for example, an external interface, learning data, affiliation-related information, learning time, learning goals, learning devices, learning methods, and ability tests. There is.

The external interface is a factor related to the interface between the learning program and the outside, such as the type of input data and the type of output data.

In addition, learning data is a factor related to the data used for learning. In the factor information DB 116, by assigning an ID to the learning data and managing it, the details of the learning data can be concealed and managed.

Affiliation-related information is a factor based on information on affiliation of learning programs and the like. When each of the affiliation-related information has a right relationship, it is preferable that the factor information DB 116 manages the affiliation-related information as a subordinate factor of the affiliation-related information.

The time of learning is a factor based on when the learning took place. In addition, learning goals are factors based on various goals set for each learning.

The learning device is a factor based on the device on which learning is performed. When learning is performed on the cloud, the name and network address of the cloud operating company are used instead of the model of the device. It can be used as a factor of.

The method of learning is a factor based on the way learning takes place. Although not shown in FIG. 7B, the configuration of the neural network, which is a lower factor of the learning method, is a lower factor, for example, a method such as SVM or regression model that has existed before deep learning, or a deep belief network. Deep Boltzmann Machine, Stacked Boltzmann Machine, Autoencoder, Restricted Boltzmann Machine (RBM), Dropout, Sparse Coating, Regularization, Noise Removal Self-Encoder, Activation Function Type, Sigmoid Function, Soft Sign, Soft Plus , Relu, and the type of random number sequence, it is preferable to have a specific learning method. In addition, the factors below the setting of the neural network configuration include hyperparameters, input layer configuration, number of layers, number of hidden layer units, content of each layer (CNN (Convolutional Neural Network), RNN (Recurrent Neural Network)). , Elman network, Jordan network, ESN (Echo state network), RSTM (Long short term memory network), BRNN (Bi-directional RNN), and the like.

Ability testing is a factor based on the test requirements that learning results must meet. The ability test is also managed as one of the influential factors because the learning result is affected by being changed to meet the requirement by performing the ability test.

In a learning method using random numbers, for example, deep learning, there may be a slight difference in the acquired ability each time learning is performed. In such a case, it is preferable to assign and manage the identification ID with the learning start time and learning end time and the ID of the learning device as influential factors.

In addition, among the influential factors, there may be cases where multiple items can be regarded as equivalent. In that case, the types of a plurality of factors may be grouped and the same identification ID may be assigned. Furthermore, the influential factors are not limited to the above. Various influential factors occur depending on the subject and purpose of learning and the environment.

By providing the above-mentioned configuration, the learning result database 11 according to the present embodiment can identify and manage different learning results, whereby the same learning result can be obtained in the learning service providing system 1. Can be determined. Each DB of the learning result database 11 can be used from the learning request receiving device 12, the learning device 13, the learning management device 140, and the like via a network such as a local network. The details of the search process of the learning result database 11 will be described later.

FIG. 8 is a functional block diagram of the learning request receiving device 12 according to the present embodiment. The learning request receiving device 12 has a function of receiving learning request information from the learning request device 22 and transmitting the learning request information to the learning device 13. As shown in FIG. 8, the learning request receiving device 12 includes, for example, a learning request receiving unit 121, a learning data storage unit 123, a learning request content storage unit 124, and a communication unit 125.

The learning request reception unit 121 receives a learning request from the learning request device 22. The learning request reception unit 121 registers the learning request information included in the received learning request in the learning request DB 112. At this time, the learning request receiving unit 121 transmits a notification to the effect that the learning request has been accepted to the learning management device 140 via the communication unit 125. Further, the learning request receiving device 12 can temporarily store the received learning request in the learning data storage unit 123 or the learning request content storage unit 124.

FIG. 9 is a functional block diagram of the learning device 13 in the present embodiment. The learning device 13 has a function of performing learning based on learning request information and acquiring a new ability as a learning result. As shown in FIG. 9, the learning device 13 has a learning control unit 131, a neural network 132, a learning result extraction unit 133, a communication unit 134, and a learning result output unit 135 as functional units. ..

In the learning device 13, the learning control unit 131 can control the neural network 132 and perform learning based on the learning request information. The learning result is extracted by the learning result extraction unit 133, and is output by the learning result output unit 135 via the communication unit 134.

When learning is performed a plurality of times in the learning device 13, a plurality of learning results can be obtained. In each learning, if the requirements that influence the learning result such as learning data, learning program, learning time, and learning goal are different, the abilities obtained by multiple learning results may not be the same. Details will be described later, but in the learning service providing system 1 according to the present embodiment, when there is a possibility that the abilities obtained by a plurality of learning results are not the same, different identification information is given and managed as a variation. be able to.

The learning device 13 can also make a learning request by itself. In this case, the learning device 13 enables autonomous learning.

FIG. 10 is a schematic diagram showing the configuration of the learning management system 14. As shown in FIG. 10, the learning management system 14 includes a learning management device 140, a learning identification information generation device 15, a learning result identification information generation device 16, and a learning result utilization environment information generation device 17. Are connected to each other via the local network.

The learning management device 140 manages the learning result and the result. In the learning service providing system 1 that provides the learning service, it is desirable to individually manage all the learning results in the learning service providing system 1 and the learning results brought about by the learning results. Therefore, inside the learning service providing system 1, all learning results and their results are managed so as to be mutually distinguishable by the learning management device 140. For example, if learning is performed under conditions where any of the influencing factors is different, the learning results may be different. That is, since the learning result is affected differently by changing the composition of the influential factors, many variations occur in the learning result. Moreover, even if the same learning result is used in different devices and environments, the result may be different. The learning management device 140 identifies variations in order to manage learning results and their outcomes.

As shown in FIG. 10, the learning management device 140 includes a learning information acquisition unit 141, a learning control unit 142, a factor information acquisition unit 143, an identification range setting unit 144, an influence determination unit 145, and a communication unit 147. ,have.

The learning information acquisition unit 141 acquires learning request information via the communication unit 147.

The learning control unit 142 selects a learning device 13 for learning and a learning program based on the learning request information, and instructs learning. Further, the learning control unit 142 executes a process of comparing the learning results. In this respect, the learning management device 140 can be said to be a comparison device. Further, the learning control unit 142 generates the first information generated by the learning identification information generation device 15 and the learning result identification information generation device 16 for the learning result that has acquired a predetermined ability by learning. It has a function of giving identification information including the second information. Further, the learning control unit 142 associates the result brought about by the learning result with the identification information including the first information and the second information, and the third information generated by the learning result utilization environment information generation device 17. It has a function of giving information including. That is, the learning control unit 142 provides identification information including first information that can identify learning, second information that can identify the learning result, and third information that can identify the learning outcome with respect to the learning result. It acts as a generator to generate, which allows the outcomes of the learning results to be mutually distinguishable. Here, adding identification information to the learning result is a concept including associating the learning result with the identification information. For example, adding the identification information to the learning result data or identifying the learning result with the storage device. Includes storing information in association with each other. In addition to directly associating the learning result with the identification information, it also includes indirectly associating these with each other. For example, when the learning result is associated with an identifier (for example, a serial number) for identifying the learning result itself and then the identifier and the identification information are associated with each other, the identification information is added to the learning result. included.

The factor information acquisition unit 143 acquires a list of factors that may affect the learning result from the factor information DB 116. For example, the factor information acquisition unit 143 can acquire information corresponding to the tree structure of the influential factors shown in FIG. 7B (hereinafter, also referred to as “list of influential factors”).

The identification range setting unit 144 sets a range (identification range) in which variation identification is required. Identifying the identification range is equivalent to grouping learning results and learning outcomes. It suffices if the learning result and the variation of the learning result can be identified within the range that needs to be identified. That is, when the identification ranges are different, the learning results and learning results may not be mutually distinguishable between the different identification ranges. For example, when the learning requesters are different, it may not be a problem even if the variations cannot be distinguished from each other. Specifically, a certain learning requester only needs to be able to identify within the range requested by himself / herself, and it is not necessary for the variation to be uniquely identified from the learning results and learning results of others. For example, the identification range setting unit 144 can set the identification range using the list of identification ranges.

FIG. 11 is a diagram showing an example of a list of identification ranges. As shown in FIG. 11, the identification range is configured hierarchically, for example, and the lower identification range can be included in the upper identification range. Specifically, the set including all the identification ranges is the identification range of the entire top level (hereinafter, referred to as "first layer"). In the identification range of the first layer, it is necessary that all learning results and learning results can be mutually distinguished. For example, when it is necessary to identify all the learning results and learning results of the learning service provider, the identification range setting unit 144 sets the identification range of the first layer.

In the example of FIG. 11, the identification range of the second layer below the first layer is a set defined by the learning requester and the application field. When the identification range is set for each learning requester (second layer) by the identification range setting unit 144, the learning result and the learning result are the identification information that can be identified for each learning requester. It is generated by the learning result identification information generation device 16 and the learning result utilization environment information generation device 17. Further, the identification range of the third layer below the second layer is, for example, the type of company (company A, company B, company C in the case of FIG. 11) and the type of application field (in the case of FIG. 11, the industrial field, etc.). It can be specified for each commercial field and agricultural field). In this case, when the identification range is set for each company (third layer), identification information that can be identified for each company is generated. Further, for example, when a certain company (Company A) owns a plurality of factories and is learning about different objects for each factory or factory line, the identification range setting unit 144 may be used by the company A. The identification range may be set for each factory (fourth layer) or for each line (fifth layer). In this case, the identification range setting unit 144 may set a lower identification range for another company (company B or company C) based on an element different from that of company A.

Since the identification range has a hierarchical structure, the identification information corresponding to the upper identification range can be excluded from the identification information corresponding to the lower identification range, and thus the capacity of the identification information can be reduced.

The identification range set by the identification range setting unit 144 is not limited to the above example, and can be set arbitrarily. For example, the identification range may be set based on a basic factor or an influential factor. In this case, the list of identification ranges may have a configuration corresponding to the list of influencing factors. For example, it is preferable that the tree structure of influential factors is inherited by the hierarchical structure of the corresponding identification range. Specifically, assuming that the identification range classified by the influential factor "external interface" in FIG. 7B is the identification range of the third layer, the influential factor "input data" positioned below the external interface is, for example, the fourth. It becomes the identification range of the layer. At this time, the identification range setting unit 144 can set the identification range based on which layer of the tree structure of the influential factors shown in FIG. 7B the influential factors included in the learning request information belong to. The identification range setting unit 144 can also be set according to the combination of basic factors included in the learning request information.

Further, the method in which the identification range setting unit 144 sets the identification range is not limited to the method using the list of identification ranges. For example, a set can be handled directly by using a programming language that includes the set as a class.

The influence determination unit 145 determines the degree of influence that the influence factor has on the learning result based on the list of the influence factors acquired by the factor information acquisition unit 143.

For example, the influence determination unit 145 can determine the degree of influence that the basic factor or the influence factor has on the learning result by using the influence determination logic created for each identification range. The influence determination logic is logic that determines the influence of the basic factor and the influence factor on the learning program included in the identification range. When factors (basic factors, influencing factors) that may affect the learning result are given as inputs, the influence determination logic determines whether or not there is an influence on the learning result for each factor. Specifically, the influence determination logic includes factors that influence the learning result (basic factors, influential factors) and conditions given to the factors (for example, learning data, learning requester ID, learning) for each learning program. The degree of influence on the learning result is associated with the degree of influence on the learning result according to the goal, etc.), and the degree of influence is determined based on the correspondence.

In the influence determination logic, for example, in two specific learnings, it may be considered that the factors that completely match the learning results are not affected. Further, it may be considered that the factor that determines that the learning results are close to each other under a predetermined condition has no influence.

The impact determination logic is created, for example, by the creator of the learning program. If it is difficult to create impact determination logic that can be applied to the entire learning program included in a certain identification range, it is also possible to divide the identification range and create impact determination logic for each divided identification range. is there.

The creator of the impact determination logic is not limited to the creator of the learning logic as long as the person knows the details of the learning logic. In order to create the influence judgment logic, the learning logic is based on the contents of the configuration such as whether or not a random number is used inside the learning logic, and the predetermined judgment conditions according to the type and number of input data. In the case of a configuration in which is switched, it is necessary to know the details of the learning logic such as the switching condition and the designated items related to the learning method (neural network configuration, end condition, etc.). Therefore, anyone who knows the details of the learning logic can create the impact judgment logic.

On the other hand, the data given as input to the influence determination logic includes data that cannot be known by the creator of the learning logic. For example, if the learning requester and the creator of the learning logic are different, the creator of the learning logic cannot know the learning data at the time of learning and the influential factors specified by the learning requester. The following are examples of influential factors specified by the learning requester.
・ The model of the device that uses the learning result, the model of the sensor that is used, the number of classifications of the classification ability to be acquired, and the language targeted by the language processing ability

The impact determination unit 145 extracts the above-mentioned information that cannot be known by the creator of the learning program from the learning request information, and provides such information to the impact determination logic that only a person who knows the structure of the learning program can know. Give as input. As a result, when learning is performed, it is possible to determine the influence of basic factors and influential factors on the learning result, and it is possible to manage variations of the learning result. Further, in the case of a learning service, there is some information that can be grasped only by the learning service providing system 1, such as information when a learning request is received, information on a selected learning program, and information on selected learning data. It is preferable that these pieces of information are also input to the impact determination logic by the impact determination unit 145 when the learning is performed.

Further, the influence determination logic may be used for determining the influence of the learning data. In many cases, the difference in learning data affects the learning result, but the learning data is specified at the time of learning, and the creator of the learning program may not be able to specify the learning data. Even in such a case, by making it possible to acquire information on the content of the used learning data, it is possible to create a logic for determining whether or not there is an influence according to the learning data. Specifically, when the learning data preparation device 21 prepares the learning data, the detailed information about the learning data can be recorded by acquiring the information about the learning data to be prepared and recording the identification ID.

The method of determining the influence of the factor on the learning result in the influence determination unit 145 is not limited to the above example. For example, when a predetermined learning program is used as a learning tool, the influence determination unit 145 may determine the presence or absence of an influence based on the change in the usage method or setting in the program. For example, by adding the information regarding the changed setting to the learning data, the influence determination unit 145 can detect the change in the setting. Specifically, when the threshold value of the sensor is changed, the influence determination unit 145 can detect the setting change by adding the information regarding the threshold value after the change in addition to the threshold value before the change. .. Furthermore, setting changes can be detected by adding information such as IDs and tags.

On the other hand, for example, when the learning is performed twice without changing the learning settings or the learning data at all, there is no change in the influential factors other than the two times. However, when the learning program uses random numbers internally, the acquired ability may differ from the learning result performed multiple times under the same conditions. In such a case, the influence determination logic determines that the number of learnings affects the learning result. As a result, the learning identification information generation device 15, the learning result identification information generation device 16, and the learning result utilization environment information generation device 17, which will be described later, can add identification information such that the number of learning times = the first time and the number of learning times = the second time. it can.

Further, the method for determining the presence or absence of influence is not limited to the above, and any method can be used. For example, the person in charge of acquiring learning data may operate the keyboard and mouse (operation unit 211) of the learning data preparation device 21 to input information for identification. Further, based on the information included in the learning request information from the learning request device 22, the difference in the request contents may be grasped to determine the presence or absence of the influence.

Returning to FIG. 10, the continuation of the configuration of the learning management system 14 will be described.
The learning identification information generation device 15 uses identification information (hereinafter, "first") so that learning variations can be mutually identified according to the range in which the learning result is used (for example, the identification range set by the identification range setting unit 144). Information ") is generated. For example, the learning identification information generation device 15 adds the first information to the learning based on the factors that influence the learning result. Specifically, the learning identification information generation device 15 selects the influence determination logic based on the learning program corresponding to the learning request information. Next, the learning identification information generation device 15 inputs the identification range and the list of influencing factors into the selected influence determination logic. The learning identification information generation device 15 generates data that is the source of the first information so that the influence determination logic includes all the influential factors that are determined to affect the learning result. Further, the learning identification information generation device 15 generates the first learning information by converting the data that is the source of the first information by a predetermined method.

The processing of the learning identification information generation device 15 will be described in more detail. The influential factors have a tree structure as described above (see FIG. 8B). The learning identification information generation device 15 deletes from the tree structure a branch that does not include an influential factor determined to be an influential factor that is influenced by the influence determination logic. This process is sequentially executed from the upper layer to the lower layer of the tree structure to create a tree (also referred to as "influence factor explanation information"). The learning identification information generation device 15 can use the data string obtained by sequentially listing the IDs of the influencing factors included in the influencing factor explanation information from the upper layer as the data that is the source of the first information. The data string at this time may be, for example, a character string, a binary format, an XML format, or an expression method such as JSON (Javascript (registered trademark) Object Notification). It is preferable that the data string obtained in this procedure includes information on the degree of influence of a predetermined influencing factor on the learning result. In this case, for example, the impact determination unit 145 describes the impact degree in the tree structure according to the output of the impact determination logic, and the learning identification information generation device 15 also lists the impact degree in the data string. Is preferable.

The learning identification information generation device 15 converts the data string by a predetermined method such as listing, compression, or encryption, and generates the first information. The predetermined method is a method in which different first information is generated if the original data strings are different at this time.

As an example, the first information generated at this time is a predetermined character string such as "qijsudi48fuhu" or "nvfsuke3fjn45ip". Note that this character string may be randomly generated. In this case, the learning identification information generation device 15 can prevent the same first information from being assigned to different learning results by managing the generated character string and the original data string in a list. However, the example of the first information is not limited to the character string, and any method that can reduce the amount of information from the data string can be taken.

The learning identification information generation device 15 can generate the first information by any method other than the above. For example, it is possible to create a program in an object-oriented language, define factor information in a hierarchical class, and serialize the result of marking an instance of the class and register it in the database.

For example, in a learning method using random numbers such as deep learning, the acquired ability may be slightly different each time the learning is performed as described above. In such a case, the learning identification information generation device 15 adds information (derivative identification information) indicating the number of times as in the first learning and the second learning to the first information generated by the above method. May be good. Further, when a large number of PCs are used as the learning device 13, the first information can be generated by combining the PC names used (for example, "C2145.33rd learning" or the like).

The first information can be hierarchically configured according to the basic factors and the influential factors.

The learning result identification information generation device 16 generates identification information (hereinafter, also referred to as “second information”) for identifying the learning result which is the result of learning. The learning results may differ even if the influencing factors are the same (that is, even if the learning with the same first information is performed). In such a case, even if the influencing factors are the same, the outcome (learning outcome) using the learning result may be different. Since the user who uses the learning result is interested in the learning result, it is preferable to identify it as another learning result when the learning result may be different even if the influencing factors are the same. The learning result identification information generation device 16 assigns second information to the learning result and identifies the learning result so that the learning result can be identified even in such a case.

The second information is, for example, information for identifying learning (information indicating what kind of learning was performed, for example, first information) and information regarding the learning implementation environment (where and what device is used for learning). Information about the device that uses the learning result (information that shows where and how it is used), information about the ability that the learning result acquired (information that shows what kind of ability was acquired) Information) and the like may be included.

Information that identifies such a learning result is preferably generated based on a learning request. For example, in a learning request, when learning related to control of robots having different sizes is requested, it is preferable that the information for identifying the learning result includes information indicating the size of the robot in which the learning result is used.

The second information may further include information capable of identifying the content of the learning result. That is, the second information can include information that can determine whether or not the learning result matches the content requested by the client. For example, the learning result identification information generation device 16 refers to the learning request DB 112 based on the second information, and provides detailed information (for example, learning) regarding individual influential factors necessary for identifying the content of the learning result. It is possible to search the data identifier, learning target value, etc.) and notify the requester together with the first information and the second information. The client can identify the content of the learning result by taking into consideration the detailed information notified.

The learning result usage environment information generation device 17 generates usage identification information (third information) which is information on the result of the learning result and the usage status. When the learning result is used in the learning result utilization device 23, it is preferable to manage matters such as duplication of the learning result, transfer of the duplication, and usage status of the learning result, in addition to information on the result of the learning result. Therefore, it is preferable that the learning result utilization environment information generation device 17 stores the following information on the results and usage status in the learning result utilization information DB 113 in association with the second information.
・ Ability acquired by learning ・ Users of learning results ・ Devices that use learning results ・ Places where learning results are used ・ Purpose of use of learning results ・ Targets of use of learning results ・ Combination of learning results ・ Division of learning results・ Usage history of learning results

In the learning result utilization information DB 113, the above information is converted into recordable data and recorded in time series. At this time, it is preferable to generate and record the usage identification information for identifying the result and the usage status by the same method as the above-mentioned learning and learning result identification information.

<3. Data structure>
An example of the configuration of information related to learning stored in the learning result utilization information DB 113 will be described with reference to FIG. As shown in FIG. 12, the learning result utilization information DB 113 stores information related to learning in a tree structure. Specifically, the learning identifier (first information) ID1 is associated with a plurality of learning result identifiers (second information) ID2, and each of the plurality of learning result identifiers ID2 is associated with a result or The usage identification information (third information) regarding the usage status is linked. In this way, in the learning result utilization information DB 113, the storage capacity can be reduced by storing the information related to learning in a tree structure.

<4. Search process>
The search function of the learning result database 11 will be described. As an example, the following persons are assumed as users of the learning result database 11.
・ Creator of learning framework ・ Creator of learning program ・ Manager of learning ・ Manager of learning data ・ Manager of learning result ・ Developer of learning result utilization device 23 ・ Administrator of learning result utilization device 23 ・ Learning User of result utilization device 23

The above users may use the learning result database 11 in different usage modes. For example, the creator of a learning framework uses it to associate detailed learning information with learning results. It is also used by the creator of the learning program to record the details of the learning program. The learning manager also uses it to record detailed information for managing learning. In addition, the learning data manager uses it to associate and manage detailed information on where and how the learning data is used. In addition, the manager of the learning result is used to manage the learning result by using detailed information on how the learning result was obtained.

Further, the developer of the learning result utilization device 23 may incorporate the function of the learning result database 11 into the database function for using the learning result provided by the learning result utilization device 23. In addition, the manager of the learning result utilization device 23 may manage the contents of the learning result incorporated in the learning result utilization device 23 in detail. Further, the user of the learning result utilization device 23 asks whether the learning result to be used is obtained as a result of similar learning and whether the learning result is useful or not. It may be grasped and used.

Since the learning result database 11 according to the present embodiment manages information related to learning by a tree structure that can be configured in an arbitrary shape, it can correspond to the above-mentioned various usage forms.

For example, the user can search the learning result database 11 by combining influential factors. Further, for example, the user can search by arbitrarily combining the above-mentioned first information, second information, information on the usage environment, and the like. The learning result database 11 can output the search results as a list or a set for such a search. Further, the learning result database 11 may be configured to perform various logical operations such as sum, product, and negation on the list or set of search results and output them. Further, the learning result database 11 can output detailed information in a hierarchical manner for each individual matter. Further, the learning result database 11 may be configured to output a summary of search results and statistical data based on the search results. As a result, when searching for learning results for robot control, for example, the user of the learning result database 11 can search for learning results showing high results in an environment similar to that of his own factory, or the learning results can be searched. You can search for what kind of learning it is.

<5. Use of identification information>
Next, an example of using the identification information of the learning result will be described.
By using the identification information of the learning result, the client can mutually identify the variation of the acquired ability. That is, the learning results can be compared, the degree of suitability for the purpose can be compared, and an appropriate variation can be selected for each object.

For example, the learning result comparison process is performed as follows. The learning management system 14 searches the learning result database 11 based on the conditions received from the requester (operations expected by the requester, etc.), and creates a list of learning results to be compared. Next, the learning target included in the list of learning results is specified, and the comparison program for the specified learning target is specified. Then, the learning results are compared with each other using the specified comparison program, and the comparison results are output to the requester in a tabular format or the like. In this way, the learning service providing system 1 not only simply stores individual learning and learning results, but also performs comparison processing of learning results based on identification information, which is more desirable learning for the expectations of the client. You can search for the result and output it.

In addition, by being able to identify the learning result, the correspondence between the device using the learning result and the learning result being used, the correspondence between the learning result being used and the target, and the ability acquired by the learning result. It is possible to identify and manage the affiliation of the affiliation, the affiliation of the factor that brought about the learning result, etc. Furthermore, by being able to identify the learning result, it becomes possible to manage the usage history of the acquired ability, the place where the acquired ability exists, the output of the acquired ability, and the like.

The identification information generated by the specific use of the present invention and a specific example of the method of using the identification information will be described. In the following usage example, in order to facilitate understanding, the learning identification information generation device 15 and the learning result identification information generation device are used for the above-mentioned data string obtained from the list of influential factors using the influence determination logic. An example is shown in which the 16 and the learning result utilization environment information generation device 17 do not perform encryption or the like. Even in this case, by using a different encryption key for each user, it is possible to prevent others from knowing the contents of the identification information. In addition, the examples given below can be arbitrarily combined.

(Usage example A): Example in which one influencing factor is different ・ Usage example A-1: Example in which the target is different When the target is different in the same learning requester, different learning result identification information is given to each learning result. To. For example, identification information such as "target device = SX9100.jdsjhfsd" or "target device = SX9200.djsfhsjfk" is given.
On the other hand, when the target sensor or the like is different in the same target device, identification information such as "target sensor = group a74.jhsjuo" or "target sensor = group a90.jsdfhjkg" is given (however, the target device). Is common to the SX9100, and the identification range is an example in the case of the SX9100).

-Usage example A-2: Example when learning multiple times (however, when it is necessary to identify the target device)
In this case, "target device = SX9100.1th learning", "target device = SX9100.2th learning", "target device = SX9100. Learning end time = 20160707123521", "target device = SX9100. Learning". Identification information such as "end time = 20160708114335" is added.

-Usage example A-3: Example of different learning data In this case, "target device = SX9100. Learning data = 000001 to 019999", "target device = SX9100. Learning data = 02000 to 029999", "target device = SX9100". Identification information such as "learning data = April 2015" and "target device = SX9100. Learning data = May 2015" is added.

-Usage example A-4: Example of different learning programs In this case, identification information such as "learning program = DQ2939. Target device = SX9100" or "learning program = DQ3200. Target device = SX9100" is given.

-Usage example A-5: Example of different requesters For example, if the requesting department is different, "Target device = SX9100. Requesting department = 1st construction section" or "Target device = SX9100. Requesting department = 2nd construction section" Identification information such as is given.

(Usage example B): Examples in which multiple influential factors are different For example, in the example where the requester, which is the basic factor, and the target and the learning data are different, "Target device = SX9100. Request department = 1st construction section. Learning data = 2015. Identification information such as "April" or "Target device = SX9200. Request department = 1st technical section. Learning data = July 2016" is added.

(Usage example C): Example in which the range in which the identification information is used is different For example, the identification information in the learning service provider is "learning requester = company A. Target device = SX9100. Learning program DQ2939. Learning data = 02000- It is "029999" and is distinguished between different clients such as "learning requester = company B. Target device = JQR939400. Learning program DQ2939. Learning data = for July 2016". In this case, the following identification information can be obtained by excluding the identification information about the learning requester from the identification information provided to the user.
-Identification information provided to the learning requester A: "Target device = SX9100. Learning program DQ2939. Learning data = 02000 to 029999"
-Identification information provided to the learning requester B: "Target device = JQR939400. Learning program DQ2939. Learning data = July 2016"

<6. System flow>
Next, the processing flow of the system according to the present embodiment will be described with reference to FIGS. 13 to 15. FIG. 13 is a flowchart showing a processing flow in the learning request system 2.

First, the client prepares the learning data by using the learning data preparation device 21 (S101). Next, the requester inputs the learning request information using the learning request device 22 (S102). Further, the client requests learning from the learning request device 22 to the learning service providing system 1 (S103). When learning is performed in the learning service providing system 1, the learning requesting device 22 receives the learning result from the learning service providing system 1 (S104), and stores the received learning result (S105).

The learning request device 22 selects an appropriate learning result from the stored learning results when the learning result is used in the learning result utilization device 23 (S106). At this time, the learning request device 22 refers to the identification information of the learning result and determines whether or not the selected learning result is suitable for the purpose of use (S108). When the purpose of use is met (S108: Yes), the learning result is incorporated into the learning result utilization device 23 (S109), and the learning result is used in the learning result utilization device 23 (S110).

On the other hand, when the learning result selected in S108 does not match the purpose of use (S108: No), a nonconformity notification is given (S119), and learning is performed again by changing the conditions as necessary (S120).

Next, the processing flow of the learning service providing system 1 will be described. FIG. 14 is a flowchart showing a processing flow in the learning service providing system 1.

First, the learning request receiving device 12 receives the learning request information from the learning request device 22 and receives the learning request (S201). Next, the identification range setting unit 144 in the learning management device 140 acquires the information necessary for setting the identification range (S202). The identification range setting unit 144 sets the identification range from the acquired information and the learning request information (S203).

The learning control unit 142 selects the learning device 13 and the learning program based on the learning request information and the set identification range (S204), notifies the selected learning device 13 of the learning request information (S205), and learns. (S206).

The influence determination unit 145 refers to the factor information DB 116 and extracts a list of influence factors. Further, the influence determination unit 145 acquires the influence determination logic corresponding to the learning program from the learning device 13 (S207), and determines the degree of influence that each factor has on the learning result by using the acquired influence determination logic (S208). ). The influence determination unit 145 sets the determination result for each influence factor described in the list of influence factors (S209).

Next, the learning identification information generation device 15, the learning result identification information generation device 16, and the learning result utilization environment information generation device 17 exclude the influential factors set to have no influence from the list of influential factors (S210). The learning result identification information is generated using the list of influential factors after the removal (S211), and the learning control unit 142 registers the generated identification information and the learning result in the version management DB 114 in association with each other (S212). ..

As described above, according to the learning service providing system 1 according to the present embodiment, it is possible to appropriately manage the variation of the ability acquired by the machine.

FIG. 15 is a flowchart of a process for comparing learning results. As a preparation before performing the learning result comparison process, it is assumed that a learning program or a program for comparing the ability of the learning result for each learning target is registered in the learning result database 11. The comparison program is created by the creator of the learning program, the learning manager, and the like. Further, it is assumed that the above-mentioned identification information is added to the learning result obtained by machine learning, associated with the influential factor, and registered in the learning result database 11.

The learning management system 14 starts the learning result comparison process in response to a predetermined trigger. For example, when the learning management system 14 receives an instruction from the user via an input device or the like, the comparison process may be started. Further, the learning management system 14 may start the comparison process when a new learning result is obtained.

First, the learning control unit 142 of the learning management system 14 extracts a plurality of learning results to be compared with each other from the learning result database 11 and creates a list of a plurality of learning results to be compared (S301). .. At this time, the learning management system 14 may extract a plurality of learning results directly selected by the user via an input device or the like from the learning result database 11. By designating the abilities and actions that the user expects from the learning results, the learning results that have acquired the specified abilities and actions may be extracted from the learning result database 11. In addition, the learning management system 14 may extract from the learning result database 11 the learning results that have acquired the abilities and actions obtained as a result of learning based on the learning request requested by the user. When a new learning result is obtained, a learning result having the same ability or action as the ability or action obtained by the learning result may be extracted from the learning result database 11.

Next, the learning target included in the extracted learning result list is specified (S302). For example, the learning target can be specified by referring to the identification information given to the extracted plurality of learning results. Then, the learning management system 14 identifies a comparison program for the specified learning target and acquires it from the learning result database 11 (S303).

The learning management system 14 executes the acquired comparison program and compares the plurality of learning results extracted in step S301 with each other (S304). In the comparison of learning results, a plurality of learning results to be compared may be relatively compared based on a predetermined comparison standard, or a comparison based on an absolute evaluation may be performed with respect to a predetermined comparison standard. Good. For example, the comparison program may determine whether or not each of the plurality of learning results includes a specific ability or action expected by the user himself / herself. The comparison program may determine which learning results are better and how better than the user's expectations. In addition, the comparison program may extract more desirable learning results from the plurality of learning results with respect to the expectations of the user.

For example, when a user wants to compare learning results that have acquired the ability to classify agricultural products, the user first specifies to classify the agricultural products as the ability expected from the learning results. In response to the user's designation, the learning control unit 142 extracts a plurality of learning results that have acquired the ability to classify the grades of agricultural products from the learning result database 11 and refers to the identification information given to each learning result. Then, the learning target is identified as an agricultural product. Then, the learning management system 14 acquires a comparison program for agricultural products from the learning result database 11, compares and evaluates the learning results. At this time, the comparison program compares the learning results according to, for example, one or more evaluation criteria as follows.
・ Number of neurons and number of layers included in the learning result ・ Size of learning program ・ Time required for learning ・ Power required for learning ・ Cost (charge) required for learning
・ Learning quality (for example, the number of learning data used for learning and the number of repetitions of learning)
-Time from input to output-Goodness of fit to the evaluation function-Correct answer rate (for example, when predicting the waveform of a signal, the degree of agreement with it-Ratio of output unprocessable output-Stability of output result (for example, 10 evaluation tests) And the degree of variation at that time)
-Number of applicable devices The comparison program may compare and evaluate each learning result by referring to the identification information given to each learning result. Further, by applying the sample data to each learning result, each learning result may be compared and evaluated. For example, a comparison program can compare and evaluate each learning result by applying sample data in which an image of an agricultural product and its grade are associated with each learning result.

The learning management system 14 outputs the result of mutual comparison of the learning results by using the comparison program in a format such as a tabular format (S305). The learning management system 14 can output the learning result to a terminal device such as a PC or a smartphone of the user who requested the comparison processing.

<7. Example of inspection machine>
An example in which the comparison device according to the present embodiment is applied to an inspection machine will be described. For example, the manufacturer Z of an electronic circuit board uses an inspection machine to sort out non-defective products and defective products. In this example, the manufacturer Z is a learning requester, and when the inspection machine acquires the ability to sort out good and defective products by learning, it has acquired the ability to sort out good and defective products. A case where a plurality of learning results are compared and a learning result selected based on a predetermined evaluation is applied to an inspection machine will be described.

First, the configuration of the inspection machine a will be described with reference to FIG. FIG. 16 is a block diagram showing an example of the configuration of the inspection machine a. As shown in FIG. 16, the inspection machine a includes a storage unit A1, an input unit A2, a discrimination unit A3, and an output unit A4. The storage unit A1 stores, for example, the learning result of learning performed by the learning service providing system 1. The input unit A2 has a function of acquiring information to be inspected, and is composed of, for example, a sensor such as a camera or a microphone. The discrimination unit A3 has a function of discriminating a predetermined inspection target such as a non-defective product or a defective product based on the information (for example, an image) of the inspection target input from the input unit A2. Further, the output unit A4 has a function of outputting the determination result of the determination unit A3. Note that outputting the judgment result here means not only presenting the judgment result to the user, such as displaying the judgment result on a display device or the like, but also an inspection machine for selecting a predetermined inspection target from other inspection targets. It includes the operation of.

Next, the flow of processing in this embodiment will be described. First, the manufacturer Z requests the learning request receiving device 12 of the learning service provider A to deliver the learning model operating on the inspection machine a owned by the manufacturer Z from the learning request device 22. .. Specifically, the manufacturer Z determined a plurality of images of the electronic circuit board captured by the camera provided in the inspection machine a at the time of the learning request, and the manufacturer Z determined a good product or a defective product for each image. Information including data that combines the results is provided as learning request information to the learning service provider A to request learning.

At this time, the learning request DB 112 stores, for example, the following information as information such as the ability to acquire, the learning requester, the target of learning data, and the acquisition period.
-The ability to acquire is to distinguish between good and defective circuit board mounting, and the correct answer rate based on the learning results is 90% or more.-The learning requester is the circuit board manufacturer A. The target of the training data is the captured image of the circuit board acquired by the inspection machine a of the manufacturer A of the circuit board, and the result of the manufacturer Z determining a non-defective product or a defective product for each captured image. The acquisition period is, for example, from June 1, 2016 to June 15, 2016.

Further, in addition to these request information, the learning service provider A is provided by the manufacturer Z after adding information such as what kind of algorithm the learning service provider A uses for learning such as deep learning. The image is used as learning data, and the determination result of a good product or a defective product is used as training data, and learning is performed by the own learning device 13.

For example, the learning device 13 uses the learning data of the manufacturer Z as an input to perform deep learning so that a good product or a defective product can be classified. After the deep learning is completed, for example, the learning service provider A acquires the configuration information of the learning model such as the number of layers of the neural network 132 in the learning device 13 and the weight of each node, or the trained neural network 132 itself. And deliver to manufacturer Z.

At this time, the learning service provider A generates the first information by using the learning management device 140 and the learning identification information generation device 15. Specifically, the influence determination unit 145 of the learning management device 140 extracts the influence factor by the influence factor determination logic, and the learning identification information generation device 15 generates the first information based on the influence factor. In this case, in addition to the above-mentioned learning request information, the influence factor is determined including the information such as the learning algorithm provided by the learning service provider A side, and it can be the first information for identifying the learning. Further, for example, when the learning service provider A has a plurality of learning devices 13, it learns whether the learning device A has learned or the learning device B has learned among the plurality of learning devices 13. Second information that can identify the result may be generated by the learning result information generation device 16. Further, as the third information, even if the learning result utilization environment information generation device 17 generates information for identifying whether the learning result is used by the inspection machine A of the manufacturer A or the inspection machine B. Good.

When the inspection machine of the manufacturer Z acquires the learning result, another learning result having the same ability and operation as the acquired learning result is extracted from the learning result database 11. The inspection machine compares a plurality of learning results obtained in this way, outputs a comparison result, and outputs one learning result selected based on a predetermined evaluation standard from the output comparison results to the inspection machine. Apply. In this way, the inspection machine can select and apply the learning result having a desired ability or motion from the plurality of learning results based on a predetermined evaluation.

The embodiment of the present invention has been described above. It should be noted that the present embodiment is for facilitating the understanding of the present invention, and is not for limiting and interpreting the present invention. Further, the present invention can be changed or improved without deviating from the gist thereof. For example, each step in the above-mentioned processing flow can omit a part of each step within a range that does not cause a contradiction in the processing contents, or can arbitrarily change the order of each processing step or execute them in parallel.

In the above-described embodiment, an example in which the system according to the present invention is used to manage the ability acquired by the machine by AI technology such as deep learning has been described, but the present invention is not limited to this and covers a wide range of fields. Can be applied. For example, it can be applied to distinguish between good and bad products, various industrial fields such as food, mechanical parts, chemical products and chemicals, fishery field, agriculture field, forestry field, service industry, medical and health field. Further, the present invention may be applied to a case where AI technology is applied to a product in the embedded field, a system utilizing IT technology such as a social system, big data analysis, a classification function in a wide range of control devices, and the like.

In the present specification, the "part" or "means" does not simply mean a physical configuration, but also includes a case where the function of the "part" is realized by software. Further, even if the function of one "part" or "means" or device is realized by two or more physical configurations or devices, the function of two or more "parts" or "means" or device is one. It may be realized by physical means or equipment.

In addition, some or all of the above embodiments may be described as in the following appendix, but are not limited to the following.
(Appendix 1)
With at least one hardware processor
The hardware processor
Connected to a database that stores the learning results obtained by performing predetermined learning by machine learning,
Multiple learning results are extracted from the database and
Comparing the abilities acquired by each of the extracted learning results,
A comparison system characterized by outputting the result of the comparison.
(Appendix 2)
By at least one or more hardware processors
A step of extracting multiple learning results from a database that stores the learning results obtained by performing a predetermined learning by machine learning, and
A step of comparing the abilities acquired by each of the plurality of extracted learning results, and
The step to output the result of the comparison and
How to compare to perform.

1 Learning service provision system, 2 Learning request system, 11 Learning result database, 12 Learning request reception device, 13 Learning device, 14 Learning management system, 15 Learning identification information generation device, 16 Learning result identification information generation device, 17 Learning result utilization Environmental information generator, 21 Learning data preparation device, 22 Learning request device, 23 Learning result utilization device, 24 Learning data input device, 121 Learning request reception unit, 123 Learning data storage unit, 124 Learning request content storage unit, 125 Communication unit , 131 learning control unit, 132 neural network, 133 learning result extraction unit, 134 communication unit, 135 learning result output unit, 140 learning management device, 141 learning information acquisition unit, 142 learning control unit, 143 factor information acquisition unit, 144 identification Range setting unit, 145 Impact judgment unit, 147 Communication unit, 211 Operation unit, 212 Learning data acquisition unit, 213 Learning data storage unit, 214 Data acquisition control unit, 216 Communication unit, 221 Learning request unit, 222 Learning request content storage unit , 223 Learning data storage unit, 224 communication unit, 231 learning result input unit, 232 neural network setting unit, 233 neural network, 234 control unit, 235 input unit, 236 communication unit, 237 data acquisition unit, 238 output unit

Claims (6)

It is a comparison device connected to a database that stores the learning results obtained by performing predetermined learning by machine learning.
An extraction unit that extracts a plurality of learning results from the database,
A comparison unit that compares the abilities acquired by each of the plurality of learning results extracted by the extraction unit, and a comparison unit.
An output unit that outputs the result of comparison by the comparison unit and
Equipped with a,
The comparison unit is a comparison device that performs the comparison using a program specified based on a plurality of learning results extracted by the extraction unit. Each of the learning results stored in the database is given identification information that can distinguish the learning results from each other.
The comparison unit performs the comparison based on the identification information given to the learning result.
The comparison device according to claim 1. The identification information can mutually identify a plurality of learning results extracted by the extraction unit within a specific identification range.
The comparison device according to claim 2. An inspection machine provided with the comparison device according to any one of claims 1 to 3 , to which one learning result selected from the comparison results output by the comparison device is applied. It is a method of comparing learning results executed by a computer connected to a database that has a control unit and stores learning results obtained by performing predetermined learning by machine learning .
Steps to extract multiple learning results from the database,
A step of comparing the abilities acquired by each of the plurality of extracted learning results, and
The step to output the result of the comparison and
Including
The comparison step is a comparison method in which the comparison is performed using a program specified based on a plurality of learning results extracted in the extraction step . A computer connected to a database that stores the learning results obtained by performing predetermined learning by machine learning,
A means for extracting a plurality of learning results from the database,
A means for comparing the abilities acquired by each of the plurality of extracted learning results, and a means for outputting the results of the comparison.
To function as,
The means for comparison is a program that performs the comparison using a program specified based on a plurality of learning results extracted by the means for extracting .