JP2018147261A - Model integration device, model integration system, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a model integration device, a model integration system, a method, and a program that realize reflection and integration of results of individual learning on a new model without holding sample data used for individual learning by comparing an individual model and a new common model and relearning them in the individual model derived from a common model.SOLUTION: A model integration device 10 for integrating a plurality of neural network models includes: an output acquisition unit 12 that acquires outputs from one learned common model that has accepted the same input, an individual model that has advanced learning based on the common model, and a new common model that has advanced learning based on the common model, as a first output, a second output, and a third output, respectively; and a selector 14 that selects, among the acquired outputs, one of them and outputs in accordance with a criterion for giving priority to the second output based on at least accuracy of the first output and accuracy of the second output.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a model integration device, a model integration system, a method, and a program for integrating a plurality of neural network models.

  In recent years, deep learning using a neural network has made it possible to understand advanced contexts such as a person's movement from an image that has been difficult until now with mechanical judgment. However, in this image recognition by deep learning, features can be extracted automatically, unlike conventional recognition technology based on manually designed feature points, but a large number of sample images are used to improve accuracy. Learning and creating a model.

  In addition, since the accuracy generally increases as the number of samples increases, it is important to construct a system that collects samples, updates the model, and continues to increase accuracy after modeling once. Is important. For example, instead of being able to use a photo storage service for free, the data is used as a sample for learning (eg https://www.google.com/intl/en/policies/terms/) In order to spread models and technologies that can be used for construction and in-house systems, learned models have been made public and open sourced (Non-Patent Document 1).

Abadi, Martin, et al. "Tensorflow: Large-scale machine learning on heterogeneous distributed systems." ArXiv preprint arXiv: 1603.04467 (2016). R. Girshick, J. Donahue, T. Darrell, U. C. Berkeley, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation.In Proc.IEEE CVPR, 2014. Jeffrey Dean, Gregory S. Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Quoc V. Le, Mark Z. Mao, Marc'Aurelio Ranzato, Andrew Senior, Paul Tucker, 17 Ke Yang, and Andrew Y. Ng. Large scale distributed deep networks. In NIPS, 2012.

  The accuracy increases by collecting a large number of samples as described above, but the judgment content approaches an average answer determined by a so-called majority vote. On the other hand, personalization is also required in deep learning, as each family has its own culture, and it is possible to make judgments that are more appropriate for each family by learning using individual samples from each family. It becomes like this. If the former is called a common model and the latter is called an individual model, the individual model gives high accuracy in a part of the area, but the common model is superior to the others.

  A method called Fine Tuning (Non-Patent Document 2), in which re-learning is performed based on the base model, has been established in order to create a model that is individualized for a part, and individualized to each household based on the common model Can learn. However, as described above, the common model serving as the base is subsequently learned and evolved to a higher accuracy. Therefore, it can be considered that the individualized model is updated based on the model. However, if the learned individual model is replaced with a new common model and learning is performed again, the contents learned at each user and at home disappear, and a gap in the user's feeling of use occurs.

  Therefore, it becomes necessary to integrate new common models and individual models without gaps. It is difficult to integrate the parameters of each model with a calculation formula because it loses the meaning of the learning result, and the neural network itself of the common model may have changed. On the other hand, it is possible to give the sample data used for learning the individual model again to the new common model and perform Fine Tuning again, but since these sample data become enormous as personalization progresses, keep them constantly. It is difficult to leave. There is also a parallel learning method (Non-Patent Document 3) in which learning is performed in parallel with multiple devices, but these are methods in which a single learning model is generated by task distribution, learning individually, and different network models You cannot integrate a trained model with

  There is a learned common model that is regularly updated, and an individual model derived therefrom needs to be integrated with the updated common model. Until now, it was necessary to re-learn using sample data used for learning of individual models based on a common model. However, as learning progresses, the amount of sample data increases and it becomes difficult to keep it for integration.

  The present invention has been made in view of such circumstances. In an individual model derived from a common model, sample data used for individual learning is obtained by comparing and re-learning the individual model and a new common model. It is an object of the present invention to provide a model integration device, a model integration system, a method, and a program that realizes reflecting and integrating the results of individual learning into a new model without maintaining the above.

  (1) In order to achieve the above object, the present invention has taken the following measures. That is, the model integration apparatus of the present invention is a model integration apparatus that integrates a plurality of neural network models, and has advanced learning based on one common model that has received the same input and the common model. An output acquisition unit that acquires an output from the individual model and a new common model that has been learned based on the common model as a first output, a second output, and a third output, respectively; A selector that selects and outputs one of the outputs according to a criterion that prioritizes the second output based on at least the accuracy of the first output and the accuracy of the second output; Prepare.

  Thereby, in the individual model derived from the learned common model, it can be integrated with a new common model without losing the individually learned result.

  (2) Further, the model integration device of the present invention includes a model determination unit that outputs a determination result with respect to an input based on a new common model learned based on the common model, and the selected one A re-learning unit that re-learns the new common model based on the output, and when the re-learning is performed on the new common model, the model determination unit is configured to re-learn the common model And the output acquisition unit acquires the re-learned output of the common model as a third output.

  As a result, even if there is no data for generating the individual model, a new individual model based on the new common model can be generated gradually by using the determination result for the new data. It is also possible to incorporate parts with improved accuracy in the new common model.

  (3) Further, the model integration device of the present invention determines whether or not there is a third common model that has been learned based on the new common model, and if there is the third common model, the third common model An update unit that takes in a common model and replaces it with the new common model is provided, and when the third common model is taken in, the model determination unit determines a determination result for an input based on the third common model. The output acquisition unit receives the same input, and outputs the new common model, the re-learned model, and the third common model as the first output and the second common model, respectively. The re-learning unit performs re-learning on the third common model based on the selected one output.

  Thereby, a new individual model can be generated based on the third common model in which the accuracy is further improved with respect to the new common model. In addition, it is possible to incorporate a portion whose accuracy has been improved by the third common model.

  (4) In the model integration device of the present invention, the re-learning unit stops re-learning when the second output and the third output match at a rate equal to or greater than a predetermined threshold. To do.

  As a result, the re-learned model is not repeatedly re-learned, and unnecessary processing is not performed.

  (5) Further, in the model integration device of the present invention, the output acquisition unit further acquires outputs from one or more other common models different from the common model that have received the same input, and the selector Selecting any one of all the outputs acquired by the output acquisition unit according to a criterion for prioritizing the second output based on at least the accuracy of the first output and the accuracy of the second output And output.

  Thereby, in the individual model derived from the learned common model, the output result of the new common model or other common model can be used without losing the individually learned result.

  (6) The model integration system of the present invention is a model integration system that integrates a plurality of neural network models, manages the common model, and advances the learning to improve the accuracy, and (3 The common model server periodically publishes a new common model that has been learned, and the model integration device periodically fetches the new common model.

  Thereby, a new individual model based on a new common model published by the server can be periodically generated. The common model managed by the common model server includes various common models such as a learned common model, a new common model, a third common model, and other common models.

  (7) Further, the model integration method of the present invention is a model integration method for integrating a plurality of neural network models, one learned common model that receives the same input, and learning based on the common model Obtaining the outputs from the individual model that has been advanced and the new common model that has been learned based on the common model as the first output, the second output, and the third output, respectively, And selecting and outputting any one of the first output according to a criterion that prioritizes the second output based on the accuracy of the first output and the accuracy of the second output.

  Thereby, in the individual model derived from the learned common model, it can be integrated with a new common model without losing the individually learned result.

  (8) The model integration program according to the present invention is a model integration program that integrates a plurality of neural network models, and receives a single input common model that has received the same input, and learning based on the common model. And a process of acquiring outputs from the individual model that has been advanced and the new common model that has been learned based on the common model as the first output, the second output, and the third output, respectively, and the respective outputs A process of selecting and outputting any one of the first output in accordance with a criterion that prioritizes the second output based on at least the accuracy of the first output and the accuracy of the second output. Have the computer execute the process.

  Thereby, in the individual model derived from the learned common model, it can be integrated with a new common model without losing the individually learned result.

  According to the present invention, an individual model derived from a learned common model can be integrated with a new common model without losing the individually learned result.

It is a conceptual diagram which shows the utilization structure of a model. It is a conceptual diagram which shows the time course of development of a model. It is a conceptual diagram which shows the time course of development of a model. It is a block diagram which shows an example of a structure of the model integration apparatus which concerns on 1st Embodiment. It is a table | surface which shows an example of the information output as a determination result with respect to input by the model determination function. It is a block diagram which shows an example of a structure of the model integration apparatus which concerns on 1st Embodiment. It is a block diagram which shows an example of a structure of the model integration apparatus which concerns on 1st Embodiment. It is a flowchart which shows the operation | movement which integrates an individual model and a new common model. It is a block diagram which shows an example of a structure of the model integration apparatus which concerns on 2nd Embodiment. It is a block diagram which shows an example of a structure of the model integration apparatus which concerns on 2nd Embodiment. (A) is a flowchart which shows operation | movement until it produces | generates an individual model based on a common model. (B) is a flowchart showing an operation from generation of an individual model to integration of the individual model and a new common model and generation of a new individual model based on the new common model. It is a block diagram which shows an example of a structure of the model integration apparatus and model integration system which concern on 3rd Embodiment. It is a block diagram which shows an example of a structure of the model integration apparatus and model integration system which concern on 3rd Embodiment after a model is replaced. It is a flowchart which shows the operation | movement until it replaces a model.

  The definitions of terms in the present application will be described below. Except as otherwise noted, it shall be interpreted according to the following definitions.

  (1) The common model is a neural network model in which data collected from an unspecified number of people and nodes is learned as sample data. The learned common model is a common model learned to the extent that it can be used normally.

  (2) The individual model is a neural network model obtained by further learning based on a common model using data collected from a specific small number of people or nodes (specific individuals, homes, companies, etc.) as sample data. Due to the bias of the sample data, an output with higher accuracy than that of the base common model can be obtained for an input in a part of the region.

  (3) Learning is a process of finding an optimal solution of a problem, and approaches a better solution by trial and error and feedback of results. Neural network models usually improve accuracy by learning.

  (4) Common learning is to learn data collected from an unspecified number of people or nodes as sample data in order to obtain a common model or improve the accuracy of the common model.

  (5) With individual learning (individual learning), in order to obtain an individual model based on a common model or to improve the accuracy of an individual model, data collected from a specific small number of people or nodes is used as sample data. To learn. Re-learning is used in the same meaning. Moreover, it includes performing automatically by using the output of the model integration device, and performing by inputting user feedback.

  Next, embodiments of the present invention will be described with reference to the drawings. In order to facilitate understanding of the description, the same reference numerals are given to the same components in the respective drawings, and duplicate descriptions are omitted.

(Model usage structure)
FIG. 1 shows a use configuration of a model assumed by the present invention. In general, a model by deep learning or the like is created based on a large amount of sample data. This is not only created once, but it is constantly updated based on continuous collection of sample data. For example, data collected at each home is collected and learned on the cloud, and data collected with a smartphone is collected and learned on the cloud. The model generated by this learning is a general-purpose common model and can be used widely. On the other hand, learning is progressing individually in each home and each smartphone. For example, learning is performed according to the content of the user and usage environment, such as specific pronunciation, specific operation, and personal image. In this case, sample data is likely to be reduced, so that it is often used after re-learning based on the common model. Therefore, as shown in FIG. 1, the relationship between collection of sample data and learning of the common model, individual learning based on the learning, and use of the individual model is established.

(Model development flow)
Next, the flow of model development over time is shown in FIG. A learned common model that already exists at a certain point in time is referred to as “model A1”, a duplicated model is referred to as “model Z1”, and a model individually learned based on this is referred to as “model Z1 ′”. On the other hand, “model A1” is further learned as time passes, and “model A2” is generated. The “model A2” is basically higher in accuracy than the “model A1” and can make a wider determination. However, this is a general decision, and it may be better to use “model Z1 ′”. Therefore, by using the model integration device, an individual model derived from a learned common model can be integrated with a new common model without losing the individually learned result.

  Also, it is desired to generate a model “model Z2 ′” that is individually learned based on “model A2” again. However, “model Z2 ′” cannot generally be obtained by calculation from “model Z1 ′” or “model A2”. Therefore, using the output of the model integration device, a model “model Z2 ′” that is relearned based on “model Z2” that is a duplicate of “model A2” is generated gradually.

  Further, as shown in FIG. 3, it is possible to capture other common models along the way. This is to integrate common models other than the original “model A” when applying “model Z1 ′” learned individually to a new common model. This common model to be integrated can be handled as B, C, D, E,.

[First Embodiment]
(Configuration of model integration device)
FIG. 4 is a block diagram illustrating an example of the configuration of the model integration device 10 according to the present embodiment. The model integration device 10 includes an output acquisition unit 12 and a selector 14. In the model integration device 10, the selector 14 selects and outputs one of the three or more outputs acquired by the output acquisition unit 12.

  The output acquisition unit 12 receives at least the same input, “model Z1” (learned common model), “model Z1 ′” (individual model), and “model Z2” (“model A2”). Outputs from the new common model) are acquired as a first output, a second output, and a third output, respectively. The model output is information output as a determination result with respect to the input by the model determination unit 16, and for example, determination of an object such as a chair or a desk, depending on the learning content model. Judgment of people and pets. There may also be a selection of a task to respond to the conversation. For example, the information is as shown in the table of FIG. The output acquisition unit 12 includes, for example, “ClassID_X_k” (the determination content having the kth highest accuracy based on the model X) and “Confidence_X_k” (the accuracy of the kth determination content based on the model X). ) And other necessary information. When k = 1, “_k” is omitted. Although the model determination unit 16 is provided for each model, the function is the same except that the base model is different.

  The model Z1 determination unit 16a has a function of outputting a determination result with respect to an input based on “model Z1”. The output contents are a table composed of a set of “ClassID_Z1_k” indicating the determination contents and accuracy “Confidence_Z1_k” indicating the certainty thereof. For example, the input is a still image, “ClassID_Z1” is “table”, “Confidence_Z1” is 0.788, and the like.

  The model Z1 'determination unit 16b has a function equivalent to that of the model Z1 determination unit 16a described above, but the learning model used for the determination is a model in which learning is advanced individually. The output is a table composed of a set of determination content “ClassID_Z1′_k” and accuracy “Confidence_Z1′_k”.

  The model Z2 determination unit 16c has a function equivalent to that of the model Z1 determination unit 16a described above, but the learning model used for the determination is a common model with improved accuracy. The output is a table composed of a set of “ClassID_Z2_k” of determination contents and accuracy “Confidence_Z2_k”.

The selector 14 receives at least the outputs of the “model Z1”, “model Z1 ′”, and “model Z2” acquired by the output acquisition unit 12, and selects and outputs one of them. This process can be selected according to the following criteria, for example.
If Confidence_Z1 ′> Confidence_Z1, ClassID_Z1 ′ is output.
Otherwise, ClassID_Z2 is output.

This is divided into cases as follows.
a) When ClassID_Z1, ClassID_Z1 ', and ClassID_Z2 are all equal
ClassID_Z2 is output.
b) When ClassID_Z1 and ClassID_Z1 'are equal and ClassID_Z2 is different
If Confidence_Z1 ′> Confidence_Z1, ClassID_Z1 ′ is output.
Otherwise, ClassID_Z2 is output.
c) When ClassID_Z1 and ClassID_Z2 are equal and ClassID_Z1 'is different
If Confidence_Z1 ′> Confidence_Z1, ClassID_Z1 ′ is output.
Otherwise, ClassID_Z2 is output.
d) When ClassID_Z1 'and ClassID_Z2 are equal and ClassID_Z1 is different
ClassID_Z2 is output.
e) When ClassID_Z1, ClassID_Z1 ', and ClassID_Z2 are all different
If Confidence_Z1 ′> Confidence_Z1, ClassID_Z1 ′ is output.
Otherwise, ClassID_Z2 is output.

  As illustrated in FIG. 6, the model integration device 10 may select N outputs by integrating N common models. Compared to FIG. 4, the following functional blocks are different.

  The model B2 determination unit 16e has a function equivalent to that of the model Z1 determination unit 16a described above, except that the learning model used for the determination is “model B2”. The output is a table composed of a set of determination content “ClassID_B2_k” and accuracy “Confidence_B2_k”. Similarly, there are a model C2 determination unit, a model D2 determination unit, and N determination units.

The selector 14 receives an output of determination results of “model Z1”, “model Z1 ′”, “model Z2”, “model B2”..., And selects and outputs one of them. This process can be selected according to the following criteria, for example.
If Confidence_Z1 ′> Confidence_Z1, ClassID_Z1 ′ is output.
Other than that, class ID with high accuracy excluding “model Z1” and “model Z1 ′” is output.

  Note that the model integration device 10 according to the present embodiment has been described with respect to an apparatus that does not include the model determination unit 16 as illustrated in FIGS. 4 and 6, but a part of the model determination unit 16 for each model as illustrated in FIG. 7. Or you may have everything.

(Operation of model integration device)
Next, the operation of the model integration device 10 will be described. FIG. 8 is a flowchart showing an operation of integrating the individual model “model Z1 ′” and the new common model “model Z2”.

  The output acquisition unit 12 receives at least the same input, and outputs the output from “model Z2” obtained by duplicating “model Z1”, “model Z1 ′”, and “model A2” as the first output, 2 and the third output, and when there are other outputs, the other outputs are also acquired (step S1). Next, the output acquisition unit 12 inputs the other outputs to the selector 14 when there are the acquired first to third outputs and other outputs (step S2). Then, the selector 14 selects and outputs one of them according to a criterion that prioritizes the second output based on at least the accuracy of the first output and the accuracy of the second output as described above ( Step S3). With such an operation, in the individual model derived from the learned common model, it is possible to output the determination result integrated with the new common model without losing the individually learned result.

[Second Embodiment]
In the first embodiment, the model integration device 10 outputs a determination result integrated with a new common model without losing the individually learned result in the individual model derived from the learned common model. However, a configuration may be adopted in which an individual model based on a new common model can be generated by re-learning a new common model using the integrated determination result.

(Configuration of model integration device)
FIG. 9 is a block diagram illustrating an example of the configuration of the model integration device 10 according to the present embodiment. In addition to the configuration of the output acquisition unit 12 and the selector 14 shown in the first embodiment, the model integration device 10 includes a model Z1 determination unit 16a, a model Z1 ′ determination unit 16b, a model Z2 ′ determination unit 16d, and relearning The unit 20 is configured. The model integration device 10 selects one of the three or more outputs acquired by the output acquisition unit 12 and outputs it, and the relearning unit 20 selects “model” based on the output selected by the selector 14. Re-learning is performed on “Z2 ′”, and new individual models based on “model Z2” are gradually generated. In the block diagram, black arrows indicate that the contents of the model change due to learning. The initial value of “model Z2 ′” is “model Z2”, which changes to “model Z2 ′” by learning.

  The model Z1 determination unit 16a, the model Z1 ′ determination unit 16b, and the model Z2 ′ determination unit 16d respectively output determination results for inputs based on “model Z1”, “model Z1 ′”, and “model Z2 ′”. Output. The functions of the output acquisition unit 12 and the selector 14 are the same as those in the first embodiment. However, the output acquisition unit 12 acquires the third output from the model Z2 ′ determination unit 16d. In addition, it is assumed that the criterion for output selection of the selector 14 is obtained by replacing the reference Z2 with Z2 '.

  The relearning unit 20 has a function of performing relearning on the “model Z2 ′” using the input and output as sample data when the output of the selector 14 is not “ClassID_Z2 ′”. By repeating this relearning, the contents of “model Z2 ′” gradually change to those individually learned.

  The model Z2 ′ determination unit 16d outputs a determination result with respect to the input based on “model Z2 ′”, but before “model Z2 ′” is relearned, the determination result based on “model Z2” It becomes. Further, the model Z2 ′ determination unit 16d determines the determination result for the input based on the re-learned “model Z2 ′” after the relearning as the “model Z2 ′” gradually changes due to the relearning. Is output.

  The model integration device 10 according to the present embodiment has been described with respect to the device including the model determination unit 16 for each model other than the model Z2 ′ determination unit 16d as illustrated in FIG. 9. However, as illustrated in FIG. The apparatus may include only the Z2 ′ determination unit 16d.

(Operation of model integration device)
Next, the operation of the model integration device 10 will be described. FIG. 11A is a flowchart showing an operation until the individual model “model Z1 ′” is generated based on “model Z1” obtained by duplicating the common model “model A1”. In FIG. 11B, after the individual model “model Z1 ′” is generated, “model Z1 ′” and the new common model “model A2 (Z2)” are integrated, and based on “model Z2”. It is a flowchart which shows operation | movement until it produces | generates a new separate model "model Z2 '".

  The model integration device 10 downloads and imports the commonly learned “model A1”. For reference later, this is duplicated and stored as “model Z1”. Separately from this, it is separately copied and held as a base of a model to be individually learned, and this is designated as “model Z1 ′” (step T1). When actually used in each node, data such as video and audio is given, and a determination is made based on this data and “model Z1 ′” (step T2). In the initial state, there is no difference between “model Z1” and “model Z1 ′”, but a difference occurs as learning progresses. The determination result is output and used as it is as a function of each node (step T3). In this phase, the learning result of the common model can be used, and a service based on the common model can be provided.

  On the other hand, feedback is obtained from the user through a user interface (UI) such as a separate display or voice for the output result. For example, correction is input for an incorrect output determination, or an answer is inquired because the output is unknown (step T4). Based on the feedback result, the “model Z1 ′” is re-learned (step T5). This is individual learning for each home and user, such as specific videos and words.

  The model integration device 10 downloads and imports a new common model “model A2” periodically or as an update by a user instruction. This is copied for later reference and saved as “model Z2”. Separately from this, it is separately copied and stored as a base of a model to be individually learned, and this is designated as “model Z2 ′” (step T6). Even in this phase, data such as video and audio is given when actually used in each node. This data is determined based on “model Z1”, “model Z1 ′”, and “model Z2 ′” (the content is “model Z2” before being re-learned), and is selected by the selector 14 (step T7). For example, when an image is input, “Model Z1” has an accuracy of 0.77, “Model Z1 ′” has an accuracy of β, an accuracy of 0.89, and “Model Z2 ′” has an accuracy of Mr. α, an accuracy of 0.90. In this case, the output from the selector 14 is Mr. β by “model Z1 ′”. The learning result is prioritized over the new common model result, and there is no gap in updating for the user. This selection result is used as it is as a function of each node, and a service based on this can be provided in the same manner (step T8).

  Similarly, in this phase, feedback is obtained from the user through the UI for the output result. For example, correction is input for an incorrect output determination, or an answer is inquired because the output is unknown (step T9). Based on the fed back result, the “model Z2 ′” is re-learned (step T10). This is unique learning for each user, such as specific images and words. The difference from the phase from Step T1 to Step T5 is that in the re-learning of “model Z2 ′”, the re-learning is performed by regarding the output selected by the selector 14 as the correct answer even if there is no user feedback. is there. Thus, when data is repeatedly given, the content of “model Z2 ′” approaches the content individually learned by “model Z1 ′”.

  Even if there is no data that has generated “model Z1 ′”, the individual model “model Z2 ′” based on the new common model “model Z2” can be used by using the determination result for new data. Can be generated gradually. In addition, with respect to a portion of the “model Z2” whose accuracy has been improved, the accuracy of the “model Z1” and “model Z1 ′” is confirmed to avoid being dragged by old determinations. When integrating a plurality of common models, other common models are captured and stored in step T6, and used when obtaining a determination output based on the selector 14.

  Note that the input of the selector 14 may be two outputs of “model Z1 ′” and “model Z2 ′”, and the higher accuracy may be used as the output of the selector 14 to perform relearning. In this case, since highly accurate error data remains, the accuracy of relearning decreases when feedback from the user is not performed. Further, instead of automatically selecting the function of the selector 14, if the device has a screen display or the like, the user may be inquired about which to select through the UI. The selection result is learned as an output. Re-learning may be performed in real time. Alternatively, batch processing may be performed when the amount of processing is small such as at night.

  When the selector 14 outputs a large number (for example, 98%) of “ClassID_Z2 ′” results, it is preferable to stop the relearning for integration. This is because a re-learned model is not repeatedly re-learned and a useless process is not performed. In this case, only the re-learned “model Z2 ′” is used. Similar learning may be repeated using the re-learned “model Z2 ′” and “model Z2” and the newly improved “model A3” shown in FIG.

Note that the selector 14 is a functional block that gives priority to the result of individually learning by comparing the output accuracy of at least “model Z1” and “model Z1 ′”. The method of comparing the output accuracy of the model Z1 ′ ”is not limited to the above method. For example, the following criteria may be used.
Confidence_Z1'−Confidence_Z1> τ
This is on condition that the output accuracy of “model Z1 ′” is greater than a certain value (τ) and larger than “model Z1”.
In addition, when the second most accurate output of “Model Z1” is “Confidence_Z1_2” and the second most accurate output of “Model Z1 ′” is “Confidence_Z1'_2”,
Confidence_Z1'-Confidence_Z1'_2> Confidence_Z1-Confidence_Z1_2
This is on condition that there is no other output with the same precision for the output result. It can also be described as follows.
Confidence_Z1 '/ Confidence_Z1'_2> Confidence_Z1 / Confidence_Z1_2
Also, combining these ideas,
Confidence_Z1'-Confidence_Z1'_2> τ and Confidence_Z1 '> Confidence_Z1
And so on.

[Third Embodiment]
In the second embodiment, the model integration device 10 can generate an individual model based on a new common model by causing the new common model to be relearned using the integrated determination result. However, since it is considered that new common models will also be developed sequentially, in addition to the configuration of the second embodiment, the presence or absence of a third common model that has advanced learning based on the new common model is determined, and the third If there is a common model, the third common model may be taken in and replaced with a new common model.

(Configuration of model integration device)
FIG. 12 is a block diagram illustrating an example of the configuration of the model integration device 10 and the model integration system 100 according to the present embodiment. In addition to the configuration shown in the second embodiment, the model integration device 10 determines whether or not “model A3” (third common model) has been learned based on “model A2” (new common model). When there is “model A3”, an update unit 22 is provided that takes in “model A3” and replaces it with “model Z2 ′”. Since functions other than the update unit 22 are the same as those in the second embodiment, a description thereof is omitted.

  The update unit 22 determines the presence or absence of “model A3” that has been learned based on “model A2” periodically or according to an instruction from the user. The duplicated “model Z3 ′” and “model Z2 ′” are replaced.

  When “model A3” is captured and “model Z2 ′” and “model Z3 ′” are replaced, each model determination unit 16 changes the model to be based on and changes to “model Z2” (new common model). ), “Model Z2 ′” (re-learned model), and “model Z3 ′” (third common model), the same input is received and a determination result is output.

  Further, the output acquisition unit 12 outputs the outputs from the “model Z2”, “model Z2 ′”, and “model Z3 ′” output by each model determination unit 16, respectively, as a first output, a second output, Get as output of 3. An example of the configuration of the model integration device 10 after “model Z2 ′” and “model Z3 ′” are replaced is as shown in FIG.

  Note that the model integration device 10 according to the present embodiment is not the model Z2 ′ determination unit 16d (after being replaced by the update unit, becomes the model Z3 ′ determination unit 16f) as illustrated in FIGS. 12 and 13. Although the apparatus including the model determination unit 16 for each model has been described, the apparatus may include only the model Z2 ′ determination unit 16d as illustrated in FIG. 10 of the second embodiment.

(Operation of model integration device)
Next, the operation of the model integration device 10 will be described. FIG. 14 is a flowchart showing an operation until a model is replaced. First, the updating unit 22 determines the presence or absence of “model A3” that has been learned based on “model A2” periodically or according to a user instruction (step U1). If there is “model A3”, “model A3” is taken in, and the duplicated “model Z3 ′” and “model Z2 ′” are replaced (step U2). On the other hand, if there is no “model A3”, the process ends. Next, each model determination unit 16 replaces the base model (step U3). With this operation, the latest common model is acquired.

  The operation after replacing the model is the same as the operation of FIG. 11B shown in the second embodiment except that the base model is different. By such an operation, a new individual model can be generated based on the third common model that is improved in accuracy with respect to the new common model. In addition, it is possible to incorporate a portion whose accuracy has been improved by the third common model.

(Model integrated system configuration)
The model integration device 10 according to the present embodiment can be configured as a model integration system 100 in combination with a common model server 50 that regularly discloses a common model that has been learned. The common model server 50 manages a series of common models and improves accuracy by proceeding with learning. Further, the common model server 50 periodically publishes the common model that has been learned.

  The model integration device 10 periodically takes in a common model that is regularly released, and generates a new individual model based on the latest common model. Thereby, a new individual model based on a new common model published by the common model server 50 can be periodically generated.

DESCRIPTION OF SYMBOLS 10 Model integration apparatus 12 Output acquisition part 14 Selector 16, 16a-16f Model determination part 20 Relearning part 22 Update part 50 Common model server 100 Model integration system

Claims (8)

A model integration device that integrates a plurality of neural network models,
The output from one learned common model that has received the same input, an individual model that has been trained based on the common model, and a new common model that has been trained based on the common model, respectively. An output acquisition unit that acquires the first output, the second output, and the third output;
Among the acquired outputs, any one is selected and output according to a criterion that prioritizes the second output based on at least the accuracy of the first output and the accuracy of the second output. A model integration device comprising: a selector; A model determination unit that outputs a determination result in response to an input based on a new common model that has been learned based on the common model;
A re-learning unit that re-learns the new common model based on the selected one output,
When re-learning is performed on the new common model,
The model determination unit outputs a determination result with respect to an input based on the relearned common model,
The model integration apparatus according to claim 1, wherein the output acquisition unit acquires a re-learned output of the common model as a third output. An update that determines whether or not there is a third common model that has been trained based on the new common model, and if there is the third common model, imports the third common model and replaces it with the new common model Part
When the third common model is captured,
The model determination unit outputs a determination result with respect to an input based on the third common model,
The output acquisition unit receives the same input, and outputs outputs from the new common model, the relearned model, and the third common model as a first output, a second output, and a third output, respectively. As the output of
The model integration device according to claim 2, wherein the re-learning unit performs re-learning on the third common model based on the selected one output.   The re-learning unit stops re-learning when the second output and the third output coincide with each other at a rate equal to or greater than a predetermined threshold. The model integration device described. The output acquisition unit further receives outputs from one or more other common models that accept the same input and are different from the common model,
The selector is any one of all outputs acquired by the output acquisition unit according to a criterion that prioritizes the second output based on at least the accuracy of the first output and the accuracy of the second output. 5. The model integration apparatus according to claim 1, wherein one of the two models is selected and output. A model integration system that integrates multiple neural network models,
A common model server that manages the common model and improves accuracy by advancing learning;
A model integration device according to claim 3,
The common model server periodically publishes a new common model that has been learned,
The model integration system, wherein the model integration device periodically captures the new common model. A model integration method for integrating a plurality of neural network models,
The first input is an output from one learned common model that has received the same input, an individual model that has been trained based on the common model, and a new common model that has been trained based on the common model. Obtaining as a second output, a second output, and a third output;
Selecting and outputting any one of the respective outputs according to a criterion for prioritizing the second output based on at least the accuracy of the first output and the accuracy of the second output; A method comprising the steps of: A model integration program that integrates multiple neural network models,
The first input is an output from one learned common model that has received the same input, an individual model that has been trained based on the common model, and a new common model that has been trained based on the common model. Processing to obtain the output, the second output, and the third output,
A process of selecting and outputting any one of the respective outputs according to a criterion for prioritizing the second output based on at least the accuracy of the first output and the accuracy of the second output; A program for causing a computer to execute a series of processes.

Images