JP6172315B2 - Method and apparatus for mixed model selection - Google Patents

Description

  Embodiments of the present invention relate to machine learning, and in particular, to a mixed model selection method and apparatus.

  The mixed model is a kind of probability model related to density estimation using a mixed distribution, and represents a population composed of partial populations. Mixed models include mixed Gaussian models, piecewise linear mixed models, and the like, and are applied to a wide range of fields such as document classification, handwriting recognition, and fuzzy image separation.

  Model selection for a mixed model involves optimization of the representation of the model, such as model parameters, and is the most important and difficult problem to implement. Several model selections for mixed models are disclosed. In the model selection of the mixed model, the variational reasoning method that repeats the log likelihood approximation locally is a more effective method. However, the variational reasoning method is greatly affected by initialization. If initialization of the variational reasoning method is not appropriate, the effect of the variational reasoning method is low, and the model cannot be estimated accurately and effectively. Therefore, initialization is an important factor that affects the accuracy and effectiveness of the variation estimation method.

  In recent years, initialization methods for variational reasoning methods include, for example, random initialization and cluster-based initialization. However, random initialization generally requires a long processing time because many samples are used for initialization, and model selection is performed at a low speed. In addition, cluster-based initialization works well for cluster-type mixed models (eg, mixed Gaussian models), but does not provide appropriate results for mixed models for recognition / classification purposes. . For this reason, cluster-based initialization is less versatile.

  For this reason, in selecting a mixed model, an initialization method with a short processing time and high versatility is required.

  Embodiments of the present invention aim to provide a technical solution for mixed model selection.

In one aspect of the present invention, a method for selecting a mixed model is provided. The method of selecting this mixed model is
Generating a plurality of candidate models by training a plurality of first initial latent variables;
Determining a plurality of second initial latent variables based on a plurality of said candidate models;
Determining a target model based on the second initial latent variable;
including.

In another aspect of the present invention, an apparatus for selecting a mixed model is provided. The selection device for this mixed model is
A candidate model generation unit that generates a plurality of candidate models by training a plurality of first initial latent variables;
A first determination unit for determining a second initial latent variable based on the plurality of candidate models;
A second determination unit for determining a target model based on the second initial latent variable;
Is provided.

According to an embodiment of the present invention, the training results of the candidate model are used to generate a new initial latent variable. Thereafter, based on the new initial latent variable, a new model is obtained without being limited to the type of the mixed model. The solution of the present invention is therefore highly versatile. In addition, according to the embodiment of the present invention, a more reliable new initial latent variable is obtained based on a locally optimized model. Next, a target model is obtained based on the new initial latent variable.
The above target model acquisition method has a processing time significantly shorter than the method of acquiring the target model directly using some randomly initialized latent variables while having the same accuracy. . That is, the solution according to the invention boasts a high execution speed. Other features and advantages of the present invention will be described later.

  The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments of the present invention in the accompanying drawings.

It is a flowchart regarding the process of the method of the mixed model selection in embodiment of this invention. It is a flowchart regarding the process of the method of the mixed model selection in embodiment of this invention. It is a flowchart regarding the process of the method of determining the 2nd initial latent variable in embodiment of this invention. It is a schematic diagram for demonstrating the method of selection of the mixed model in embodiment of this invention. It is a block diagram of the apparatus of the mixed model selection in embodiment of this invention. It is a block diagram of an example of a computer system applicable to execution of an embodiment of the invention.

  Throughout the figures, the same or similar numbers represent the same or similar elements.

  Preferred embodiments of the present invention will be described in detail with reference to the drawings. The drawings show preferred embodiments of the present disclosure. However, the present disclosure can be implemented by various methods and should not be construed as limited to the embodiments described below. On the contrary, the embodiments are shown for a thorough understanding of this disclosure, and a person skilled in the art can understand everything about the scope of the present invention.

  The method and principle of the present invention will be described in detail. Unless stated otherwise, in the following and claims, the word “based” means “based at least in part”. The word “comprising” means “including but not limited to”. The word “plurality” means “two or more”. The phrase “one embodiment” means “at least one embodiment”. The phrase “another embodiment” means “at least another embodiment”. Definitions of other relevant language are given in the following description.

  FIG. 1 shows a flowchart relating to the processing of the mixed model selection method 100 according to the embodiment of the present invention. Conventionally, in initialization of variational reasoning, procedures such as random initialization and cluster-based initialization are used. However, these initialization methods generally have problems such as a long processing time and low versatility. All of these problems arise in mixed model selection. The mixed model selection method 100 according to this embodiment can efficiently solve this problem, as will be described later.

  As shown in FIG. 1, the mixed model selection method 100 first generates a plurality of candidate models by training using a plurality of first initial latent variables in step S110.

  In the embodiment of the present invention, a “latent variable” is a variable that cannot be directly observed but can be acquired from sample data. The variational distribution of latent variables can be used to represent the probability of clustered sample data for the corresponding category. It should be noted that in the embodiment of the present invention, the “latent variable” is not limited to one type but is composed of “variation distribution of latent variables” and / or other appropriate information. In this disclosure, latent variables include initial latent variables, updated latent variables, and the like. The initial latent variable represents a latent variable for training, and the updated latent variable represents a latent variable as a result of training.

In the embodiment of the present invention, the “model” may include a candidate model, an intermediate model, a target model, and the like. The model can be generated by training with initial latent variables. The generated model has elements such as a model structure, model parameters, and variational distribution of update latent variables.
Model parameters vary depending on the type of mixed model. In general, the mixed model is a term representing the type of model, and the mixed model may be composed of a plurality of lower models. That is, model parameters are associated with a particular type of mixed model. For example, the model parameters of the mixed Gaussian model include the mean and variance of the Gaussian distribution possessed by each lower model. The model parameters of the piecewise linear mixed model include a gate node condition control parameter, a leaf node deviation, and a regression coefficient.
The model structure is also related to a particular mixed model type. For example, the model structure of the mixed Gaussian model includes the number of lower models and the coupling coefficients of the lower models. The model structure of the piecewise linear mixed model includes a learned tree structure.
The above examples are given for illustrative purposes only and are not intended to limit the scope of the present invention.

In one embodiment, in step S110, a plurality of first initial latent variables are determined based on the training sample set.
For example, collect samples randomly from the training sample set, or cluster the samples to obtain a plurality of first initial latent variables. The first initial latent variable is a matrix, data set, or other suitable form.
In one embodiment, the first initial latent variable may be represented by a k × n dimensional matrix. Where k is the number of rows in the matrix, representing the number of sample groups of the first initial latent variable, and n is the number of columns in the matrix, representing the number of samples in the training sample set. One row of the matrix corresponds to one sample group. The value of each element in each row is 0 or 1. For example, if the i-th element in a row is 1, it means that the sample group corresponding to this row contains the i-th sample of the training sample set. If the jth element of this row is 0, it means that the sample group corresponding to this row does not include the jth sample of the training sample set. The element values in this matrix need not be 0 or 1, but other suitable numbers may be used.
The above examples are given for illustrative purposes only and do not limit the scope of the invention.
In other embodiments, the first initial latent variable may be, for example, a data set that includes k sample groups, each sample group having one or more samples.
The above examples are given for illustrative purposes only and do not limit the scope of the invention. One skilled in the art can obtain the first initial latent variable by other suitable techniques.

Next, training is performed on a training sample set for each first initial latent variable in the plurality of first initial latent variables, and a candidate model corresponding to each first initial latent variable is generated.
For example, the supervised learning using the training sample set for each first initial latent variable or unsupervised learning automatically learns the model structure of the mixed model, the model parameters, and the variational distribution of the updated latent variables. . Thereby, a corresponding candidate model is obtained.

  Next, the process of method 100 proceeds to step S120, where a second initial latent variable is determined based on the plurality of candidate models.

In embodiments of the present invention, the second initial latent variable may be determined based on a plurality of candidate models in various ways. In one embodiment, a plurality of candidate models are combined in pairs, and a second initial latent variable is determined based on the pair of candidate models.
For example, if there are two candidate models, a pair of candidate models is formed, and a second initial latent variable is determined based on the pair of candidate models. If there are a plurality of candidate models combined in pairs, a second initial latent variable is determined based on each pair of candidate models. As a result, a plurality of second initial latent variables are obtained.

In another embodiment, two or more candidate models are selected from a plurality of candidate models based on the performance of the plurality of candidate models. The performance of the model includes various factors such as accuracy and time effectiveness. In various ways, for example, the square root error, mean absolute error, likelihood ratio, etc. may be calculated to determine the performance of the model.
Taking accuracy as an example, in a mixed prediction / classification model, prediction / classification of test data can be used to obtain accuracy for the corresponding prediction / classification. In addition, in the clustering model, for example, the accuracy of the candidate model can be calculated by using common reference information.
By evaluating the performance of this model, two or more candidate models having higher performance can be selected from a plurality of candidate models. Thereafter, two or more selected candidate models may be combined in a pair, and the second initial latent variable may be determined based on the pair of candidate models.
In a further embodiment of the present invention, if the candidate models forming each pair of candidate models are a first candidate model and a second candidate model, the first candidate model of each pair of candidate models A second initial latent variable is determined based on the updated latent variable and the updated latent variable of the second candidate model. In this way, one or more second initial latent variables may be determined based on one or more pairs of candidate models.

According to the embodiment of the present invention, the determination of the second initial variable may be repeated to obtain a model with higher performance.
In one embodiment, the following i) to iii) may be executed once or repeatedly. i) Obtaining a plurality of intermediate models by training using a plurality of second initial latent variables. ii) Determine a third initial latent variable based on the plurality of intermediate models. iii) Update the second initial latent variable with the third initial latent variable.
In this embodiment, the method of determining the third initial latent variable based on the plurality of intermediate models may employ a method similar to step S120 of determining the second initial latent variable based on the plurality of candidate models. Good. In this method, the second initial latent variable is determined repeatedly until a certain state is reached. This state is, for example, a state in which the number of executions has reached a predefined number of repetitions, a state in which the accuracy of the second initial latent variable has reached a predefined accuracy, or other similar states.

  Further, as shown in FIG. 1, the process of method 100 proceeds to step S130, where a target model is determined based on the second initial latent variable.

In the present embodiment, one or more second initial latent variables may be determined in step S120.
In one embodiment, assuming that there is one second initial latent variable determined in step S120, the acquired model is determined as a target model by training the second initial latent variable in step S130.

  In another embodiment, if there are a plurality of second initial latent variables determined in step S120, a plurality of intermediate models are obtained by training the second initial latent variables in step S130, and a plurality of intermediate models are obtained. The performance may be determined, and one of a plurality of intermediate models may be selected as a target model based on the performance.

As can be seen from the above, the above-described embodiment is not related to the characteristics resulting from a specific type of the mixed model, and thus is highly versatile and is not limited to the specific type of the mixed model. Also, a partially optimal model is determined according to the performance, and a more reliable initial latent variable is determined based on this model. For this reason, the target model can be calculated in a shorter processing time, which is an efficient solution.
Furthermore, further advantages of the embodiment of the present invention will be described through the embodiment shown in FIG.

  FIG. 2 shows a flowchart relating to the processing of the mixed model selection method 200 according to the embodiment of the present invention. The mixed model selection method 200 may be considered an embodiment of the mixed model selection method 100. When determining a second initial latent variable from a plurality of candidate models, the method 200 first selects a high performance candidate model and combines the selected candidate models in pairs to determine a second initial latent variable. To do. This embodiment is shown for illustrative purposes only and does not limit the scope of the present invention.

  The method 200 first determines a plurality of first initial latent variables based on the training sample set in step S210.

In the embodiment shown in FIG. 2, the training sample set is a set composed of a plurality of samples (also called sample data). Each sample is characterized by a number of parameters.
A sample can be expressed as (x, y). Here, x represents a feature of the sample, and y represents an estimated value by the sample. For example, x in a sample of power consumption data includes today's power consumption, temperature, humidity, etc., and y is an estimate of tomorrow's power consumption. In other words, x = [today's power consumption, temperature, humidity...], Y = tomorrow's power consumption.
Hereinafter, a group of samples including one or more samples as described above is referred to as a “sample group”.

  In an embodiment of the present invention, a plurality of first initial latent variables are randomly generated from a training sample set. Alternatively, the samples of the training sample set may be clustered to generate a plurality of first initial latent variables. Alternatively, one skilled in the art may determine the first initial latent variable in other ways based on the training sample set. These illustrated embodiments do not limit the scope of the invention.

  In step S220, each first initial latent variable in the plurality of first initial latent variables is learned with the training sample set, and a candidate model corresponding to each first initial latent variable is generated.

  In one embodiment, for each first initial latent variable, the model structure of the mixed model, the model parameters, and the variational distribution of the updated latent variables are obtained by supervised learning using the training sample set or unsupervised learning. Learn automatically. Thereby, a corresponding candidate model is obtained. With this method, a plurality of candidate models corresponding to a plurality of first initial latent variables are obtained.

  Next, the process of method 200 proceeds to step S230, where two or more candidate models are selected from the plurality of candidate models based on the performance of the plurality of candidate models.

  Model performance includes accuracy, time effectiveness, and the like. In a mixed prediction / classification model, the model structure, model parameters, and variational distribution of updated latent variables in the candidate model can be used for test data prediction / classification to obtain prediction / classification accuracy. On the other hand, in the clustering model, the clustering accuracy of the candidate model can be calculated by using common reference information. In addition to accuracy, model performance may be obtained by determining other relevant elements of the model, such as time validity. The foregoing examples are given for illustrative purposes only and do not limit the scope of the invention.

In selecting candidate models, a large set composed of a plurality of candidate models may be reduced to a small set, a candidate model with higher performance may be selected, and initial latent variables may be further determined.
In one embodiment, the performance of each candidate model among the plurality of candidate models is determined based on the model structure, model parameters, and variational distribution of updated latent variables in each candidate model. The candidate models are ranked based on this performance. For example, a plurality of candidate models are arranged in order of high performance, and two or more candidate models are selected from the plurality of candidate models based on this order. For example, the first two or more candidate models are selected from a plurality of arranged candidate models, and a candidate model with higher performance is selected.

  In step S240, two or more candidate models are combined in pairs.

There are various methods for combining the pairs.
In one embodiment, two models out of two or more candidate models are combined as a pair. For example, when the number of candidate models is four, six pairs of candidate models are obtained by permutation combination.
In another embodiment, two candidate models may be selected from the candidate models and formed as a pair. For example, when there are four candidate models, 1, 2,..., Or 6 pairs of candidate models are obtained.

  In addition to the above paired methods, those skilled in the art may use other optimal methods. The foregoing examples are given for illustrative purposes only and do not limit the scope of the invention.

  In step S250, a second initial latent variable is determined based on the pair of candidate models.

The second initial latent variable is also an initial latent variable, but is different from the first initial latent variable. According to the embodiment of the present invention, each pair of candidate models in the candidate model combined with the pair includes a first candidate model and a second candidate model, and the update latent variable of the first candidate model includes: Then, one second initial latent variable is determined based on the updated latent variable of the second candidate model.
In this method, in step S250, one or more second initial latent variables are determined. Note that the number of second initial latent variables depends on the number of paired candidate models.

According to embodiments of the present invention, the second initial latent variable may be determined in various ways.
In one embodiment, the following i) to v) are performed once or repeatedly to determine a second initial latent variable. i) A sample group is selected as the first sample group from among the update latent variables of the first candidate model that forms each pair of candidate models. ii) Based on the first sample group, a second sample group is determined from the sample groups in the updated latent variables of the second candidate model forming each pair of candidate models. iii) Determine the intersection of the first sample group and the second sample group. iv) Based on this intersection, the first sample group is divided into two subsets. v) Build a second initial latent variable based on the two subsets. This is repeated until a predetermined number of times or until the construction of the second initial latent variable is completed.
FIG. 3 shows a flowchart for the processing of a method 300 for determining a second initial latent variable in one embodiment.

  In step S310, a sample group is selected as a first sample group from among the update latent variables of the first candidate model forming each pair of candidate models.

In one embodiment, the update latent variable of the first candidate model is represented as G1 = {I ₁ ,..., I _g1 }. Here, I is a sample group in the update latent variable of the first candidate model and represents a group of samples. G1 represents the number of sample groups in the update latent variable of the first candidate model.
Similarly, the update latent variable of the second candidate model is represented as G2 = {I ′ ₁ ,..., I ′ _g2 }. Here, I ′ represents a sample group in the update latent variable of the second candidate model. g2 represents the number of sample groups in the update latent variable of the second candidate model.
For example, in step S310, a sample group (eg, I ₁ ) in G ₁ is selected as the first sample group.

  In step S320, based on the first sample group, a second sample group is determined from the sample groups in the update latent variables of the second candidate model forming each pair of candidate models.

According to embodiments of the present invention, the second sample group may be determined in various ways.
In one embodiment, the intersection of the first sample group and each sample group in the updated latent variables of the second candidate model is calculated. Then, the sample group having the largest intersection is determined as the second sample group.

Alternatively, the example in which the sample group having the largest intersection is the second sample group has been described above, but the sample has an intersection with the first sample group and is included in the update latent variable of the second candidate model. The group may be determined as the second sample group.
The above-described embodiments are merely illustrative and do not limit the scope of the present invention.

  In step S330, the intersection of the first sample group and the second sample group is determined.

なお、Ｉ_ｉは第１のサンプルグループを表し、第１の候補モデルの更新潜在変数におけるｉ番目のサンプルグループである。Ｉ_ｊは、第２の候補モデルの更新潜在変数におけるｊ番目のサンプルグループを表す。Ｇ２は、第２の候補モデルの更新潜在変数を表す。 If the second sample group has the largest intersection with the first sample group, the intersection I _new between the first sample group and the second sample group is determined by the following equation.

Note that I _i represents the first sample group and is the i-th sample group in the update latent variable of the first candidate model. I _j represents the j th sample group in the update latent variable of the second candidate model. G2 represents an update latent variable of the second candidate model.

  In S340, the first sample group is divided into two subsets based on the intersection.

In one embodiment, a first subset of the first sample group is determined as an intersection. The part other than the intersection in the first sample group is determined as the second subset. For example, the first sample group I _i is divided into two subsets, I _new and I _i -I _new .

  In step S350, a second initial latent variable is constructed based on the two subsets.

  According to an embodiment of the present invention, the second initial latent variable may include a plurality of sample groups. In step S350, the second subset obtained by dividing the first sample group is used as a sample group for the second initial latent variable to construct a second initial latent variable.

のように、第２の初期潜在変数はサンプルグループを含まない。なお、Ｓは第２の初期潜在変数を表す。また、ステップＳ３５０において、第１のサンプルグループを分割した二のサブセットを、第２の初期潜在変数のサンプルグループとして、使用する。つまり、Ｓ＝Ｓ∪｛Ｉ_ｎｅｗ｝∪｛Ｉ_ｉ−Ｉ_ｎｅｗ｝が得られる。 For example, the second initial latent variable may be initialized as an empty set. In other words, for example,

As such, the second initial latent variable does not include a sample group. Note that S represents a second initial latent variable. In step S350, the two subsets obtained by dividing the first sample group are used as the sample group of the second initial latent variable. That is, S = S∪ {I _new } ∪ {I _i −I _new } is obtained.

  In step S360, the first sample group is removed from the updated latent variables of the first candidate model. Thereby, the first sample group is not repeatedly used in the subsequent process.

  In step S370, it is determined whether the construction of the second initial latent variable is completed.

  In one embodiment, the number of sample groups in the second initial latent variable (e.g., represented as | S |) and the number of sample groups by the updated latent variable of the first candidate model (e.g., | G1). It is determined whether the construction of the second initial latent variable has been completed by determining whether the total number of the first initial latent variable has reached a predefined number (for example, represented by k). The predefined number may be, for example, the number of sample groups of the first initial latent variable. If the total number does not reach the predefined number, in other words, if | S | + | G1 | <k, the process of method 300 proceeds to step S380. When the total number reaches the predefined number, in other words, when | S | + | G1 | ≧ k, it is determined that the construction of the second initial latent variable is completed.

  In step S380, it is determined whether the number of sample groups in the update latent variable of the first candidate model is zero. If not, the process of method 300 returns to step S310 and is repeated. If 0, the number of sample groups of the second initial latent variable constructed is first less than the number of sample groups of the initial latent variable and there are no sample groups in the updated latent variable of the first candidate model And the next step S381 is performed.

  In step S381, one sample group is selected from the second initial latent variable.

As a result of steps S310-S380, the second initial latent variable may include one or more sample groups. Therefore, in step S381, one sample group is selected from the sample groups of the second initial latent variable.
In one embodiment, select the largest sample group in the second initial latent variable. According to the embodiment of the present invention, selecting the largest sample group means selecting the sample group having the largest number of samples.
Alternatively, a sample group that reaches a predefined threshold may be selected. In other words, in step S381, a sample group having a large number of samples is selected, and it is not always necessary to select the largest sample group. The predefined threshold is determined according to various factors, such as system limitations, model categories, and execution speed requirements, which are not detailed here.

  In step S382, the selected sample group is divided into two and included in the sample group of the second initial latent variable.

According to the embodiment of the present invention, the selected sample group can be divided by various methods.
For example, the sample group selected in step S381 may be divided into two randomly or equally. Or it may be divided according to specific requirements.
The above dividing method is merely an example and does not limit the present invention.

  In step S383, the selected sample group is deleted from the second initial latent variable. Thereby, the divided sample groups are not repeatedly used in the subsequent processes.

  In step S384, it is determined whether the construction of the second initial latent variable is completed.

  In one embodiment, in step S384, it is first determined whether the number of sample groups in the second initial latent variable has reached the number of sample groups in the initial latent variable. If so, it is determined that the construction of the second initial latent variable has been completed. If not, the process of method 300 returns to step S381 and continues.

The embodiment shown in FIG. 3 is merely for explaining the procedure for determining the second initial latent variable, and does not limit the scope of the present invention.
According to the present embodiment, steps S310 to S384 are not necessarily executed in order to determine the second initial latent variable.
For example, in another embodiment, determining a first subset of each sample group for each sample group included in the update latent variable of the first candidate model forming each pair of candidate models. You can also. A sample group having the largest intersection with each sample group included in the update latent variable of the first candidate model is determined from the sample group included in the update latent variable of the second candidate model in each pair of candidate models. The largest intersection is determined as the first subset. Next, a portion excluding the first subset in each sample group is determined as the second subset.
In this method, each sample group included in the update latent variable of the first candidate model is divided into a first subset and a second subset. Assuming that the number of update latent variables of the first candidate model is g1, a new sample set of 2g1 is obtained after dividing each sample group into a first subset and a second subset.
The number of sample groups of the first initial latent variable is represented by k, and if 2g1> k, the two sample sets with fewer elements in the new sample set of 2g1 are merged into one sample set. By removing the two merged sample sets from the new sample set and including the merged sample set, a 2g1-1 sample set is obtained.
Thereafter, if (2g1-1)> k, the two sample sets of the small elements included in the new sample set of 2g1 are merged into one sample set, and the two sample sets merged including the merged sample set A sample set of 2g1-2 is acquired. This procedure is repeated until the number of sample sets obtained is the same as k. By this method, the second initial latent variable is determined based on the update latent variable of the first candidate model in the candidate model and the update latent variable of the second candidate model in the candidate model.

  Next, as shown in FIG. 2, the process of the method 200 proceeds to step S260, and a plurality of intermediate models are obtained by training a plurality of second initial latent variables.

  In one embodiment, supervised learning or unsupervised learning is performed for each second initial latent variable in the second initial latent variable using a training sample set, and the model structure of the mixed model, model parameters Automatically learning the variational distribution of the updated latent variables and generating a model corresponding to each second initial latent variable. This generated model is called an intermediate model. By this method, a plurality of intermediate models corresponding to a plurality of second initial latent variables are obtained.

  In step S270, the performance of a plurality of intermediate models is determined.

  The performance of the model can be determined by various means, including various factors such as accuracy, time validity, etc. For example, taking accuracy as an example, in a mixed prediction / classification model, an intermediate model may be used for prediction / classification of test data to obtain accuracy for prediction / classification. In the clustering model, the accuracy of the intermediate model can be calculated by using common information as a reference.

  In one embodiment, the accuracy, time validity, and other characteristics of each intermediate model may be determined based on the model structure, model parameters, and variational distribution of updated latent variables for each intermediate model. As a result, the performance of each intermediate model is acquired.

  In step S280, one target model is selected from a plurality of intermediate models based on performance.

  In one embodiment, the plurality of intermediate models are ranked according to the performance determined in step S270. For example, a plurality of intermediate models are arranged in descending order of performance. Then, the intermediate model having the highest performance is selected from the plurality of intermediate models as the target model.

  As a step after step S280, the target model acquired according to the embodiment of the present invention may be provided and used by the user.

FIG. 4 shows a schematic diagram 400 for a method of selecting a mixed model, according to one embodiment of the present invention. The method for selecting a mixed model shown in FIG. 4 corresponds to steps S210 to 280 in FIG. As shown in FIG. 4, first, based on the training sample set, a plurality of first initial latent variables denoted Z ₁ , Z ₂ ,..., Z _k1 are determined. Z represents the first initial latent variable, and k1 represents the number of first initial latent variables determined. This corresponds to step S210 of determining a plurality of first initial latent variables based on the training sample set.

Next, as shown in FIG. 4, a large number of candidate models (k1 candidate models) related to a plurality of first initial latent variables Z ₁ , Z ₂ ..., Z _k1 are generated. Here, candidate model 1, candidate model 2,..., Candidate model k1 represent candidate models. This corresponds to step S220 of generating a candidate model corresponding to each first initial latent variable.

  Furthermore, as shown in FIG. 4, candidate model 1, candidate model 5, candidate model 7,... Are selected from candidate model 1, candidate model 2,. This corresponds to step S230 in which two or more candidate models are selected from the plurality of candidate models based on the performance of the plurality of candidate models.

  Next, as shown in FIG. 4, candidate model 1 and candidate model 7, candidate model 5 and candidate model 7, and candidate model 1 and candidate model 5 are combined in pairs. This corresponds to step S240 in which two or more candidate models are combined in pairs. Although not shown for simplification, the other candidate models selected in step S230 are further combined as a pair.

Next, as shown in FIG. 4, second initial latent variables Z ′ ₁ , Z ′ ₂ , Z ′ ₃ ,..., Z ′ _k2 are obtained from candidate model 1, candidate model 5, candidate model 7, and the like. decide. Here, k2 represents the determined number of second initial latent variables. This corresponds to step S250 for determining the second initial latent variable based on the pair of candidate models.

Thereafter, as shown in FIG. 4, from the second initial latent variables Z ′ ₁ , Z ′ ₂ , Z ′ ₃ ,..., Z ′ _k2 , intermediate model 1, intermediate model 2, intermediate model 3 _,.・ ・ Acquire intermediate model k2. This corresponds to step S260 in which a plurality of intermediate models are acquired by training the second initial latent variable.

  Finally, as shown in FIG. 4, the target model is determined from the intermediate model 1, the intermediate model 2, the intermediate model 3,. This corresponds to steps S270 and S280 in which the performance of the intermediate model is determined and one target model is selected from a plurality of intermediate models based on this performance.

  The schematic diagram 400 for the method of selecting a mixed model is merely exemplary and does not limit the scope of the invention in any way. According to the embodiment of the present invention, it is not always necessary to select candidate model 1, candidate model 2,..., Candidate model 1, candidate model 5, candidate model 7, etc. from candidate model k1, and then combine them in pairs. Absent. For example, in another embodiment, candidate model 1, candidate model 2,..., Candidate model k1 are directly combined into a pair, and a second initial latent variable is determined based on the pair of candidate models. May be.

Further, according to the embodiment of the present invention, the target model is not necessarily determined directly from the intermediate model 1, the intermediate model 2, the intermediate model 3, ..., the intermediate model k2.
For example, in another embodiment, based on the intermediate model of k2, a third initial latent variable is determined, the second initial latent variable is updated with the third initial latent variable, and the updated second initial variable is determined. An updated intermediate model may be obtained based on the latent variable. This procedure may be performed once or several times. For example, the number of executions may be counted and compared with a predefined number of executions, and the procedure may be stopped when a predefined number of executions is reached. Alternatively, the procedure may be repeated until a predefined requirement is satisfied. The predefined requirement can be defined in various ways, for example, that the updated intermediate model has sufficient performance, or that the number of updated intermediate models reaches a predefined number.
These embodiments are merely illustrative and do not limit the scope of the invention. Those skilled in the art can make various modifications to the embodiments of the present invention within the scope of the present invention.

  FIG. 5 shows a block diagram of an apparatus 500 for selecting a mixed model according to an embodiment of the present invention. As illustrated in FIG. 5, the apparatus 500 includes a candidate model generation unit 510 for generating a plurality of candidate models by training a plurality of first initial latent variables, and a second initial latent variable based on the plurality of candidate models. The first determination unit 520 for determining and the second determination unit 530 for determining the target model based on the second initial latent variable are configured.

  In one embodiment, the candidate model generation unit 510 includes a “first initial latent variable determination unit” for determining a plurality of first initial latent variables based on the training sample set, and a plurality of first initial latent variables. For each first initial latent variable, a training sample set may be used to learn, and a “training unit” for generating a candidate model corresponding to each first initial latent variable.

  In one embodiment, the first determination unit 520 includes a “combination unit” for combining a plurality of candidate models in pairs and a “second unit” for determining a second initial latent variable based on the pair of candidate models. An initial latent variable determination unit ”may be included.

  In one embodiment, the first determination unit 520 includes two or more “first candidate model selection units” for selecting two or more candidate models from the plurality of candidate models based on the performance of the plurality of candidate models. A “combination unit” for combining the candidate models of the pair and a “second initial latent variable determination unit” for determining the second initial latent variable based on the pair of candidate models. .

  In one embodiment, the “first candidate model selection unit” of the first determination unit 520 performs each of the plurality of candidate models based on the model structure, model parameters, and variational distribution of update latent variables of each candidate model. A “first performance determination unit” for determining the performance of the candidate model, a “ranking unit” for ranking the candidate models based on the determined performance, and a candidate from a plurality of candidate models based on the ranking You may comprise from the "2nd candidate model selection part" for selecting two or more models.

  In one embodiment, each pair of candidate models in the pair of candidate models is composed of a first candidate model and a second candidate model, and the “second initial latent variable determination unit” of the first determination unit 520 Furthermore, the second initial latent variable may be determined based on the latent variable of the first candidate model and the updated latent variable of the second candidate model in each pair of candidate models.

  In one embodiment, the “second initialization latent variable determining unit” of the first determining unit 520 performs the first group of updated latent variables of the first candidate model that forms each pair of candidate models, the first group, “First sample group determination unit” for selecting as a sample group of the second candidate model based on the first sample group, and the second from the sample group in the update latent variable of the second candidate model forming each pair of candidate models A "second sample group determining unit" for determining two sample groups, a "intersection determining unit" for determining the intersection of the first sample group and the second sample group, and the first based on the intersection A “divider” for dividing the sample group into two subsets, and a “second initial latent variable for constructing a second initial latent variable based on the two subsets”. And construction unit ", it may be constructed from.

  In one embodiment, the “second initial latent variable structuring unit” of the “second initial latent variable deciding unit” of the first determining unit 520 is the number of sample groups of the constructed second initial latent variables. If the number of sample groups of the first initial latent variable is less than the number of sample groups of the first initial latent variable, the following i) to iii ) May be repeated. i) Select a sample group for the second initial latent variable. ii) Next, the selected sample group is divided into two subgroups, and the two subgroups are included in the sample group of the second initial latent variable. iii) Next, delete the selected sample group from the second initial latent variable.

  In one embodiment, the first determination unit 520 includes an “intermediate model generation unit” for acquiring a plurality of intermediate models by training the second initial latent variable, and a third initial latent variable based on the intermediate model. May be composed of a “third initial latent variable determining unit” for determining and an “updating unit” for updating the second initial latent variable with the third initial latent variable.

  In one embodiment, there are a plurality of second initial latent variables, and the second determining unit 530 obtains a plurality of intermediate models by training the second initial latent variables, You may comprise from the "2nd performance determination part" for determining the performance of an intermediate model, and the "target model selection part" for selecting one as a target model from an intermediate model based on performance.

In order to make the drawing easier to see, the option part and the lower part of the apparatus 500 are omitted in FIG. Apparatus 500 may be implemented in a variety of ways.
For example, the device 500 may be implemented in software, firmware, or both. For example, apparatus 500 may be implemented as a computer program product stored on a computer readable medium. In this case, each unit corresponds to a program module, and a computer instruction by the program module executes a function of each unit.
Device 500 may also be partially or fully implemented based on hardware such as, for example, an integrated circuit (IC), an application specific integrated circuit (ASIC), or a system on chip (SOC). In addition, it may be implemented by what is known in recent years or developed in the future and applicable.
In this respect, the scope of the present invention is not limited.

FIG. 6 illustrates a block diagram of an example computer system 600 suitable for implementing embodiments of the present invention. As shown in FIG. 6, the computer system 600 performs various processes in the CPU 601 of the central processing unit according to a program stored in the ROM 602 of the read-only memory or according to a program read from the storage unit 608 to the RAM 603 of the random access memory. Run the process.
The RAM 603 stores requested data in response to a request during execution of various processes by the CPU 601.
The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604.
An input / output interface I / O interface 605 is also connected to the bus 604.

Input unit 606 such as keyboard and mouse, display such as cathode ray tube display (CRT) and liquid crystal display (LCD), output unit 607 such as speaker, storage unit 608 such as hard disk, network communication card such as LAN card and modem The communication unit 609 is connected to the I / O interface 605.
The communication unit 609 executes a communication process via a network such as the Internet.
The drive 610 is connected to the I / O interface 605 as necessary.
A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is installed in the drive 610 as necessary, and a computer program read from the removable medium 611 is stored in the storage unit 608 as necessary. To be installed.

  In particular, the above-described procedure according to the embodiment of the present invention can be implemented as a computer software program. For example, one embodiment comprises a computer product that includes program code that implements the above procedures and that incorporates a computer program that is tangibly stored on a machine-readable medium. In this embodiment, a computer program may be downloaded from a network via the communication unit 609. Further, the computer program may be installed from the removable medium 611.

  In general, the various methods according to embodiments of the invention may be implemented on hardware, special purpose circuits, software, logic circuits, or combinations thereof. As long as the firmware or software is executed by a controller, microprocessor, or other computing device, it can be said to be implemented on hardware. Various methods according to embodiments of the present invention will be described as block diagrams, flowcharts, and other drawings, and as far as described, block diagrams, devices, systems, techniques or methods described herein may be used in hardware. , Software, firmware, special purpose circuits, logic circuits, general purpose hardware, controllers, other computer devices, combinations thereof, etc.

  Further, the various illustrated blocks may be considered as a plurality of logic circuit elements configured to perform method steps, computer program code execution, or associated functions. For example, an embodiment includes a computer product that incorporates a computer program that is constructed from program code that implements the above-described method and that is tangibly stored on a machine-readable medium.

  A machine-readable medium may be a tangible medium that is connected to an instruction execution system, apparatus, or device and that stores a program. Machine-readable media include machine-readable signal media and machine-readable storage media. A machine-readable medium is composed of electrical, magnetic, optical, electromagnetic, infrared, semiconductor systems, apparatus, devices, suitable combinations thereof, etc., and is not limited to the type of medium. Examples of machine-readable storage media include portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, electronically connected by wire Portable compact disk read only memory (CD-ROM), optical storage devices, electromagnetic storage devices, suitable combinations thereof, and the like.

  The computer program code for executing various methods according to embodiments of the present invention may combine a plurality of programming languages. Program codes for executing the functions of the flowcharts and block diagrams on the computer processing device and other programmable data processing devices may be provided to general-purpose computers, special computers, and other programmable data processing devices. This program code may be executed completely or partially on a computer. Further, as a single software package, it may be partially executed on a local computer and a remote computer, or may be completely executed on a remote computer or a server.

Further, even if functions are indicated by individual instructions, the implementation does not necessarily execute the indicated individual instructions or consecutive instructions, and does not necessarily execute all the indicated functions. Multitasking and parallel processing may be advantageous.
Further, even if the detailed implementation is described in the above-described embodiment, it should not be construed as limiting the scope of the present invention, but only in the embodiment.
Further, the contents described in different embodiments may be combined and executed.
Conversely, the contents described in the embodiment may be executed separately in parallel or in combination with appropriate parts.

  Various modifications and applications from the above embodiments can be understood from the attached drawings and the above contents. All modifications are also included in the scope of the present invention. Furthermore, from the above description and drawings, the present invention can be modified to another embodiment included in the scope of the present invention.

  It will be appreciated that embodiments of the invention are not limited to the embodiments described above, and that variations and other embodiments are within the scope of the appended claims. Moreover, the wording in this is only used in a general and descriptive notion, and has no limiting purpose.

(Appendix 1)
Generating a plurality of candidate models by training a plurality of first initial latent variables;
Determining a second initial latent variable based on a plurality of said candidate models;
Determining a target model based on the second initial latent variable;
A method for selecting a mixed model, comprising:

(Appendix 2)
Generating a plurality of candidate models by training a plurality of first initial latent variables;
Determining a plurality of said first initial latent variables based on a training sample set;
Learning for each first initial latent variable in a plurality of said first initial latent variables with said training sample set to generate said candidate model corresponding to each said first initial latent variable;
The method for selecting a mixed model according to appendix 1, wherein:

(Appendix 3)
Said step of determining a second initial latent variable based on a plurality of said candidate models;
Combining the plurality of candidate models as pairs of candidate models;
Determining the second latent variable based on the pair of candidate models;
The method for selecting a mixed model according to appendix 1, wherein:

(Appendix 4)
Said step of determining a second initial latent variable based on a plurality of said candidate models;
Selecting two or more candidate models from the plurality of candidate models based on the performance of the plurality of candidate models;
Combining two or more candidate models as pairs of candidate models;
Determining the second initial latent variable based on the pair of candidate models;
The method for selecting a mixed model according to appendix 1, wherein:

(Appendix 5)
The step of selecting two or more candidate models based on the performance of the plurality of candidate models from the plurality of candidate models,
Determining the performance of each candidate model in the plurality of candidate models based on the model structure, model parameters and variational distribution of updated latent variables of each candidate model;
Ranking the plurality of candidate models based on the determined performance;
Selecting two or more candidate models from the plurality of candidate models based on the ranking results;
The method for selecting a mixed model according to appendix 4, characterized by comprising:

(Appendix 6)
Each of the pair of candidate models in the pair of candidate models includes a first candidate model and a second candidate model, and the step of determining the second initial latent variable based on the pair of candidate models includes: ,
Determining the second initial latent variable based on the updated latent variable of the first candidate model and the updated latent variable of the second candidate model in each pair of candidate models;
The method for selecting a mixed model according to Supplementary Note 3 or 4, characterized by comprising:

(Appendix 7)
Determining the second initial latent variable based on the updated latent variable of the first candidate model and the updated latent variable of the second candidate model in each pair of candidate models;
The step of performing the following i) to v) one or several times:
i) selecting a sample group as a first sample group from the updated latent variables of the first candidate model in each pair of candidate models;
ii) determining, based on the first sample group, a second sample group from a sample group in an update latent variable of the second candidate model in each pair of candidate models;
iii) determining the intersection of the first sample group and the second sample group;
iv) dividing the first sample group into two subsets based on the intersection;
v) constructing the second initial latent variable based on the two subsets;
The method for selecting a mixed model according to appendix 6, characterized by comprising:

(Appendix 8)
Constructing the second initial latent variable based on the two subsets;
If the number of sample groups of the second initial latent variable constructed is less than the number of sample groups of the first initial latent variable, the number of sample groups of the second initial latent variable and the first initial variable Performing the following i) to iii) until the number of sample groups of latent variables match:
i) selecting a sample group from the second initial latent variables;
ii) dividing the selected sample group into two groups, and including the divided two groups as second sample groups in the second initial latent variable;
iii) deleting the selected sample group from the sample group of the second initial latent variable;
The method for selecting a mixed model according to appendix 7, wherein

(Appendix 9)
Said step of determining a second initial latent variable based on a plurality of said candidate models;
Performing the following i) to iii) one or several times:
i) obtaining a plurality of intermediate models by training a plurality of said second initial latent variables;
ii) determining a third initial latent variable based on a plurality of said intermediate models;
iii) updating the second initial latent variable with the third initial latent variable;
The method for selecting a mixed model according to appendix 1, wherein:

(Appendix 10)
A plurality of the second initial latent variables, and the step of determining a target model based on the second initial latent variables includes:
Obtaining a plurality of intermediate models by training a plurality of said second initial latent variables;
Determining the performance of a plurality of said intermediate models;
Selecting one as the target model from a plurality of the intermediate models based on the performance;
The method for selecting a mixed model according to appendix 1, wherein:

(Appendix 11)
A candidate model generation unit that generates a plurality of candidate models by training a plurality of first initial latent variables;
A first determination unit for determining a second initial latent variable based on the plurality of candidate models;
A second determination unit for determining a target model based on the second initial latent variable;
An apparatus for selecting a mixed model, comprising:

(Appendix 12)
The candidate model generation unit
A first initial latent variable determining unit that determines the plurality of first initial latent variables based on a training sample set;
A training unit that learns the training sample set for each first initial latent variable in the plurality of first initial latent variables and generates the candidate model corresponding to each first initial latent variable;
The mixed model selecting device according to claim 11, further comprising:

(Appendix 13)
The first determination unit is
A combination unit that combines the plurality of candidate models as a pair of candidate models;
A second initial latent variable determining unit that determines the second initial latent variable based on the pair of candidate models;
The mixed model selecting device according to claim 11, further comprising:

(Appendix 14)
The first determination unit is
A first candidate model selection unit that selects two or more candidate models from the plurality of candidate models based on the performance of the plurality of candidate models;
A combination unit that combines two or more candidate models as a pair of candidate models;
A second initial latent variable determining unit that determines the second initial latent variable based on the pair of candidate models;
The mixed model selecting device according to claim 11, further comprising:

(Appendix 15)
The first candidate model selection unit
A first performance determination unit that determines the performance of each candidate model in the plurality of candidate models based on the model structure, model parameters, and variational distribution of updated latent variables of each candidate model;
A ranking unit that ranks the plurality of candidate models based on the determined performance;
A second candidate model selection unit that selects two or more from the plurality of candidate models based on the ranking result;
15. The mixed model selection apparatus according to appendix 14, characterized by comprising:

(Appendix 16)
Each pair of candidate models in the pair of candidate models is composed of a first candidate model and a second candidate model;
The second initial latent variable determination unit further includes the second initial latent potential based on the update latent variable of the first candidate model and the update latent variable of the second candidate model in each pair of candidate models. Determine the variables,
15. The mixed model selection device according to appendix 13 or 14, characterized by:

(Appendix 17)
The second initial latent variable determination unit
A first sample group determination unit that selects a sample group as a first sample group from the update latent variables of the first candidate model in each pair of candidate models;
A second sample group determining unit that determines a second sample group from the sample groups in the update latent variables of the second candidate model in each pair of candidate models based on the first sample group;
An intersection determining unit for determining an intersection of the first sample group and the second sample group;
A divider for dividing the first sample group into two subsets based on the intersection;
A second initial latent variable construction unit for constructing a second initial latent variable based on the two subsets;
The mixed model selecting device according to appendix 16, wherein the mixed model selecting device is provided.

(Appendix 18)
The second initial latent variable construction unit
If the number of constructed sample groups of the second initial latent variable is less than the number of sample groups of the first initial latent variable, the number of sample groups of the second initial latent variable is the first initial variable. Perform the following i) to iii) until it matches the number of sample groups of latent variables: i) select a sample group from among the second initial latent variables, and ii) select the selected sample group Dividing into two groups, including the divided two groups as second sample groups in the second initial latent variable, and iii) selecting the sample group selected from the sample group of the second initial latent variable delete,
18. The mixed model selection device according to appendix 17, characterized in that.

(Appendix 19)
The first determination unit is
An intermediate model generation unit for acquiring a plurality of intermediate models by training the plurality of second initial latent variables;
A third initial latent variable determining unit that determines a third initial latent variable based on the plurality of intermediate models;
An update unit for updating the second initial latent variable with the third initial latent variable;
The mixed model selecting device according to claim 11, further comprising:

(Appendix 20)
There are a plurality of the second initial latent variables, and the second determination unit includes:
An intermediate model determining unit for acquiring a plurality of intermediate models by training the plurality of second initial latent variables;
A second performance determining unit that determines the performance of the plurality of intermediate models;
A target model selection unit that selects one as the target model from the plurality of intermediate models based on the performance;
The mixed model selecting device according to claim 11, further comprising:

Claims (10)

A candidate model generation unit that generates a plurality of candidate models by training a plurality of first initial latent variables;
A first determination unit for determining a second initial latent variable based on the plurality of candidate models;
A second determination unit for determining a target model based on the second initial latent variable;
An apparatus for selecting a mixed model, comprising: The candidate model generation unit
A first initial latent variable determining unit that determines the plurality of first initial latent variables based on a training sample set;
A training unit that learns the training sample set for each first initial latent variable in the plurality of first initial latent variables and generates the candidate model corresponding to each first initial latent variable;
The mixed model selection apparatus according to claim 1, further comprising: The first determination unit is
A combination unit that combines the plurality of candidate models as a pair of candidate models;
A second initial latent variable determining unit that determines the second initial latent variable based on the pair of candidate models;
The mixed model selection apparatus according to claim 1, further comprising: The first determination unit is
A first candidate model selection unit that selects two or more candidate models from the plurality of candidate models based on the performance of the plurality of candidate models;
A combination unit that combines two or more candidate models as a pair of candidate models;
A second initial latent variable determining unit that determines the second initial latent variable based on the pair of candidate models;
The mixed model selection apparatus according to claim 1, further comprising: The first candidate model selection unit
A first performance determination unit that determines the performance of each candidate model in the plurality of candidate models based on the model structure, model parameters, and variational distribution of updated latent variables of each candidate model;
A ranking unit that ranks the plurality of candidate models based on the determined performance;
A second candidate model selection unit that selects two or more from the plurality of candidate models based on the ranking result;
The mixed model selection apparatus according to claim 4, further comprising: Each pair of candidate models in the pair of candidate models is composed of a first candidate model and a second candidate model;
The second initial latent variable determination unit further includes the second initial latent potential based on the update latent variable of the first candidate model and the update latent variable of the second candidate model in each pair of candidate models. Determine the variables,
5. The mixed model selection apparatus according to claim 3, wherein the mixed model selection apparatus is provided. The second initial latent variable determination unit
A first sample group determination unit that selects a sample group as a first sample group from the update latent variables of the first candidate model in each pair of candidate models;
A second sample group determining unit that determines a second sample group from the sample groups in the update latent variables of the second candidate model in each pair of candidate models based on the first sample group;
An intersection determining unit for determining an intersection of the first sample group and the second sample group;
A divider for dividing the first sample group into two subsets based on the intersection;
A second initial latent variable construction unit for constructing a second initial latent variable based on the two subsets;
The mixed model selecting apparatus according to claim 6, further comprising: The first determination unit is
An intermediate model generation unit for acquiring a plurality of intermediate models by training the plurality of second initial latent variables;
A third initial latent variable determining unit that determines a third initial latent variable based on the plurality of intermediate models;
An update unit for updating the second initial latent variable with the third initial latent variable;
The mixed model selection apparatus according to claim 1, further comprising: There are a plurality of the second initial latent variables, and the second determination unit includes:
An intermediate model determining unit for acquiring a plurality of intermediate models by training the plurality of second initial latent variables;
A second performance determining unit that determines the performance of the plurality of intermediate models;
A target model selection unit that selects one as the target model from the plurality of intermediate models based on the performance;
The mixed model selection apparatus according to claim 1, further comprising: Generating a plurality of candidate models by training a plurality of first initial latent variables;
Determining a second initial latent variable based on a plurality of said candidate models;
Determining a target model based on the second initial latent variable;
A method for selecting a mixed model, comprising:

Images