JP2016194909A - Sectional linear model generation system and generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To learn both of a model structure and model parameter in a sectional linear model such as a hierarchical mixture of experts (HME).SOLUTION: A sectional linear model generation system includes: an acquisition module 201 that is adapted to acquire data on a plurality of tasks; a first setting module 202 that is adapted to set a task correlation latent variable expressing a correlation among the plurality of tasks, in which each task further correlates to one sectional liner model; a second setting module 203 that sets a model structure to a plurality of sectional linear models; and a model optimization module 207 that is adapted to optimize the model structure and model parameter on the basis of the data on the plurality of tasks, the task correlation latent variable and hierarchical latent variable.SELECTED DRAWING: Figure 2

Description

  Various embodiments of the present disclosure relate to the field of machine learning, and more specifically to piecewise linear model generation systems and methods.

  Piecewise linear models (e.g., hierarchical mixture of experts (HME)) are widely used in many enterprise-level machine learning applications. HME is more flexible than general linear models, And the important advantage of using feature space probabilistic rule-based partitioning and piecewise local linear experts remains: The HME learning procedure consists of determining and modeling the partition structure of individual experts. This can be challenging if the sample is noisy and the sample size is not enough, and a flexible HME can overfit the training data set, resulting in poor generalization performance. is there.

  In practical applications, there is often a set of correlated machine learning tasks, for example to predict the energy demand of a community building. A simple approach would be to solve these tasks independently, ignoring the commonality between these correlated tasks (Single Task Learning (STL)), Multitask Learning (Multi-). In Task Learning (MTL), these correlated tasks are learned jointly by appropriately sharing and using information across the tasks, by learning multiple correlated tasks jointly, It can effectively increase the sample size of each task and, as a result, improve generalization performance.

  Multitask learning collaboratively processes samples across different tasks through information shared between tasks to increase the amount of samples so that fitting noise can be prevented. However, current multitask learning techniques are mainly used for scenarios with a fixed model structure (eg, linear regression). For piecewise linear models such as HME, both model structure and model parameters should be learned. However, current multitask learning cannot do that.

  An object of the present disclosure is to provide a piecewise linear model generation system and a generation method in order to solve at least a part of the above-described problems of the prior art.

  According to a first aspect of the present disclosure, an acquisition module configured to acquire data of a plurality of tasks and a task correlation latent variable representing correlation between the plurality of tasks are configured. Each of the tasks is correlated to a piecewise linear model, sets the model structure to a plurality of piecewise linear models, and initializes the corresponding model parameters and hierarchical latent variables. Optimizing the model structure and the model parameters based on a second setting module configured to enable, the data of the plurality of tasks, the task correlation latent variables, and the hierarchical latent variables A piecewise linear model generation system comprising: a model optimization module configured to:

  According to one exemplary embodiment of the present disclosure, the task correlation latent variable includes a correspondence relationship between each task of the plurality of tasks and each piecewise linear model of the plurality of piecewise linear models. Configured as shown.

  According to one exemplary embodiment of the present disclosure, the model structure includes a tree structure.

  According to one exemplary embodiment of the present disclosure, the system is configured to generate the hierarchical data based on the data of the plurality of tasks, the task correlation latent variable, the model structure, and the model parameters. Further included is a hierarchical latent variable optimization module configured to optimize the latent variables.

  According to one exemplary embodiment of the present disclosure, the system includes the task correlation latency based on the data of the plurality of tasks, the hierarchical latent variables, the model structure, and the model parameters. It further includes a task correlation latent variable optimization module configured to optimize the variables.

  According to an exemplary embodiment of the present disclosure, the system, after optimizing the task correlation latent variable, based on the task correlation latent variable and the updated task correlation latent variable, A hard clustering module configured to perform hard clustering of the task is further included.

  According to an exemplary embodiment of the present disclosure, the system further includes a determination module configured to determine whether the model structure and the model parameters are optimal.

  According to a representative embodiment of the present disclosure, the determination module determines whether or not the model structure and the model parameter are optimal based on a fitting degree between the model structure and the model parameter.

  According to another aspect of the present disclosure, the method includes: acquiring data of a plurality of tasks; and setting a task correlation latent variable that represents a correlation between the plurality of tasks, each task having one category Correlating to a linear model, setting a model structure in a plurality of piecewise linear models, initializing corresponding model parameters and hierarchical latent variables, the data of the plurality of tasks, and the tasks A piecewise linear model generation method is provided that includes optimizing the model structure and the model parameters based on correlated latent variables and the hierarchical latent variables.

  According to one exemplary embodiment of the present disclosure, the task correlation latent variable includes a correspondence relationship between each task of the plurality of tasks and each piecewise linear model of the plurality of piecewise linear models. Configured as shown.

  According to one exemplary embodiment of the present disclosure, the model structure includes a tree structure.

  According to one exemplary embodiment of the present disclosure, the method includes the hierarchical method based on the data of the plurality of tasks, the task correlation latent variables, the model structure, and the model parameters. It further includes optimizing the latent variable.

  According to one exemplary embodiment of the present disclosure, the method includes the task correlation based on the data of the plurality of tasks, the hierarchical latent variable, the model structure, and the model parameters. It further includes optimizing the latent variable.

  According to an exemplary embodiment of the present disclosure, the method includes: after optimizing the task correlation latent variable, based on the task correlation latent variable and the updated task correlation latent variable, It further includes performing hard clustering of tasks.

  According to an exemplary embodiment of the present disclosure, the method further includes determining whether the model structure and the model parameters are optimal.

  According to a representative embodiment of the present disclosure, the determining whether the model structure and the model parameter are optimal is based on a fitting degree between the model structure and the model parameter. Determining whether the structure and the model parameters are optimal.

  In the technical solutions of the various embodiments of the present disclosure, new latent variables are added during the model generation process to indicate correlation between tasks to facilitate the generation of piecewise linear models, Data from multiple tasks can be used for modeling, thereby increasing the accuracy of model generation.

  In addition, since each task is set to correlate with only one piecewise linear model, the piecewise linear model generation system and method according to the present disclosure can process larger data.

  These and other objects, features and advantages will become apparent when the following detailed description of the exemplary embodiments is read in conjunction with the accompanying drawings.

FIG. 3 is a flow diagram schematically illustrating a piecewise linear model generation method according to one embodiment of the present disclosure. 1 is a block diagram schematically illustrating a piecewise linear model generation system according to one embodiment of the present disclosure. FIG. 6 is a chart schematically illustrating steps of a piecewise linear model generation method according to an embodiment of the present disclosure.

  In the following, the principles and methods of the present disclosure will be described with reference to some exemplary embodiments of the accompanying drawings. These embodiments are understood to be better understood by those skilled in the art and are further described to enable the present disclosure, and are not intended to limit the scope of the present disclosure in any way. Should be.

  Before describing the piecewise linear model generation method and system according to various embodiments of the present disclosure, some of the technical concepts that appear here will first be presented.

Piecewise Linear Model The predecessor of the piecewise linear model is a regression tree. In the regression tree, the partition is specified by a chain of rules, and the local expert has a certain value. Regression tree representations are easy to understand, but the predictability of individual experts is not high, and the tree depth tends to increase to achieve good prediction performance.

Hierarchical mixture of experts (HIERARCHICAL MIXTURE OF EXPERTS (HME))
HME adopts a divide-and-conquer strategy to create a piecewise model. The division structure of these trees is determined by a “gating” function which is a stochastic soft division function. HMEs can be expressed in any partition structure depending on the design of the gating function, but only a regular chain (tree) partition structure is adopted here so that the learned partition structure can be actually understood. Is done.

ここで、θとγとは、ゲーティング・ノードからのパラメータであり、Ｅはエキスパートの数であり、ε_ｊはルート・ノードからｊ番目のエキスパート・ノードまでの唯一のパスのゲーティング・ノードの全ての索引を含む索引セットである。ｇ（ｘ，α_ｉ）は、α_ｉ＝（β_ｉ；γ_ｉ）によりパラメータ化されたゲーティング関数と呼ばれ、ψ_ｇ（ｘ，ｉ，ｊ）は、エキスパートｊとｉの左サブツリーに属するｘの確率であり、ｐ（ｙ｜ｘ，φ_ｊ）は、

によりパラメータ化されるガウス形雑音を有するｊ番目のエキスパートの条件付き分布である。 The HME model is defined as follows.

Where θ and γ are parameters from the gating node, E is the number of experts, and ε _j is the only path gating node from the root node to the jth expert node. Is an index set that includes all of the indexes. g (x, α _i ) is called the gating function parameterized by α _i = (β _i ; γ _i ), and ψ _g (x, i, j) is in the left subtree of experts j and i The probability of x belonging to p (y | x, φ _j ) is

Is a conditional distribution of the j th expert with Gaussian noise parameterized by:

ｚ^Ｎ＝ｚ^（１），・・・，ｚ^（Ｎ），ｚ^（ｎ）∈｛１，０｝^Ｃは、ｘ^Ｎに対応する潜在変数を示し、どの混合成分がｘ^（ｎ）を生成するかを示す。このような潜在変数モデルは、因子化漸近ベイジアン（Ｆａｃｔｏｒｉｚｅｄ Ａｓｙｍｐｔｏｔｉｃ Ｂａｙｅｓｉａｎ（ＦＡＢ））推論により解ける。ＦＡＢは、因子化情報量基準（Ｆａｃｔｏｒｉｚｅｄ Ｉｎｆｏｒｍａｔｉｏｎ Ｃｒｉｔｅｒｉｏｎ（ＦＩＣ））に基づいた、近年開発されたベイジアン近似である。ＦＩＣは、以下のように、周辺対数尤度の漸近近似として導き出される。

ここで、ｑ（ｚ^Ｎ）はｚ^Ｎの変分分布であり、θ⁻は、モデルパラメータの最尤推定量である。ＦＩＣ^ｍｍで最も重要な項は

であり、この項は、無関係な潜在変数の自動的な減少とパラメータ識別可能性という、ＦＡＢ推理における、理論的に望ましい、これらの特性をもたらす。 Factorized Asymptotic Bayesian (FAB) in mixed models
Assume that the following observation data exists.

z ^N = z ⁽¹⁾ ,..., z ^(N) , z ⁽ⁿ⁾ ∈ {1, 0} ^C denotes a latent variable corresponding to x ^N , and which mixture component generates x ⁽ⁿ⁾ Indicates what to do. Such latent variable models can be solved by factorized asymptotic Bayesian (FAB) inference. FAB is a recently developed Bayesian approximation based on the Factorized Information Criterion (FIC). The FIC is derived as an asymptotic approximation of the marginal log likelihood as follows.

Here, q (z ^N ) is a variational distribution of z ^N , and θ ⁻ is a maximum likelihood estimator of model parameters. The most important term in FIC ^mm is

This term provides these theoretically desirable properties in FAB inference: automatic reduction of irrelevant latent variables and parameter identifiability.

  FAB optimizes the manageable lower bound of FIC. Alternately maximizing the lower limits on q and θ guarantees a monotonic increase in the lower limit of FIC.

ここで、Ｗは、トレーニング・サンプルから推定されるパラメータであり、Ｌ（Ｗ）は、トレーニング・セットの経験的損失であり、Ω（Ｗ）はタスク相関性をコード化する正則項である。タスク相関性に関して異なる仮定をすると、異なる正則項になる。マルチタスク学習の分野では、新しい正則化を使用してタスク間の関係をモデル化する先行研究がたくさんある。 Multi-task learning (Multi-Task Learning (MTL))
A common paradigm in machine learning MTL is to minimize penalty experience loss.

Where W is a parameter estimated from the training sample, L (W) is the empirical loss of the training set, and Ω (W) is a regular term encoding task correlation. Different assumptions about task correlation result in different regular terms. In the field of multitasking learning, there are many previous studies that use new regularization to model the relationships between tasks.

  In the following, the concept of the present invention will be illustrated with respect to the exemplary embodiment shown in FIGS. 1-3. In the following description, the principles of the present disclosure are illustrated with an HME model, which is an example of a piecewise linear model. However, the present disclosure is not limited to the HME model and can be applied to other piecewise linear models.

  The gist of the present disclosure is to address the challenging task of generating piecewise linear models that use multitask learning. In the present disclosure, the theory of Factorized Asymptotic Bayesian (FAB) and the theory of multi-task learning (Multi-Task Learning (MTL)) are combined. First, assume that a subset of tasks follows the same distribution. In the context of this disclosure, multitask learning is aimed at detecting clusters of tasks belonging to the same distribution and fitting each cluster using a piecewise linear model (eg, HME model). . Tasks in the same cluster can improve learning performance with each other's strength.

  FIG. 1 schematically illustrates a flow diagram of a piecewise linear model generation method according to one embodiment of the present disclosure. As shown in FIG. 1, the piecewise linear model generation method generally includes obtaining data of a plurality of tasks at 101 and setting a task correlation latent variable representing the correlation between the plurality of tasks at 102. Each task is correlated to a piecewise linear model, and at 103, a model structure is set for a plurality of piecewise linear models, and corresponding model parameters and hierarchical latent variables are initialized; and , 107 may include optimizing the model structure and model parameters based on a plurality of task data, task correlation latent variables, and hierarchical latent variables. Each step will be described in detail below.

  In step 101, data of N tasks is acquired.

  In step 102, a task correlation latent variable representing the correlation between a plurality of tasks is set. Here, each task is correlated to one piecewise linear model. As an example, the task correlation latent variable is configured to represent the correspondence between each task of the plurality of tasks and each piecewise linear model of the plurality of piecewise linear models. In other words, the correlation between tasks may be represented by the probability that each task belongs to the model. Two tasks are considered correlated if they belong to the same model. In the present disclosure, each task corresponds to only one piecewise linear model, and thus larger data can be processed.

  In step 103, model structures are set in a plurality of piecewise linear models, and their model parameters and hierarchical latent variables are initialized. As an example, the model structure may be a tree structure, and the initialization process may be executed at random.

  According to one exemplary embodiment, the piecewise linear model generation method includes a plurality of task data, a task correlation latent variable, a model structure, a model parameter, after step 103 and before step 107. May further include a step 104 of optimizing the hierarchical latent variable. Since hierarchical latent variables are initialized randomly at step 103, they can be optimized at this step. The hierarchical latent variable is for indicating how to divide the data, and the feature space is divided into regions. Each region is represented by a local expert. In the process of dividing into regions by hierarchical latent variables, clustering is performed not by features but by samples. That is, which samples are placed in one area and which samples are placed in other areas. In a specific example, each sample is clustered according to probability. In a specific optimization method, a process represented by the following expression (13) may be adopted.

  According to one exemplary embodiment, the piecewise linear model generation method includes a plurality of task data, a hierarchical latent variable, a model structure, a model parameter, after step 104 and before step 107. May further include a step 105 of optimizing the task correlation latent variable. Since the task correlation latent variables set in step 102 were generated through random initialization, they need to be optimized. In this step, the task correlation latent variable may be optimized using the hierarchical latent variable optimized in step 104 together with the data of a plurality of tasks, the model structure, and the model parameters. In a specific optimization method, a process represented by the following formula (11) may be adopted.

  According to one exemplary embodiment, the piecewise linear model generation method includes a plurality of tasks based on the task correlation latent variable and the updated task correlation latent variable after step 105 and before step 107. The method may further include a step 106 of performing the hard clustering. Hard clustering forces a task to belong to only one model, either to belong to this model or to that model. This avoids an increase in space overhead. Thus, the piecewise linear model of the present disclosure according to the present disclosure not only increases prediction accuracy, but also optimizes computational resources. It can therefore be adapted to large data scenarios. In a specific optimization method, a process represented by the following formula (12) may be adopted.

  According to one exemplary embodiment, in step 107, the hierarchical latent variable optimized in step 104 and the task correlation latent variable optimized in step 105 are combined with data of a plurality of tasks, as well as a model. It may be used to optimize the structure and model parameters. In a specific optimization method, the process represented by the following formula (14) may be adopted.

  According to one exemplary embodiment, the piecewise linear model generation method further includes a step 108 of determining whether the model structure and model parameters are optimal after step 107, and if Step 109 for outputting a linear model may be executed, and if it is not optimal, step 104 may be returned to and executed. The evaluation index for the determination may be a fitting degree between the model and the data. The degree of fitting cannot be increased without an upper limit. On the contrary, there is an upper limit. In incremental processing, the increase due to continuing optimization increases to some extent and then becomes very small. If the increase is small enough, an optimal model is considered to have been derived. Therefore, optimization is stopped and the resulting piecewise linear model is output at step 109.

  The particular embodiment described above with respect to FIG. 1 illustrates the principle of generating a piecewise linear model according to the present disclosure. However, one of ordinary skill in the art will appreciate that the piecewise linear model generation method according to the present disclosure is not limited to the specific steps described above, but is based on the scope limited in the claims. One skilled in the art will readily imagine deleting or adding one or more steps based on various needs.

  Next, an optimization method applicable to steps 104 to 107 will be described based on a specific example.

First, some symbols are defined as follows.
N: number of tasks L: number of samples in each task x ^NL , y ^NL : each feature and label of the training data set D: original model for data generation M: number of HME models in the mixture z: task Assignment vector ζ to model: assignment vector from sample to HME expert

ここで、Ｎ_ｃはｃ番目のクラスタにおけるタスクの数である。ここで、

と仮定することによって、ｚ^Ｎ _ｃに基づいてハード・クラスタリングが適用される。 MTL takes into account task clustering (denoted by z) and sample clustering (denoted by ζ) in each task, and its main interest is marginal likelihood p (y ^NL | x ^NL , D) It is. Therefore, the lower limit of the peripheral log likelihood for q (z ^N ) and q (ζ ^NL ) is derived as follows.

Here, N _c is the number of tasks in the c-th cluster. here,

, Hard clustering is applied based on z ^N _c .

Next, the lower limit may be derived by combining Equation (4) and Equation (5).

max _q {VLB (q, x ^NL , y ^NL , D)} can be guaranteed to match logp (y ^NL | x ^NL , D).

  Next, factorized asymptotic approximation will be described.

ここで、ρ＝（α，ρ_１，・・・，ρ_Ｍ）であり、αは各クラスタの混合比率ベクトルであり、ρ_ｃ＝（β_ｃ，γ_ｃ，φ_ｃ）はｃ番目のＨＭＥモデルのパラメータであり、ｃ＝１，・・・，Ｍであり、

である。上記処理では、周辺対数尤度の近似に基づいてのみ、ＦＩＣは判定される。 Assuming mutual independence between the task clusters and the sample clusters in each task cluster, we approximately obtain a Factorized Information Criterion (FIC) with the Laplace method.

Here, ρ = (α, ρ ₁ ,..., Ρ _M ), α is a mixture ratio vector of each cluster, and ρ _c = (β _c , γ _c , φ _c ) is the c-th HME. Model parameters, c = 1,..., M,

It is. In the above processing, the FIC is determined only based on the approximation of the peripheral log likelihood.

  Next, a FAB algorithm that can be used in the piecewise linear model generation method will be described.

ここで、

であり、ｑｎｇ_ｉとｑｎｅ_ｉとは、新しいパラメータ（分布）ｑ^〜を有する、ｎｇ_ｉとｎｅ_ｉとの近似である。

The FAB algorithm actually maximizes the lower bound of the FIC asymptotic agreement. The lower limit is derived as follows.

here,

In it, the QNG _i and qne _i, a new parameter (distribution) having ^{q ~,} approximation of ng _i and ne _i.

Then the following optimization problem arises.

ここで、

である。 The FAB algorithm alternately performs maximization in E-step and M-step with respect to q and θ. The E-step optimizes the variation distribution q as follows.

here,

It is.

  As described above, equation (11) may be used at step 105 to optimize task correlation latent variables.

Then, hard clustering is applied to each task.

  As described above, equation (12) may be used at step 106 to perform hard clustering for multiple tasks.

ここで、

であり、

は、ｘ_ｎｌが、（β_ｃｉ，γ_ｃｉ）によりパラメータ化される所定のＨＭＥモデルのエキスパートｊ及びｉの左サブツリーに属する確率である。 The E-step optimizes q (ζ ^nl _cj ) for the c th model.

here,

And

Is the probability that x _nl belongs to the left subtree of experts j and i of a given HME model parameterized by (β _ci , γ _ci ).

  As described above, equation (13) may achieve hierarchical latent variable optimization at step 104.

The M-step optimizes the parameters as follows.

  As described above, equation (14) can be used in step 107 to perform optimization of the model structure and model parameters.

  FIG. 2 schematically illustrates a block diagram of a piecewise linear model generation system according to one embodiment of the present disclosure. As seen in FIG. 2, the piecewise linear model generation system generally includes an acquisition module 201 configured to acquire data for a plurality of tasks, and a task correlation latent variable representing the correlation between the plurality of tasks. A first setting module 202 configured to set, wherein each task is correlated to one piecewise linear model, setting a model structure for the plurality of piecewise linear models, and corresponding model parameters Model structure and model parameters based on the second setting module 203 configured to initialize and hierarchical latent variables, data of a plurality of tasks, task correlation latent variables, and hierarchical latent variables And a model optimization module 207 configured to optimize. Each module will be described in detail below.

  In the acquisition module 201, data of N tasks may be acquired.

  In the first setting module 202, a task correlation latent variable representing the correlation between a plurality of tasks is set, and each task is correlated to one piecewise linear model. As an example, the task correlation latent variable is configured to represent a correspondence between each task of the plurality of tasks and each piecewise linear model of the plurality of piecewise linear models. In other words, the correlation between tasks can be represented by the probability that each task belongs to the model. If two tasks belong to the same model, the two tasks are considered related. In this disclosure, each task corresponds to only one piecewise linear model, so larger data can be processed.

  In the second setting module 203, model structures are set in a plurality of piecewise linear models, and their model parameters and hierarchical latent variables are initialized. As an example, the model structure may be a tree structure, and the initialization process may be executed at random.

  According to one exemplary embodiment, the piecewise linear model generation system optimizes hierarchical latent variables based on data for multiple tasks, task correlation latent variables, model structure, and model parameters. A hierarchical latent variable optimization module 204 configured to do so may further be included. Since hierarchical latent variables are randomly initialized in the second setting module 203, they can be optimized in this step. The hierarchical latent variable is for indicating how to divide the data, and the feature space is divided into regions. Each region is represented by a local expert. In the process of dividing into regions by hierarchical latent variables, clustering is performed not by features but by samples. That is, which samples are placed in one area and which samples are placed in other areas. In a specific example, each sample is clustered according to probability. In a specific optimization method, the process represented by the above equation (13) may be employed.

  According to one exemplary embodiment, the piecewise linear model generation system optimizes a task correlation latent variable based on data of multiple tasks, hierarchical latent variables, model structure, and model parameters. A task correlation latent variable optimization module 205 may be further included. Since the task correlation latent variables in the first setting module 102 are generated by random initialization, it is necessary to optimize them. In the task correlation latent variable optimization module 205, the hierarchical latent variable optimized by the hierarchical latent variable optimization module 204 is used together with the data of a plurality of tasks, the model structure, and the model parameters to obtain a task correlation latency. Variables may be optimized. In a specific optimization method, the process represented by the above formula (11) may be adopted.

  According to one exemplary embodiment, the piecewise linear model generation system is configured to perform hard clustering of a plurality of tasks based on the task correlation latent variable and the updated task correlation latent variable. The hard clustering module 206 may be further included. Hard clustering forces a task to belong to only one model, either to belong to this model or to that model. This avoids an increase in space overhead. Thus, the piecewise linear model of the present disclosure according to the present disclosure not only increases prediction accuracy, but also optimizes computational resources. It can therefore be adapted to large data scenarios. In a specific optimization method, the process represented by the above equation (12) may be employed.

  According to one exemplary embodiment, the model optimization module 207 optimizes the hierarchical latent variables optimized by the hierarchical latent variable optimization module 204 and the task correlation latent variable optimization module 205. Task correlation latent variables may be used to optimize the model structure and model parameters along with data for multiple tasks. In a specific optimization method, the process represented by the above formula (14) may be employed.

  According to one exemplary embodiment, the piecewise linear model generation system includes a determination module 208 configured to determine whether the model structure and model parameters output by the model optimization module 207 are optimal. Further, if optimal, output module 209 outputs a piecewise linear model; if not optimal, hierarchical latent variable optimization module 204, task correlation latent variable optimization module 205, hardware The clustering module 206 and the model optimization module 207 continue to optimize the model structure and model parameters. The evaluation index for the determination may be a fitting degree between the model and the data. The degree of fitting cannot be increased without an upper limit. On the contrary, there is an upper limit. In incremental processing, the increase due to continuing optimization increases to some extent and then becomes very small. If the increase is small enough, an optimal model is considered to have been derived. Therefore, optimization is stopped and the output module 209 outputs the resulting piecewise linear model.

  The piecewise linear model generation system shown in FIG. 2 corresponds to the piecewise linear model generation method shown in FIG. Accordingly, the above description of the piecewise linear model generation method according to FIG. 1 applies to the piecewise linear model generation system shown in FIG. 2 as well, and will not be described in detail here.

  FIG. 3 schematically illustrates the steps of a piecewise linear model generation method according to one embodiment of the present disclosure. As shown in FIG. 3, first, the above figure shows input data T1, T2,. Here, the data of each task is represented by separated data points. Thereafter, in the middle figure, modeling is performed using the optimization steps shown in FIG. Each task is understood to be used to model only one model, for example, T1 and T4 may each be used to model model HME1, and T2 models model HME2. T3 may be used to model the model HME3. Finally, a plurality of optimized models, such as a hierarchical mixture of experts, HME (TI), HME (T2),..., HME (TN) are output in the diagram below.

  According to the present invention, in order to facilitate the generation of piecewise linear models, new latent variables are added during the model generation process to represent the correlation between tasks, thus modeling the data of multiple tasks. To increase the accuracy of model generation. Also, since each task is set in association with only one piecewise linear model, the piecewise linear model generation system and generation method according to the present disclosure can process larger data.

  It should further be noted that embodiments of the present invention may be implemented in software, hardware, or a combination thereof. The hardware portion can be implemented with special logic, and the software portion can be stored in memory and executed by a suitable instruction execution system such as a microprocessor or specially designed hardware. The method and system may comprise computer-executable instructions and / or a bearer medium such as, for example, a magnetic disk or CD or DVD-ROM, or a read-only memory (firmware) or a data bearer such as an optical or electronic signal bearer. Those of ordinary skill in the art will appreciate that the code may be implemented with control code included in the processor, such as code supplied in a programmable memory such as The device and its constituent areas in this embodiment include, for example, very large scale integrated circuits or gate arrays, semiconductors such as logic chips or transistors, or programmable hardware such as field programmable gate arrays. A hardware electrical circuit configuration such as a device, or a programmable logic device, or may be implemented by software executed by various types of processors, or the hardware electrical circuit configuration described above, eg, firmware It may be realized by a combination with software.

  The operations of the method according to the invention are represented in a particular order in the drawings, which means that they must be performed in that particular order, or all the desired results have been shown. It does not require or include that the above operations can only be accomplished. Rather, the steps shown in the flow diagram may be performed in a different order. Additionally or alternatively, some steps may be omitted and / or multiple steps may be combined into one step for execution and / or one step for execution A step may be decomposed into a plurality of steps.

  Although the present invention has been described in terms of presently contemplated embodiments, it should be understood that the invention is not limited to the disclosed embodiments. On the contrary, the invention is intended to cover various modifications and equivalent permutations falling within the spirit and scope of the appended claims. The scope of the appended claims is consistent with the broadest interpretation and covers all such modifications and equivalent structures and functions.

(Appendix 1)
An acquisition module configured to acquire data for multiple tasks;
A first setting module configured to set a task correlation latent variable representing the correlation between the plurality of tasks;
Including
Each task is correlated to a piecewise linear model,
A second setting module configured to set a model structure for a plurality of piecewise linear models and initialize corresponding model parameters and hierarchical latent variables;
A model optimization module configured to optimize the model structure and the model parameters based on the data of the plurality of tasks, the task correlation latent variables, and the hierarchical latent variables;
A piecewise linear model generation system including

(Appendix 2)
The piecewise correlation variable of claim 1, wherein the task correlation latent variable is configured to indicate a correspondence relationship between each task of the plurality of tasks and each piecewise linear model of the plurality of piecewise linear models. Linear model generation system.

(Appendix 3)
The piecewise linear model generation system according to attachment 1, wherein the model structure includes a tree structure.

(Appendix 4)
A hierarchical latent variable optimization module configured to optimize the hierarchical latent variable based on the data of the plurality of tasks, the task correlation latent variable, the model structure, and the model parameter. The piecewise linear model generation system according to claim 1, further comprising:

(Appendix 5)
A task correlation latent variable optimization module configured to optimize the task correlation latent variable based on the data of the plurality of tasks, the hierarchical latent variable, the model structure, and the model parameter; The piecewise linear model generation system according to appendix 1, further including:

(Appendix 6)
Hard clustering configured to perform hard clustering of the plurality of tasks based on the task correlation latent variable and the updated task correlation latent variable after optimizing the task correlation latent variable The piecewise linear model generation system according to appendix 5, further including a module.

(Appendix 7)
The piecewise linear model generation system of claim 1, further comprising a determination module configured to determine whether the model structure and the model parameters are optimal.

(Appendix 8)
The piecewise linear model generation system according to appendix 7, wherein the determination module determines whether or not the model structure and the model parameter are optimal based on a fitting degree between the model structure and the model parameter.

(Appendix 9)
Getting data for multiple tasks,
Setting a task correlation latent variable representing the correlation between the plurality of tasks;
Including
Each task is correlated to a piecewise linear model,
Setting up model structures for multiple piecewise linear models and initializing corresponding model parameters and hierarchical latent variables;
Optimizing the model structure and the model parameters based on the data of the plurality of tasks, the task correlation latent variables, and the hierarchical latent variables;
A piecewise linear model generation method including:

(Appendix 10)
The piecewise piecewise variable of claim 9, wherein the task correlation latent variable is configured to indicate a correspondence relationship between each task of the plurality of tasks and each piecewise linear model of the plurality of piecewise linear models. Linear model generation method.

(Appendix 11)
The piecewise linear model generation method according to appendix 9, wherein the model structure includes a tree structure.

(Appendix 12)
The piecewise linear of clause 9, further comprising optimizing the hierarchical latent variable based on the data of the plurality of tasks, the task correlation latent variable, the model structure, and the model parameter. Model generation method.

(Appendix 13)
The piecewise linear of clause 9, further comprising optimizing the task correlation latent variable based on the data of the plurality of tasks, the hierarchical latent variable, the model structure, and the model parameter. Model generation method.

(Appendix 14)
14. The method of claim 13, further comprising performing hard clustering of the plurality of tasks based on the task correlation latent variable and the updated task correlation latent variable after optimizing the task correlation latent variable. A piecewise linear model generation method.

(Appendix 15)
The piecewise linear model generation method according to appendix 9, further comprising determining whether or not the model structure and the model parameters are optimal.

(Appendix 16)
The determination as to whether or not the model structure and the model parameter are optimal is based on a fitting degree between the model structure and the model parameter, whether or not the model structure and the model parameter are optimal The piecewise linear model generation method according to supplementary note 15, including:

Claims (9)

An acquisition module configured to acquire data for multiple tasks;
A first setting module configured to set a task correlation latent variable representing the correlation between the plurality of tasks;
Including
Each task is correlated to a piecewise linear model,
A second setting module configured to set a model structure for a plurality of piecewise linear models and initialize corresponding model parameters and hierarchical latent variables;
A model optimization module configured to optimize the model structure and the model parameters based on the data of the plurality of tasks, the task correlation latent variables, and the hierarchical latent variables;
A piecewise linear model generation system including   The category of claim 1, wherein the task correlation latent variable is configured to indicate a correspondence relationship between each task of the plurality of tasks and each piecewise linear model of the plurality of piecewise linear models. Linear model generation system.   The piecewise linear model generation system according to claim 1, wherein the model structure includes a tree structure.   A hierarchical latent variable optimization module configured to optimize the hierarchical latent variable based on the data of the plurality of tasks, the task correlation latent variable, the model structure, and the model parameter. The piecewise linear model generation system according to claim 1, further comprising:   A task correlation latent variable optimization module configured to optimize the task correlation latent variable based on the data of the plurality of tasks, the hierarchical latent variable, the model structure, and the model parameter; The piecewise linear model generation system according to claim 1, further comprising:   Hard clustering configured to perform hard clustering of the plurality of tasks based on the task correlation latent variable and the updated task correlation latent variable after optimizing the task correlation latent variable 6. The piecewise linear model generation system of claim 5, further comprising a module.   The piecewise linear model generation system of claim 1, further comprising a determination module configured to determine whether the model structure and the model parameters are optimal.   The piecewise linear model generation system according to claim 7, wherein the determination module determines whether the model structure and the model parameter are optimal based on a fitting degree between the model structure and the model parameter. Getting data for multiple tasks,
Setting a task correlation latent variable representing the correlation between the plurality of tasks;
Including
Each task is correlated to a piecewise linear model,
Setting up model structures for multiple piecewise linear models and initializing corresponding model parameters and hierarchical latent variables;
Optimizing the model structure and the model parameters based on the data of the plurality of tasks, the task correlation latent variables, and the hierarchical latent variables;
A piecewise linear model generation method including:

Images