JP2012173793A - Predictor selection device, predictor selection method, and predictor selection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To select an optimum predictor for an asymmetric similarity relationship.SOLUTION: A cluster representation DB10 stores a feature representation for each training cluster. A transformation matrix DB40 stores a transformation matrix for maximizing asymmetric similarity for each set of training clusters after transformation of the feature representation of DB10. A similarity calculation section 50 uses the transformation matrix in DB40 to transform a feature representation of an input test cluster. Similarity between the feature representation of the transformed test cluster and each feature representation stored in DB10 is computed, and an ID of a training cluster with maximum similarity is sent to a predictor selection section 70. The predictor selection section 70 selects a predictor generated by the cluster of the received ID from a predictor DB60 and outputs the selected predictor.

Description

  The present invention relates to a technique for performing prediction (for example, classification or ranking) using information that can be expressed by a high-dimensional vector such as text or an image.

  As is well known, a predictor (model) generated for each cluster is used to express data such as text and images as a high-dimensional vector and predict a class. In that case, the training data having the same characteristics are grouped into one cluster (training cluster) to divide the training data into multiple clusters, and a predictor is generated using the training data contained in each cluster. A prediction method that selectively uses an appropriate predictor for the input test data group (test cluster) is effective.

  In this method, as a method of selecting a cluster that has generated a predictor, as shown in FIGS. 4 and 5, the performance of each model cannot be evaluated for the test cluster, so the training cluster having the maximum similarity to the test cluster is used. Is selected, and a model generated by the selected training cluster is applied to the test cluster.

  Conventionally, as shown in Non-Patent Document 1, the training data is used to convert a feature space that maximizes the similarity of the cluster that generates a model that shows the best performance for the text cluster. A transformation matrix is generated to improve training cluster selection accuracy.

Eric P. Xing, Andrew Y. Ng, Michael I. Jordan, Stuart Russell ”Distance Metric Learning with Application to Clustering with Side-Information” Proceedings of the 16th annual conference on Neural Information Processing Systems (NIPS '02), pp.505 ≡512, 2002.

  However, since the optimal predictor is evaluated by the performance of the predictor with respect to data included in a given cluster, a predictor MB that performs optimal prediction of the test cluster CD is generated as shown in FIG. The predictor MD generated from the test cluster CD is not always optimal for the training cluster MB. In the predictor selection in such a case, the similarity between clusters is not symmetric.

  However, conventionally, since a symmetric similarity distance measure is used as the similarity between clusters as in Patent Document 1, the similarity between clusters cannot be optimized for an asymmetric similarity. The optimal predictor may not be selected for the test cluster.

  The present invention has been made in order to solve the above-described problems of the prior art, and an object of the present invention is to provide a technique capable of selecting an optimal predictor for an asymmetric similarity relationship.

  Therefore, the present invention generates a transformation matrix that maximizes the asymmetric similarity based on the training data in advance, and selects a predictor of the input test cluster using the generated transformation matrix.

  One aspect of a predictor selection device according to the present invention includes a cluster expression database that stores feature expressions of training clusters, a transformation matrix database that stores transformation matrices that maximize asymmetric similarity between training clusters, and a test cluster. The feature expression is converted using the transformation matrix stored in the transformation matrix database, and the similarity between the converted feature representation of the test cluster and each feature representation saved in the cluster representation database is calculated, and the calculated similarity The similarity calculation means that identifies the training cluster with the largest training cluster as the test cluster and the predictor generated from the cluster specified by the similarity calculation means is selected, and the selected prediction is selected. Predictor selection means for outputting the output.

  Another aspect of the predictor selection apparatus according to the present invention includes a cluster expression database that stores a feature expression of a training cluster, an optimal information database that stores a training cluster that generates an optimal predictor for each training cluster as a cluster set, and A transformation matrix generating means for obtaining a feature representation of the cluster set of the optimal information database from the cluster representation database, and generating a transformation matrix that maximizes the asymmetric similarity of the cluster set after transforming the feature representation of one training cluster; The feature expression of the test cluster is converted by the conversion matrix generated by the conversion matrix generation means, and the similarity between the converted feature expression of the test cluster and each feature expression stored in the cluster expression database is calculated and calculated. Similarity that identifies the training cluster with the highest similarity to the cluster that produces the best predictor for the test cluster Comprising computing means, to select a predictor which is generated from the identified clusters by similarity calculation means, a predictor selecting means for outputting the selected predictor, the.

  According to one aspect of the predictor selection method of the present invention, a feature expression of a test cluster is transformed with a transformation matrix that maximizes asymmetric similarity between training clusters stored in a transformation matrix database, and a transformed test is performed. The similarity calculation between the feature expression of the cluster and each feature expression stored in the cluster expression database is calculated, and the training cluster having the maximum similarity is identified as the cluster that generates the best predictor for the test cluster. And a predictor selection step of selecting a predictor generated from the identified cluster and outputting the selected predictor.

  In another aspect of the predictor selection method according to the present invention, the feature information of the cluster set of the optimal information database that stores the training cluster that generates the optimal predictor for each training cluster is stored as the cluster set, and the feature expression of the training cluster. A transformation matrix generation step that generates a transformation matrix that maximizes the asymmetric similarity of a cluster set after transforming the feature representation of one training cluster obtained from the saved cluster representation database, and testing with the generated transformation matrix The cluster feature representation is converted, and the similarity between the converted test cluster feature representation and each feature representation stored in the cluster representation database is calculated, and the training cluster with the largest calculated similarity is optimal for the test cluster. A similarity calculation step for identifying a cluster for generating a predictor, and a predictor generated from the identified cluster. And-option, having a predictor selecting step of outputting the selected predictor.

  In selecting the predictor, the predictor can be selected from a predictor database that stores the predictors generated from each training cluster. In addition, this invention is good also as an aspect of the program which makes a computer function as said each apparatus. This program can be provided through a network or a recording medium.

  According to the present invention, an optimal predictor can be selected for an asymmetric similarity relationship.

The block diagram of the predictor selection apparatus which concerns on embodiment of this invention. The block diagram of the same transformation matrix production | generation apparatus. The flowchart which shows the process of the similarity calculation part. Explanatory drawing 1 which shows selection of the predictor produced | generated for every cluster. Figure 2 Explanatory drawing which shows the state in which the similarity between clusters is asymmetrical.

  A predictor selection apparatus according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2. The selection device 1 includes a transformation matrix generation device 2 that generates a transformation matrix that maximizes the similarity with respect to asymmetric similarity, and is generated by the transformation matrix generation device 2 for class prediction. The optimal predictor of the input test cluster is selected using the transformation matrix.

  Specifically, the selection device 1 is configured by a computer, and hardware resources of a normal computer such as a CPU. Memory (RAM). Includes a hard disk drive device. As a result of the cooperation between the hardware resource and the software resource (OS. Application, etc.), the selection device 1 has a cluster representation DB 10, an optimal predictor information DB 20, a transformation matrix generation unit as shown in FIGS. (Corresponding to the transformation matrix generation device 2) 30, a transformation matrix DB 40, a similarity calculation unit 50, a predictor DB 60, and a predictor selection unit 70 are implemented.

  Here, it is assumed that each DB 10.20.40.60 is constructed in a storage device such as a memory (RAM) or a hard disk drive device. According to each of the units 10 to 70, a transformation matrix creation stage that saves the transformation matrix created in advance, and a predictor that maximizes the similarity of the text clusters input in the saved transformation matrix. A predictor selection stage to select is executed.

  That is, the cluster representation DB 10 stores feature representations (feature vectors) of training clusters, and the optimal predictor information DB 20 stores training cluster sets, that is, clusters that generate optimal predictors for each training cluster in pairs. Yes. At this time, in the transformation matrix creation stage, as shown in FIG. 2, the transformation matrix generation unit 30 obtains the feature expression stored in the cluster expression database of the cluster set of the DB 20, and after one feature conversion, Generate a transformation matrix that maximizes the asymmetric similarity. The generated transformation matrix is stored in the transformation matrix DB 40.

  Further, in the predictor selection stage, the similarity calculation unit 50 converts the inputted feature expression of the test cluster with the conversion matrix of the DB 40 as shown in FIG. The similarity between the converted feature expression of the test cluster and each feature expression stored in the DB 10 is calculated. The training cluster having the maximum calculated similarity is identified as the cluster that generates the optimal predictor for the test cluster.

  The predictor selection unit 70 selects a predictor from the DB 60 that stores the parameters of the predictor generated from each training cluster. That is, referring to the DB 60, the predictor generated from the cluster specified by the similarity calculation unit 50 is selected, and the selected predictor is output. Details of each stage will be described below.

≪Transformation matrix generation stage≫
The transformation matrix generation unit 30 refers to the DB 10.20 and receives the stored data as an input. Table 1 shows an example of the data structure of the DB 10. Here, “c ₁ , c ₂ ,..., C _N ” indicates the cluster ID of the training cluster, and each row indicates the characteristic expression of the training cluster. In the data structure of Table 1, the training feature vector features of M-dimensional of each cluster "x _1, x _2, ..., x _M" is represented by, i-th row and j-th column of values training cluster c The j-th feature value of _i is shown.

  Table 2 shows an example of the data structure of the DB 20, and the test cluster ID indicates the ID of the training cluster imitated by the test cluster. That is, a training cluster that simulates a certain training cluster as a test cluster and generates a predictor showing the best prediction performance among the other training clusters is selected as the optimal predictor generation cluster. The ID of the training cluster selected here is described for each cluster ID of the training cluster imitated by the test cluster.

  Here, when receiving the storage data of the DB 10.20 as shown in Table 1.2 as an input, the transformation matrix generation unit 30 receives the cluster set “test cluster (ID): optimal predictor generation cluster ( ID) ”is obtained from the stored data of the DB 10. A transformation matrix that maximizes the similarity of the cluster set “test cluster (ID): optimal predictor generation cluster (ID)” after the transformation of the characteristic representation of the test cluster acquired here is generated, and the generated transformation The matrix is stored in the DB 40 (conversion matrix generation step).

  This similarity means an asymmetric similarity as shown in FIG. 6, and for example, generalized Kullback library divergence “Generalized Kullback-Leibler divergence (gkld)” can be used. This generalized Kullback library divergence is given by equation (1).

  Here, since “0 ≧ p, q” is necessary, the feature expression y of the test cluster (training cluster simulated by the test cluster) with respect to the feature expression x of the training cluster is expressed as “p = exp (x), q = By setting “exp (y)”, any real value can be handled. At this time, the feature expression y of the test cluster with respect to the feature expression x of the training cluster can be calculated according to the equation (2).

The similarity after the expression (2) is subjected to the characteristic conversion “y → y ^T w _i ” of the feature expression of the test cluster using the conversion matrix W can be calculated by the expression (3). Here, assuming that the square of “G (x, y)” is a loss function, an estimated value of “w _i ” can be obtained according to Equation (4).

  For the calculation of the estimated value, a non-linear optimization method such as a steepest descent method, a Newton method, or a BFGS “Broyden-Fletcher-Goldfarb-Shanno” method can be used using the gradient information of the loss function. Expressions (1) to (4) are defined in a program or the like.

≪Predictor selection stage≫
(1) Similarity calculation unit 50
The similarity calculation unit 50 receives the input text cluster and the stored data of the DB 10.40 as inputs. Here, the feature expression of the input test cluster is converted using the conversion matrix of the DB 40, and the similarity with the training cluster feature of the DB 10 is calculated. A training cluster having the maximum similarity is searched from the DB 10, and the searched training cluster is output as a cluster for generating an optimal predictor.

The processing contents will be described in detail with reference to FIG. 3. When a test cluster is input to an input unit (not shown), the processing of the similarity calculation unit 50 is started. When the process is started, first, the “maximum similarity s _max ” and “optimum cluster c _best ” stored in the memory (RAM) are initialized (S01). This initialization is executed by rewriting “s _max ← 0” and “c _best ← NONE”.

Next, the transformation matrix w _i is obtained from the DB 40, and the feature expression of the input test cluster c _{test is} subjected to feature transformation “y → y ^T w _i ” (S02). After this conversion, it is checked whether or not there is an unprocessed cluster _{ck from} S05 in the stored data of the DB 10 (S03).

If there is an unprocessed cluster c _{k as} a result of this confirmation, the similarity s _k with respect to the cluster c _k of the input test cluster c _test is calculated (S06). Equation (3) may be used to calculate the similarity s _k . It is confirmed whether or not the similarity s _k calculated here is larger than the maximum value “s _max ”, that is, whether or not “s _k > s _max ” is satisfied (S06). On the other hand, if established, the process proceeds to S07. In S07, “maximum similarity s _max ” and “optimum cluster c _best ” stored in the memory (RAM) are rewritten. Here, the “maximum similarity s _max ” is updated to the similarity s _k calculated in S06 “s _max ← s _k ”, and the cluster _ck is stored as a cluster for generating an optimal predictor “c _best ← c _k ".

After the process of S07, the process returns to S03, and S05 to S07 are repeated until there is no unprocessed cluster _{ck in} the stored data of the DB 10, and if the cluster _ck disappears, it is stored in the memory (RAM). The cluster ID of the “optimal cluster c _best ” is output to the predictor selection unit 70, and the process is terminated.

(2) Predictor selection unit 70
When the predictor selection unit 70 receives the cluster ID of “optimum cluster c _best ” from the similarity calculation unit 50, the predictor selection unit 70 refers to the DB 60 and outputs the optimum predictor to a monitor or the like through an output unit (not shown).

Table 3 shows an example of the data structure of the DB 60, and stores the parameters of the predictor generated from each training cluster. Here, the data structure of parameters in the linear model predictor is shown, but the present invention is not limited to this, and predictor information generated using any learning algorithm can be stored in the DB 60. Specifically, in the data structure example of Table 3, “c ₁ , c ₂ ,..., C _N ” indicates the cluster ID of the training cluster, each row indicates the weight for the feature expression of each training cluster, i and the row j-th value indicates the value of the weight for the j-th feature training cluster c _i.

  And the predictor selection part 70 selects the predictor of cluster ID output from the similarity calculation part 50 from the preservation | save data of said DB60, and outputs the selected predictor. The output predictor is used for prediction (classification or ranking) of information that can be expressed by a high-dimensional vector such as text or an image.

  As described above, according to the selection device 1, a transformation matrix that maximizes the similarity of a cluster that generates an optimal predictor for the cluster is generated based on an asymmetric similarity index (for example, general Cullback library divergence). And stored in the DB 40. Based on the stored data, the similarity between the input test cluster and the training cluster of the DB 10 is calculated, and the cluster ID of the training cluster having the maximum similarity is output to the predictor selection unit 70. The accuracy of predictor selection is improved. Thereby, the prediction system of the data included in the input test cluster can be improved.

  In addition, this invention is not limited to the said embodiment, A device structure etc. can be deform | transformed within the range described in each claim. For example, the transformation matrix generation device 2 (transformation matrix generation unit 30) is not necessarily incorporated in the predictor selection device 1, and may be configured as a separate device. In this case, the DB 20.40 may be shared by each device 1.2.

≪Programs≫
The present invention is configured as a document search program that causes a computer to function as a part or all of each section 10.20.30.40.50.60.70 of the predictor selection apparatus 1 (including the transformation matrix generation apparatus 2). You can also According to this program, it is possible to cause a computer to execute part or all of the processing of each stage.

  The program can be provided through a network such as a website or e-mail. The program is stored in a recording medium such as a CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, MO, HDD, BD-ROM, BD-R, or BD-RE. It is also possible to record, save and distribute. This recording medium is read using a recording medium driving device, and the program code itself realizes the processing of the above embodiment, so that the recording medium also constitutes the present invention.

DESCRIPTION OF SYMBOLS 1 ... Predictor selection apparatus 2 ... Transformation matrix production | generation apparatus 10 ... Cluster expression DB (cluster expression database)
20 ... Optimal predictor information DB (cluster expression database)
30: Conversion matrix generation unit (conversion matrix generation means)
40 ... Transformation matrix DB (transformation matrix database)
50. Similarity calculation unit (similarity calculation means)
60 ... Predictor DB (Predictor database)
70: Predictor selection unit (predictor selection means)

Claims (7)

When predicting a class of information that can be represented by a high-dimensional vector, this is a predictor selection device that generates predictors by dividing training data into training clusters and selects an appropriate predictor for the input test cluster. And
A cluster representation database that stores training cluster feature representations;
A transformation matrix database that stores transformation matrices that maximize asymmetric similarity between training clusters;
The feature expression of the test cluster is converted by the conversion matrix stored in the conversion matrix database, and the similarity between the converted feature expression of the test cluster and each feature expression stored in the cluster expression database is calculated and calculated. A similarity calculation means for identifying a training cluster having the maximum similarity as a cluster that generates an optimal predictor for the test cluster;
A predictor selection unit that selects a predictor generated from the cluster specified by the similarity calculation unit, and outputs the selected predictor;
A predictor selection device comprising: When predicting a class of information that can be represented by a high-dimensional vector, this is a predictor selection device that generates predictors by dividing training data into training clusters and selects an appropriate predictor for the input test cluster. And
A cluster representation database that stores training cluster feature representations;
An optimal information database that stores training clusters as cluster sets that generate optimal predictors for each training cluster;
A transformation matrix generating means for obtaining a feature representation of the cluster set of the optimal information database from the cluster representation database, and generating a transformation matrix that maximizes the asymmetric similarity of the cluster set after transforming the feature representation of one training cluster;
The feature expression of the test cluster is converted by the conversion matrix generated by the conversion matrix generation means, and the similarity between the converted feature expression of the test cluster and each feature expression stored in the cluster expression database is calculated and calculated. A similarity calculation means for identifying a training cluster having the maximum similarity as a cluster that generates an optimal predictor for the test cluster;
A predictor selection unit that selects a predictor generated from the cluster specified by the similarity calculation unit, and outputs the selected predictor;
A predictor selection device comprising: A predictor database for storing predictors generated from each training cluster;
3. The predictor selection unit according to claim 1, wherein the predictor selection unit selects, from the predictor database, a predictor generated from the cluster specified by the similarity calculation unit. apparatus. When predicting a class of information that can be represented by a high-dimensional vector, a predictor is executed by a device that generates a predictor by dividing training data into training clusters and selects an appropriate predictor for the input test cluster. A selection method,
The test cluster feature representation is transformed with a transformation matrix that maximizes the asymmetric similarity between the training clusters stored in the transformation matrix database, and the transformed test cluster feature representation and each saved in the cluster representation database Calculating the degree of similarity with a feature representation and identifying the training cluster with the largest degree of similarity as the cluster that generates the best predictor for the test cluster;
A predictor selection step of selecting a predictor generated from the identified cluster and outputting the selected predictor;
A predictor selection method characterized by comprising: When predicting a class of information that can be represented by a high-dimensional vector, a predictor is executed by a device that generates a predictor by dividing training data into training clusters and selects an appropriate predictor for the input test cluster. A selection method,
The feature information of the cluster set of the optimal information database that stores the training cluster that generates the optimal predictor for each training cluster as the cluster set is obtained from the cluster expression database that stores the feature representation of the training cluster, A transformation matrix generating step that generates a transformation matrix that maximizes the asymmetric similarity of the cluster set after transforming the feature representation;
The feature expression of the test cluster is converted by the generated conversion matrix, the similarity between the converted feature expression of the test cluster and each feature expression stored in the cluster expression database is calculated, and the calculated similarity is A similarity calculation step that identifies the largest training cluster as the cluster that produces the best predictor for the test cluster;
A predictor selection step of selecting a predictor generated from the identified cluster and outputting the selected predictor;
A predictor selection method characterized by comprising: The predictor selection step selects a predictor generated from the cluster specified in the similarity calculation step from a predictor database storing a predictor generated from each training cluster. 6. The predictor selection method according to any one of 5 above.   The predictor selection program which makes a computer function as each means of the predictor selection apparatus of any one of Claims 1-3.

Images