JP5429940B2 - Method and system for building multilevel classification models - Google Patents

Description

  The present invention relates to information classification, and more particularly to multi-class classification and multi-level classification for classifying information samples into a number of categories or classes. More specifically, the present invention relates to a method and system for building a multi-level classification model.

  In conventional multi-class information classification methods, classes are often independent of each other and disordered. For example, in the news classification, the news class includes politics, economics, military, science and the like.

  However, there are other special types of multi-class problems in our real life. Each class is connected regularly and distributed smoothly. This type of classification problem is called a multi-level classification problem. Also, in such a problem, information samples are classified into different levels instead of different classes. For example, in the product evaluation classification, the user's evaluation opinion level includes bad, normal, good, and very good.

  Increasing information on the Internet makes information classification requirements increasingly significant. For this reason, in recent years, much research has been conducted on the conventional multi-class classification problem. However, as a special kind of multiclass problem, the multilevel classification problem has not been well studied. Some examples of existing automatic information classification algorithms relevant to the present invention are briefly introduced below.

  First, a non-patent document 1 (the paper entitled “New Approaches to Support Vector Ordinary Regression” by Wei Chu and S. Satya Keerthy (see ICML 2005, pages 145-152)) is an ordinal regression (ordinary). Two supervised support vector approaches are proposed. Here, a number of thresholds are optimized to limit parallel classification hyperplanes for ordinal scales. See “7. Detailed Description of the Invention” for more details.

Furthermore, US Pat. No. 7,533,076 B2 (hereinafter referred to as Patent Document 1) filed on Mar. 17, 2008 proposes an effective multi-class support vector machine classification method. This method classifies data samples into multiple categories by using a set of supervised binary support vector machine classifiers. In the process of building the classification model, the method adjusts the initial classification model based on the local adjacency between adjacent levels.
FIG. 1 shows a configuration block diagram of a system 100 for generating and optimizing a classification model according to Patent Document 1.
In FIG. 1, the system 100 mainly includes a classification model initialization unit 101 and a local level adjacency-based classification model adjustment unit 102. Information classification requires that the classification model be represented as some sort of computer readable format. For example, in this method, the multi-level classification model consists of a series of parallel classification hyperplanes with corresponding level thresholds that indicate the margin between adjacent levels. At the beginning of the learning of the classification model, it is first necessary to generate the first model.
The classification model initialization unit 101 is used to generate an initial classification model based on the input labeled training data. The method of generating the initial classification model is a method well known to those skilled in the art and will not be described in detail here.
The adjustment means 102 is used to adjust and optimize the initial classification model generated based on the local level adjacency between levels. In multilevel problems, levels are related to order. That is, the closer the levels, the more similar they are.
Therefore, in this method, this relationship is expressed as follows.
The i th level threshold must be lower than the (i + 1) th level threshold.
The local level adjacency representing the level adjacency is used to adjust the initial classification model generated, thereby obtaining an optimized classification model.

As another example, P.I. N. M.M. Belkin and V.W. Sindhwani, “Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Samples, No. 24, Journal of Machines, No. 23, Journal of Machines. It shows how to learn a series of semi-supervised multi-class classification models.
This method is a typical semi-supervised learning method. The central component is “smoothing of a classification model based on data sample similarity”.
FIG. 2 shows a block diagram of a system 200 for generating and optimizing a classification model according to Non-Patent Document 2. In FIG. 2, the system 200 includes a classification model initialization unit 201 and a data sample similarity-based classification model smoothing unit 202. In the semi-supervised learning scenario, most data samples are unlabeled, ie their category labels are unknown. However, the presence of these unlabeled training data samples can be used to better shape the overall data distribution specific geometry.
It is based on the hypothesis that similar data samples are likely to be in the same category. The similarity of data samples can be calculated based on the characteristics of the data samples.
The data sample similarity is then used to adjust the predicted categories for unlabeled and labeled data samples. As a result, these labels change smoothly according to the overall data distribution and the goal of optimizing the classification model is achieved.

US Patent US7533076B2

Wei Chu, S.M. Sathya Keerthi, “New Approaches to Support Vector Ordinal Regression”, ICML, 2005, p. 145-152 P. N. M.M. Belkin, V.M. Sindhwani, “Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples, Journal of Machine 6”. 2399-2434 M.M. Qian, F.M. Nie, C.I. Zhang, “Probabilistic labeled semi-supervised svm. In Works on Optimized Bass Methods for Emerging Data Mining Problems”, IEE.

In the above-described multiclass classification model, the order relation of pairs between levels in the case of multilevel is not considered. Regardless, the multilevel classification model optimization method in the related art still has many disadvantages that cannot be avoided.
In the ordinal regression model optimization method of the prior art 1, only limited order relations between adjacent levels are considered. Further, the output level labels in the ordinal regression model are discrete and cannot be measured continuously.
Furthermore, there are no existing multilevel or ordinal regression methods that can be applied, for example, in the prior art 2 semi-supervised learning scenario.

  The present invention has been made to solve the existing problems in the related art described above.

  The multilevel classification model optimization method of the present invention includes four main classification model optimization functions respectively applied to supervised and semi-supervised scenarios. That is, based on the classification model adjustment function (first adjustment) based on the distance of the global level value, the classification model adjustment function (second adjustment) based on the order relationship of the data samples, and the similarity between the data samples The classification model smoothing function (first smoothing) and the classification model smoothing function (second smoothing) based on the similarity between levels. The first adjustment and the second adjustment are mainly applied to the supervised scenario, i.e. based on labeled data samples. Also, the first smoothing and the second smoothing are introduced on the basis of the first adjustment and the second adjustment, and can be used for a model applied to a semi-supervised scenario. That is, based on both labeled data samples and unlabeled data samples.

  The first adjustment (that is, the classification model adjustment based on the distance of the global level value) is a function of adjusting the classification model based on the global relation of all classification levels. Unlike the case where it is limited to the local order between adjacent levels as described in Related Art 1 above, the first adjustment function is to reduce the gap between the level values to punish global misclassification errors. Use. As a result, global order relationships at all levels are introduced in the process of adjusting the classification model. Thus, global misclassification errors on all data samples are minimized and the global level order relationship of the classification model is also optimized macroscopically.

The second adjustment (that is, the classification model adjustment based on the order relationship of the data samples) is designed in consideration of the following.
Based on the nature of the multi-level order, the order between the levels is also reflected in each data sample.
Therefore, the second adjustment function aims to maintain the order relation between the two level-equipped data so as to coincide with the level order relation corresponding to the level labels.
Thus, the level order relationship is further optimized microscopically according to the data samples.

The first smoothing (that is, the classification model smoothing based on the similarity between data samples) is similar to an existing method, for example, the method described in the related art 2 described above.
In the present invention, the prediction levels of both labeled and unlabeled data samples are smoothed based on their similarity.

The second smoothing (that is, the classification model smoothing based on the similarity between levels) is a function of adjusting the classification model according to the level similarity of the data samples.
The first smoothing function based on the similarity between data samples uses only the geometric structure unique to the data distribution.
The second smoothing is used to model the level distribution specific geometric structure.
Based on the distance between the levels and the probability that the data sample belongs to the level, the level similarity between the data samples is calculated.
Therefore, the classification model is further optimized according to the level similarity of the data samples, so that the predicted level label not only changes smoothly over the entire data distribution, but also matches the order relationship of the levels.

  According to another embodiment, the present invention optimizes the multi-level classification model by utilizing different combinations of the first adjustment, the second adjustment, the first smoothing and the second smoothing described above. Turn into.

A method for constructing a multi-level classification model according to the present invention includes inputting labeled data samples, generating an initial multi-level classification model using the leveled data samples, and generating an initial multi-level classification model And optimizing the initial multi-level classification model based on the global level value distance between all levels.
In another aspect, the optimization step further includes adjusting the initial multi-level classification model based on an ordered relationship between labeled data samples.
In yet another aspect, the optimization step includes smoothing the initial multi-level classification model based on the similarity between data samples and the similarity between levels.

A system for constructing a multilevel classification model according to the present invention includes a first input means for inputting labeled data samples, and a classification model initialization means for generating an initial multilevel classification model using the leveled data samples. And classification model optimizing means for optimizing the generated first multi-level classification model.
According to another aspect, the classification model optimization means is configured to adjust the initial multi-level classification model based on the global level value distance between all levels.
According to yet another aspect, the classification model optimization means is configured to adjust the initial multi-level classification model based on a global level value distance between all levels and an order relationship between data samples. The
According to another aspect, the method is extended to a semi-supervised scenario. In this case, in addition to adjusting the initial multi-level classification model based on the global level value distance between all levels and the order relationship between the data samples, the classification model optimization means further includes: The initial multi-level classification model is configured to be smoothed based on the similarity and the similarity between levels, thereby achieving final optimization.

The technical effects of the present invention are as follows.
First, by utilizing the level value and the similarity, the order relation between the levels is well connected in the classification model, thereby improving the accuracy of multi-level classification.

Furthermore, since a unified classification function is constructed for all levels, each data sample is calculated with the same measure.
Therefore, the output of the classifier is not only a discrete level label, but also a continuous measure value that quantizes the level label, and as a result, the output of the multilevel classifier can be measured.

  In addition, with the introduction of two smoothing functions, the classification model can be applied to both previous supervised problems and semi-supervised problems that are more common in real-world information processing applications. Improves the practical use of level classification.

The present invention may be better understood from the following detailed description of embodiments with reference to the accompanying drawings. The same reference numbers indicate the same or similar parts.
1 is a configuration block diagram showing a classification model optimization system 100 according to Related Technology 1. FIG. It is a block diagram which shows the classification model optimization system 200 by the related technique 2. FIG. 1 is a block diagram illustrating a classification model optimization system 300 according to the present invention. FIG. 4 is a flowchart illustrating one of three different combinational operation modes of the classification model optimization system 300 shown in FIG. FIG. 4 is a flowchart illustrating one of three different combinational operation modes of the classification model optimization system 300 shown in FIG. FIG. 4 is a flowchart illustrating one of three different combinational operation modes of the classification model optimization system 300 shown in FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
First, in order to make the explanation easy to understand, definitions of some basic symbols used in the explanation are shown below.
The l labeled data samples are denoted as X ^L = {(x _i , y _i )}, i = 1,. . . , L.
The u unlabeled data samples are denoted as X ^U = {(x _i ,?)}, i = 1 + 1,. . . , L + u.
Here, y∈ {r _k }, k = 1,. . . Is K, r _k is the value of the k-th class, K is the total number of classes.

Y _R = {y _i }, i = 1,. . . , L denote class label vectors of l labeled data samples.
The present invention constructs a uniform prediction or classification function f called a level function. The output of the function for data sample x is a numerical value f (x), that is, the value of the level to which x belongs. The optimized level function is denoted f ^* .

The present invention proposes a classification model optimization method for constructing multi-level classification models. This method combines an ordered relationship between levels into a classification model. The method proposed by the present invention can be applied to supervised and semi-supervised cases.
As an example, the unified structure can be represented by the following equation (1).
f ^* = arg _f min || f || ² _K + r ₁ V _Level (f, Y _R ) + r ₂ V _Order (f) + r ₃ (1-α) || f || ² _I + r ₃ α || f || ² _L (1)
Here, || f || ² _K is a basic optimization term of the level function f. r ₁ , r ₂ , r ₃ , and α are parameters for adjusting the effects of the terms corresponding to the first, second, third, and fourth constituent elements of the present invention, respectively.
As described above, the present invention mainly includes four optimization functions. That is, the classification model adjustment (first adjustment) based on the distance of the global level value, the classification model adjustment (second adjustment) based on the order relationship between the data samples, and the similarity between the data samples Classification model smoothing (first smoothing), classification model smoothing (second smoothing) based on similarity between levels.
In equation (1), they correspond to V _Level (f, Y _R ), V _Order (f), || f || ² _I and || f || ² _L , respectively.
Here, for convenience of explanation, each term will be described using a sum of squares as an example. However, the principle of the present invention is not limited to this example, and can be extended to various mathematical sums (for example, sum of absolute values or sum of high powers).

(1) V _Level (f, Y _R ) is a kind of loss function based on labeled data that ensures that the learned level function approaches the true level of the data. Here, Y _R is the expected level output vectors of all the data.
This term corresponds to the first adjustment of the present invention (classification model adjustment function based on a global level value distance).
(2) V _Order (f) is a kind of loss function that holds the order relationship between labeled data samples so as to match the order relationship of those level labels.
This term corresponds to the second adjustment (classification model adjustment function based on the order relationship between data samples) of the present invention.
(3) || f || ² _I is a smooth adjustment term based on the data sample similarity that reflects the geometric structure of the data sample distribution, whereby the level function f changes smoothly according to the data distribution To do.
This term corresponds to the first smoothing (classification model smoothing function based on similarity between data samples) of the present invention.
(4) || f || ² _L is a smooth adjustment term based on the level similarity reflecting the geometric structure of the level, whereby the level function f changes smoothly according to the level order distribution. .
This term corresponds to the second smoothing (classification model smoothing function based on similarity between levels) of the present invention.

  FIG. 3 shows a block diagram of a classification model optimization system 300 according to the present invention. 4A to 4C respectively show flowcharts of three different combination processing methods of the classification model optimization system 300 shown in FIG.

As shown in FIG. 3, the system 300 includes a classification model initialization unit 301 and a classification model optimization unit 302.
The classification model optimization unit 302 realizes one or more combinations of four classification model optimization functions.
As described above, the four classification model optimization functions are classified model adjustment (first adjustment) 3021 based on the distance of the global level value, and classification model adjustment (second adjustment) based on the order relationship between the data samples. 3022), classification model smoothing (first smoothing) 3023 based on similarity between data samples, and classification model smoothing (second smoothing) 3024 based on similarity between levels. .
As described above, the first adjustment and the second adjustment can be applied to a supervised scenario, and the first smoothing and the second smoothing can be applied to a semi-supervised scenario.
According to different embodiments of the present invention, the optimization of the multi-level classification model can be realized by different combinations of the first adjustment, the second adjustment, the first smoothing and the second smoothing. is there.
4A to 4C, the following applications are shown as examples.
It should be noted that the principles of the present invention are not limited to the following combinations, and that functions can be selected and combined by those skilled in the art according to the application requirements.

Application form 1 (FIG. 4A): first adjustment Application form 2 (FIG. 4B): first adjustment + second adjustment Application form 3 (FIG. 4C): first adjustment + second adjustment + first adjustment Smoothing + Second Smoothing FIGS. 4A and 4B can be applied to supervised scenarios. FIG. 4C can be extended to a semi-supervised scenario.

The process of FIG. 4A starts from step 401a.
In step 401a, the user inputs a labeled data sample set.
Thereafter, in step 402a, the classification model initialization unit 301 generates an initial multilevel classification model.
Here, the initial multi-level classification model can be generated by utilizing any existing method known in the art.
In the present invention, the first multi-level classification model consists of a uniform classification hyperplane mapping function f, a series of level values, and several other parameters.
Next, in step 403a, the classification model optimization unit 302 performs the first adjustment for the first classification model.
That is, the classification model optimizing means 302 adjusts the initial classification model based on the global level value distance between all levels.
In step 404a, an optimized multilevel classification model is obtained.

The process in FIG. 4B is similar to the process in FIG. 4A, and only the operation of the classification model optimization unit 302 in step 403b is different.
In the application mode 2 illustrated in FIG. 4B, the classification model optimization unit 302 performs the first adjustment and the second adjustment for the first classification model.
That is, the classification model optimizing means 302 adjusts the classification model based on (1) the distance of the global level value between all levels and (2) the order relationship between the data samples.

FIG. 4C can be applied to a semi-supervised scenario.
In step 401c, the user inputs a labeled data sample set and an unlabeled data sample set.
Thereafter, in step 402c, the classification model initialization unit 301 generates an initial multi-level classification model using the input labeled data sample set and unlabeled data sample set.
In step 403c, the classification model optimization unit 302 performs the first adjustment, the second adjustment, the first smoothing, and the second smoothing on the first classification model.
That is, the classification model optimizing means 302 (1) global level value distance between all levels, (2) order relation between data samples, (3) similarity between data samples, and (4) classification Optimize the classification model based on the similarity between levels.
Thereafter, in step 404c, an optimized multi-level classification model is obtained.

  The first and second adjustment functions and the first and second smoothing functions of the present invention will be described in detail below.

上記の式は、各ラベル付きデータサンプルの予測されるレベルラベル値とその対応するラベル付きレベルラベル値との間の距離の平方和を表わしている。
上記の式によれば、データサンプルが誤って分類されるレベルが真のレベルから外れるほど、損失がより大きくなる。
従って、誤った分類エラーが大局的なレベル値の距離によって最小化されれば、レベル関数は最適化されるだろう。
第１の調整の利点は、全てのレベル間の大局的な順序関係が量子化され、暗黙にかつ巨視的に最適化されるということである。
上述したように、平方和の他に、式（２）は各種の数学和（例えば、絶対値の和あるいは高累乗和（ｓｕｍ ｏｆ ｈｉｇｈｅｒ ｐｏｗｅｒ））を用いることが可能である。 First adjustment (classification model adjustment based on global level value distance)
The introduction of a level order relationship is an important feature that distinguishes multilevel classifiers from multiclass classifiers.
In the present invention, an assumption of Euclidian metric is introduced into the level space in order to represent the distance between levels according to the level value.
As a result, for example, the square loss can be employed to represent V _Level (f, Y _R ) as follows:

The above equation represents the sum of squares of the distance between the predicted level label value of each labeled data sample and its corresponding labeled level label value.
According to the above equation, the loss is greater as the level at which the data sample is misclassified deviates from the true level.
Thus, if the erroneous classification error is minimized by the global level value distance, the level function will be optimized.
The advantage of the first adjustment is that the global order relationship between all levels is quantized and optimized implicitly and macroscopically.
As described above, in addition to the sum of squares, the formula (2) can use various mathematical sums (for example, a sum of absolute values or a sum of high powers).

第２の調整において、ラベル付きデータサンプルの配列は、レベルラベルに従って再度並び替えられ、その後、その序列における隣接するデータサンプルの予測値間の順序エラーの数学和が、分類モデルを調整するために使用される。
それにより、レベル関数は、レベル順序関係によって制約されるデータ間の順序関係によって最適化される。
第２の調整の利点は、データサンプルの局所的な順序関係が明示的かつ微視的に最適化されるということである。
同様に、第２の調整における数学和は、式（３）で与えられる形式に限定されない。例えば、絶対値の和、平方和、高累乗和（ｓｕｍ ｏｆ ｈｉｇｈｅｒ ｐｏｗｅｒ）あるいはその他の形式を使用することが可能である。 Second adjustment (classification model adjustment based on the order relationship between data samples)
In order to further utilize the level order relationship, the present invention introduces a second component that maintains the local order of data samples according to their level labels.
First, l labeled data samples can be re-ordered according to their level labels.
To distinguish, different subscripts x _p are used to indicate the position in the new order.
As a result, for example, hinge loss can be employed to represent V _Order (f) as follows:

In the second adjustment, the array of labeled data samples is re-ordered according to the level label, after which the mathematical sum of the order errors between the predicted values of adjacent data samples in that order is used to adjust the classification model. used.
Thereby, the level function is optimized by the order relation between data constrained by the level order relation.
The advantage of the second adjustment is that the local ordering of the data samples is explicitly and microscopically optimized.
Similarly, the mathematical sum in the second adjustment is not limited to the form given by Equation (3). For example, absolute sums, sums of squares, sums of high powers, or other forms can be used.

次に、データサンプル類似度に基づいた平滑調整項｜｜ｆ｜｜^２ _Ｉは、以下のように定義することができる。

上記の式において、分類モデルは、データサンプル間の類似度に基づいて重み付けされた予測レベルラベル値の間の距離の平方和に従って平滑化される。
従って、式（５）の最小化は、データ幾何学的分布に従ってレベル関数を滑らかに変化させる。
第１の平滑化の利点は、分類モデルを学習するために大量のラベル無しデータサンプルを利用することができることである。
しかしながら、この方法は、また、データ分布がレベル分布と常に一致するとは限らないという問題点を有する。従って、本発明は、さらに分類モデルを最適化するために第２の平滑化を使用する。上述したように、平方和の他に、式（５）は各種の数学和（例えば、絶対値の和あるいは高累乗和（ｓｕｍ ｏｆ ｈｉｇｈｅｒ ｐｏｗｅｒ））を用いることが可能である。 First smoothing (classification model smoothing based on similarity between data samples)
The first smoothing is similar to their usual method in existing technology and utilizes unlabeled data samples to better match the classification model to the inherent geometry of the overall data distribution.
This method is based on the following assumptions.
Similar data samples are likely to be classified at the same level.
For example, the data similarity S ^I _{i, j} between all labeled and unlabeled data samples can be calculated by using the following Gaussian kernel based on their characteristics: .

Next, the smoothing adjustment term || f || ² _I based on the data sample similarity can be defined as follows.

In the above equation, the classification model is smoothed according to the sum of squares of the distances between the prediction level label values weighted based on the similarity between the data samples.
Therefore, the minimization of Equation (5) smoothly changes the level function according to the data geometric distribution.
The advantage of the first smoothing is that a large amount of unlabeled data samples can be used to learn the classification model.
However, this method also has a problem that the data distribution does not always coincide with the level distribution. Thus, the present invention uses a second smoothing to further optimize the classification model. As described above, in addition to the sum of squares, Equation (5) can use various mathematical sums (for example, sum of absolute values or sum of high power).

レベル類似度Ｓ^Ｉ _ｉ，ｊの定義に基づいて、レベル類似度に基づいた平滑調整項｜｜ｆ｜｜^２ _Ｌは以下のように定義することができる。

上記の等式において、分類モデルは、データサンプルのレベル間の類似度に基づいて重み付けされる予測レベルラベル値の間の距離の平方和に従って平滑化される。
従って、式（８）の最小化は、レベル幾何学的分布に従ってレベル関数を滑らかに変化させる。
第２の平滑化の利点は、大量のラベル無しデータサンプルがマルチレベル分類モデルの学習において役立つだけでなく、レベルの順序の関係を考慮できることである。
上述したように、平方和の他に、式（８）は各種の数学和（例えば、絶対値の和あるいは高累乗和（ｓｕｍ ｏｆ ｈｉｇｈｅｒ ｐｏｗｅｒ））を用いることが可能である。 Second smoothing (classification model smoothing based on similarity between levels)
In addition to the general sample similarity between data samples, the present invention further introduces level similarity as an important feature that can be effectively applied to semi-supervised scenarios by a multi-level classifier.
In the present invention, first, a level distance weight matrix B ^L _{k, k} is used based on an assumption of Euclidean metric to represent a distance between levels by a level value. _' Concept introduced,
For example, it can be defined as the following exponential function:
B ^L _{k, k ′} = e ^{− | rk−rk ′ |} (6)

Next, a data-level probability matrix P _{i, k} representing the probability of the data sample x _i belonging to the k level is calculated using an existing method.
Here, as this existing method, for example, Non-Patent Document 3 (M. Qian, F. Nie and C. Zhang “Probabilistic labeled semi-supervised smm. In Workshop on Optimized Basing M It is possible to use the method proposed in the paper entitled IEEE International Conference on Data Mining (ICDM), 2009)).
Finally, the concept of level similarity is introduced as follows for data samples.
For two data samples x _i and x _j , the level similarity S ^I _{i, j} between them is expressed as follows:

Based on the definition of the level similarity S ^I _{i, j} , the smoothing adjustment term || f || ² _L based on the level similarity can be defined as follows.

In the above equation, the classification model is smoothed according to the sum of squares of the distances between predicted level label values that are weighted based on the similarity between the levels of the data samples.
Therefore, the minimization of equation (8) smoothly changes the level function according to the level geometric distribution.
A second smoothing advantage is that a large number of unlabeled data samples are not only useful in learning a multi-level classification model, but can also take into account the ordering of levels.
As described above, in addition to the sum of squares, the formula (8) can use various mathematical sums (for example, sum of absolute values or sum of high power).

The classification model optimization method of the present invention has been described in detail above.
In particular, the present invention provides four classification model optimization functions: a classification model adjustment based on a global level value distance (first adjustment), and a classification model adjustment based on the order relationship between data samples (first adjustment). 2), classification model smoothing based on similarity between data samples (first smoothing), and classification model smoothing based on similarity between levels (second smoothing). These adjust and optimize the classification model based on the distance of global level values in the classification level, the order relationship between data samples, the similarity between data samples, and the similarity between levels, respectively.

  As described above, in the present invention, by using the level value and the similarity, the order relation between the levels is well-coupled in the classification model, and thereby the accuracy of multi-level classification is improved. Will improve.

Furthermore, since a unified classification function is constructed for all levels, each data sample is calculated with the same measure.
Therefore, the output of the classifier is not only a discrete level label, but also a continuous measure value that quantizes the level label, and as a result, the output of the multilevel classifier can be measured.

  In addition, with the introduction of two smoothing functions, the classification model can be applied to both previous supervised problems and semi-supervised problems that are more common in real-world information processing applications. Improves the practical use of level classification.

  While specific embodiments of the invention have been described with reference to the accompanying drawings, the invention is not limited to the specific configurations and processes shown in the drawings. In the above description, details of known methods and techniques are omitted for the sake of brevity. Also, in the above examples, some specific steps have been illustrated, but the method and process of the present invention are not limited to the specific steps used in the description and illustration, so those skilled in the art are familiar. For example, once the spirit of the present invention is understood, various modifications, changes and additions can be made, and the order of the steps can be changed.

  Each element of the invention may be implemented as hardware, software, firmware, or a combination thereof and utilized within the system, subsystem, component, or subcomponent. When implemented as software, each element of the present invention is a program or code section for performing necessary tasks. These programs or code sections can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried on a carrier wave. "Machine readable medium" includes any medium that can store or transmit information. Examples of the machine-readable medium include an electronic circuit, a semiconductor storage device, a ROM, a flash memory, an EROM, a floppy disk, a CD-ROM, an optical disk, a hard disk, an optical fiber medium, and an RF link. The code section can be downloaded via a computer network such as the Internet or an intranet.

  The present invention can be implemented in various other forms without departing from the spirit and essential characteristics thereof. For example, the algorithm described in the embodiments can be modified as long as the system architecture does not depart from the basic spirit of the present invention. Accordingly, the above embodiments are considered in all respects to be illustrative and not restrictive. Since the scope of the present invention is defined by the appended claims rather than the foregoing description, any variation or equivalent that falls within the scope of the claims is included in the scope of the invention.

  Further, a part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
Entering a labeled data sample;
Generating an initial multi-level classification model using leveled data samples;
Optimizing the first multi-level classification model generated,
A method for constructing a multi-level classification model, wherein the optimization step adjusts an initial multi-level classification model based on a global level value distance between all levels.

(Appendix 2)
Adjusting the initial multi-level classification model based on the global level value distance between all levels,
For all labeled data samples, including the step of adjusting the level classification function to minimize the sum of the following items: (1) the reciprocal of the distance between the classification hyperplanes corresponding to the level classification function (2) each A method for constructing a multi-level classification model according to claim 1, characterized in that the mathematical sum of the distance between the predicted level label value of the labeled data sample and its corresponding labeled level label value.

(Appendix 3)
The method of constructing a multi-level classification model according to claim 1, wherein the optimization step further comprises the step of adjusting the initial multi-level classification model based on an ordered relationship between labeled data samples.

(Appendix 4)
Adjusting the first multi-level classification model based on an ordered relationship between labeled data samples;
For all labeled data samples, including the step of adjusting the level classification function to minimize the sum of the following items: (1) the reciprocal of the distance between the classification hyperplanes corresponding to the level classification function (2) each Mathematical sum of the distance between the predicted level label value of the labeled data sample and its corresponding labeled level label value. (3) The predicted value of the adjacent data sample in the array of labeled data samples rearranged according to the level label. A method for constructing a multi-level classification model according to appendix 3, characterized in that:

(Appendix 5)
Applied to semi-supervised scenarios,
Entering unlabeled data samples;
The method of constructing a multi-level classification model according to claim 3, further comprising generating an initial multi-level classification model using the leveled data sample and the unlabeled data sample.

(Appendix 6)
The multilevel classification model according to appendix 5, wherein the optimization step includes the step of smoothing the initial multilevel classification model based on the similarity between data samples and the similarity between levels Method.

(Appendix 7)
The smoothing step comprises:
For all labeled data samples, including the step of adjusting the level classification function to minimize the sum of the following items: (1) the reciprocal of the distance between the classification hyperplanes corresponding to the level classification function (2) each Mathematical sum of the distance between the predicted level label value of the labeled data sample and its corresponding labeled level label value. (3) The predicted value of the adjacent data sample in the array of labeled data samples rearranged according to the level label. Mathematical sum of order errors between (4) Mathematical sum of distances between prediction level label values weighted based on similarity between data samples for all labeled and unlabeled data samples (5) Similarity between levels of data samples for labeled and unlabeled data samples How to build a multi-level classification model according to note 6, wherein the mathematical sum of the distances between the weighted predicted level label value based on.

(Appendix 8)
The method for constructing a multilevel classification model according to any one of appendix 2, appendix 4 and appendix 7, wherein the mathematical sum is a sum of absolute values, a sum of squares, or a high power sum.

(Appendix 9)
Assigning weight parameters for the mathematical sum of each item,
The method of constructing a multi-level classification model according to any one of appendix 2, appendix 4 and appendix 7, wherein the weight parameter is equal to or greater than zero.

Ｂ^Ｌ _ｋ，ｋ'＝ｅ^{−｜ｒｋ−ｒｋ'｜}
（Ｘ^Ｌ＝｛（ｘ_ｉ，ｙ_ｉ）｝， ｉ＝１，．．．，ｌは、ｌ個のラベル付きデータサンプルを示し、ｆは、レベル関数を示し、｜｜ｆ｜｜^２ _Ｋは、レベル関数ｆの基本的最適化項であり、Ｙ_Ｒ＝｛ｙ_ｉ｝， ｉ＝１，．．．，ｌは、ｌ個のラベル付きデータサンプルのクラスラベルベクトルを示す）
であることを特徴とする付記９に記載のマルチレベル分類モデルを構築する方法。 (Appendix 10)
The optimization step minimizes the following equation: f ^* = arg _f min || f || ² _K + r ₁ V _Level (f, Y _R ) + r ₂ V _Order (f) + r ₃ (1-α) | | F || ² _I + r ₃ α || f || ² _L
here,

B ^L _{k, k ′} = e ^{− | rk−rk ′ |}
(X ^L = {(x _i , y _i )}, i = 1,..., L indicates l labeled data samples, f indicates a level function, and || f || ² _K Is a basic optimization term of the level function f, and Y _R = {y _i }, i = 1,..., L indicates a class label vector of l labeled data samples)
A method for constructing a multi-level classification model according to appendix 9, characterized in that:

(Appendix 11)
A system for building a multi-level classification model,
First input means for inputting labeled data samples;
A classification model initialization means for generating an initial multi-level classification model using leveled data samples;
A classification model optimization means for optimizing the first generated multi-level classification model,
The system wherein the classification model optimization means is configured to adjust an initial multi-level classification model based on a global level value distance between all levels.

(Appendix 12)
The system of claim 11, wherein the classification model optimization means is further configured to adjust the first multi-level classification model based on an ordered relationship between labeled data samples.

(Appendix 13)
Applied to semi-supervised scenarios,
A second input means for inputting an unlabeled data sample;
13. The system according to appendix 12, wherein the classification model initialization means generates an initial multi-level classification model using leveled data samples and unlabeled data samples.

(Appendix 14)
The supplementary note 13 wherein the classification model optimization means is further configured to smooth an initial multi-level classification model based on the similarity between data samples and the similarity between levels. system.

101: Classification model initialization means 102: Classification model adjustment means based on local level adjacency 201: Classification model initialization means 202: Classification model smoothing means based on data sample similarity 300: Classification model optimization system 301: Classification model Initialization means

302: Classification model optimization means

Claims (7)

Applied to semi-supervised scenarios,
Entering a labeled data sample;
Entering unlabeled data samples;
Generating an initial multi-level classification model using labeled and unlabeled data samples ;
Optimizing the first multi-level classification model generated,
Wherein the optimization step,
A first adjustment step of adjusting an initial multi-level classification model based on a distance between a predicted level classification value and a classified level classification value for each labeled data sample ;
A second adjustment step of adjusting the first multi-level classification model based on an ordered relationship between labeled data samples;
Smoothing an initial multi-level classification model based on the similarity between data samples and the similarity between levels;
The similarity between the data samples indicates the degree of similarity between the labeled data sample and the unlabeled data sample,
A method for constructing a multi-level classification model, wherein the similarity between the levels indicates a level of similarity between data samples . The first adjustment step includes:
For all labeled data samples, adjusting the level classification function to minimize the mathematical sum of the distance between the predicted level classification value and the classified level classification value for each labeled data sample. The method of constructing a multi-level classification model according to claim 1, comprising: The second adjustment step includes:
Level classification for all labeled data samples so that the mathematical sum of the order errors between the predicted level classification values of adjacent data samples in the array of labeled data samples reordered according to the level classification is minimized Includes adjusting the function
The method of constructing a multi-level classification model according to claim 1 . The smoothing step comprises:
For all labeled data samples, including adjusting the level classification function to minimize the sum of the following items:
(1) For all labeled data samples and unlabeled data samples, mathematical sum of distances between predicted level classification values weighted based on the similarity between the data samples
(2) For all labeled and unlabeled data samples, the mathematical sum of the distances between predicted level classification values weighted based on the level similarity between the data samples
The method of constructing a multi-level classification model according to claim 1 . The method for constructing a multilevel classification model according to any one of claims 2 to 4, wherein the mathematical sum is a sum of absolute values, a sum of squares, or a high power sum . Assigning weight parameters for the mathematical sum of each item,
The method of constructing a multi-level classification model according to any one of claims 2 to 4, wherein the weight parameter is equal to or greater than zero . A system for building a multi-level classification model,
Applied to semi-supervised scenarios,
First input means for inputting labeled data samples;
A second input means for inputting unlabeled data samples;
A classification model initialization means for generating an initial multilevel classification model using labeled and unlabeled data samples;
A classification model optimization means for optimizing the first generated multi-level classification model,
The classification model optimizing means comprises:
First adjustment means for adjusting an initial multi-level classification model based on a distance between a predicted level classification value and a classified level classification value for each labeled data sample;
A second adjustment means for adjusting the first multi-level classification model based on an ordered relationship between labeled data samples;
Smoothing the initial multi-level classification model based on the similarity between data samples and the similarity between levels,
The similarity between the data samples indicates the degree of similarity between the labeled data sample and the unlabeled data sample,
The similarity between the levels indicates the level of similarity between data samples.
A system characterized by that .

Images