CN102945371A - Classifying method based on multi-label flexible support vector machine - Google Patents

Abstract

The embodiment of the invention discloses a classifying method based on a multi-label flexible support vector machine, which comprises the steps of first, defining a novel distance measuring method in a multi-label space to measure the distance from point to point in the multi-label space under a special classifying target; then, defining a neighborhood for each point in the multi-label space under a special classifying target, wherein the neighborhood of a certain point comprises several points which are closest to the central point in the novel distance measuring method; and finally, combining the neighborhood information in the multi-label space of each sample point and using the novel multi-label flexible support vector machine to carry out multi-label classifying training. The classifying method based on the multi-label flexible support vector machine is used to improve the classifying precision of an identifying classifier in multi-label classification by using information comprised in the multi-label space so as to reduce the influence of noise labels to classification.

Description

Sorting technique based on the flexible support vector machine of many labels

Technical field

The invention belongs to the label technique field, relate to especially a kind of sorting technique based on the flexible support vector machine of many labels.

Background technology

Along with the arrival of information age, multi-medium data has been realized volatile growth.Label as one of multimedia content-form, can help to solve a lot of important real world applications in data mining aspect, is particularly striding field of media, embodies very important effect.For example, utilize suitable label as the part of annotation of images, can develop powerful image labeling and image retrieval technologies; Utilize suitable label as the part of film comment, can develop effective film commending system; Utilize suitable label as the part of Web Page Tags, can develop more efficient search engine.

The kind of label is diversified, the volatile growth because data volume is maked rapid progress, and only relying on the data treatment people is unpractical to manual the tagging of all data.In this context, Social Label has just arisen at the historic moment.Social Label, claim again the cooperation label, Folksonomy is a kind ofly to allow the ordinary populace user label that online digital resource and oneself provide can be done related method, the tissue typing's system from bottom to top that is produced, Web content is organized and shared by the user.Here, general public can be by adding the label of oneself feeling suitable for own interested digital resource at thread environment in corresponding system.Just be based on this characteristics, the result of Social Label is inaccurate often, includes a lot of noises, because the subjectivity of oneself all can not be got rid of by the domestic consumer of each participation Social Label, carelessness, or even shortage patience removes to provide a perfect label.

Be further Data Management Analysis service in order better to utilize Social Label, the accuracy that must improve as much as possible labeling reduces noise to the impact of labeling.Simultaneously, because the kind of label is diversified, therefore, many labels anti-noise sorter arises at the historic moment, and boundless application prospect and very important practical value are arranged.When traditional identifying sorter is applied in many labelings problem, generally many labels problem is converted into the classification mode of one-to-many (One Vs All), namely many labelings problem is converted into a plurality of two classification problems.Traditional identifying sorter is not used the information that comprises in many Label space in this conversion process.And in fact, the label that data are marked with is more, and the information that comprises in the Label space is also just more, and these information can be utilized.When judging whether data point should put on certain label, existing other labels of this data point can play certain help to judgement.For example, the existing label of image that comprises animal when a width of cloth is sky, cloud, and the meadow, in the time of trees, the label that it more may be marked with is bird rather than fish; And the existing label of image that comprises animal when a width of cloth is water, pasture and water, and the sea, in the time of coral, the label that it more likely is marked with is fish rather than bird.The information that comprises in many Label space can help us better to classify to a certain extent, reduces noise to the impact of classification.

Summary of the invention

For addressing the above problem, the object of the present invention is to provide a kind of sorting technique based on the flexible support vector machine of many labels, the information that is used for utilizing many Label space to comprise improves the nicety of grading of identifying sorter in many labelings, reduces the noise label to the impact of classification.

For achieving the above object, technical scheme of the present invention is:

A kind of sorting technique based on the flexible support vector machine of many labels may further comprise the steps:

At first, a kind of novel distance metric method of definition in many Label space is used for weighing under specific class object distance between points in many Label space, and described novel distance metric method is: many labels training set is expressed as

Each point in the training set

All be marked with diversified label, the label dictionary of whole training set has formed many Label space of S dimension

Each point in many labels training set

At the visual angle

In proper vector be expressed as x _i, the label vector representation in label dictionary is d _i=(d _{I, 1}, d _{I, 2}..., d _{I, S}) ', be d wherein _{I, r}{ 0,1}, 1≤r≤S represent r label T in the dictionary to ∈ _rWhether at I _iY is used in middle appearance simultaneously _{I, r}Expression I _iTag along sort, y _{I, r}=2d _{I, r}-1.In the classification mode of many labels one-to-many (One Vs All), as a label T _rWhen being used as class object, remaining label will form the label characteristics space of a S-1 dimension in the label dictionary Use t _{I, r}Expression I _iIn the space

In proper vector, t _{I, r}=(d _{I, 1}..., d _{I, r-1}, d _{I, r+1}..., d _{I, S}) ',

Definition

As given d _{I, k}During=0or 1, d _{I, r}The conditional probability of=0or1 is as follows:

$P_{10}\stackrel{\mathrm{\Δ}}{=}P(d_{i,r}=1|d_{i,k}=0)=1-P_{00}$

$P_{01}\stackrel{\mathrm{\Δ}}{=}P(d_{i,r}=0|d_{i,k}=1)=1-P_{11}$

With each label T _rDegree of association vector be labeled as g _r,

g _r＝(g _r，1，...，g _r，r-1，g _r，r+1，...，g _r，S)′，

Each element representation label T of vector _rWith the degree of association of other labels,

Degree of association element g _{R, k}(k ∈ 1 ..., and r-1, r+1 ..., S}) be defined as follows shown in the formula: g _{R, k}=P ₀₀P ₁₁+ P ₁₀P ₀₁, with sample point in the space

In proper vector and each label T _rDegree of association vector combines, and obtains that a kind of novel distance metric method is defined as follows shown in the formula in many Label space: dis _r(I _i, I _j)=|| (t _{I, r}-t _{J, r}) ⊙ g _r|| _p, wherein ⊙ represents Hadamard (Hadamard) product between the vector;

Then, to the neighborhood of each some definition in many Label space, certain neighborhood of a point is included in the nearest several points of decentering point under the novel distance metric method under specific class object.The method for expressing of described neighborhood is: I _iIn the space The neighborhood of this new distance metric method definition of middle usefulness does not comprise I _iOneself is expressed as

I _iWith its field

The classification results similarity of middle data point is high, and the classification results similarity at non-Neighborhood Number strong point is low, neighborhood

Big or small u represent I _iIn the space

In the most contiguous neighbours number of ordering,

At last, in conjunction with the neighborhood information of each sample point in many Label space, a kind of new flexible support vector machine classifiers of many labels that utilization proposes carry out many labelings training, and the method for building up of the described flexible support vector machine classifiers of many labels newly is: with each some I _iTag along sort y _{I, r}Span from two points 1 ,+1} has expanded in the flexible scope [1 ,+1], and each the some I _iFlexible tag along sort be labeled as l _{I, r}, l _{I, r}Value not only depend on l _iTag along sort y _{I, r}, also depend on I _iIn the space

In the most contiguous neighbours tag along sort of ordering, l _{I, r}Shown in being defined as follows:

D is constant, and 0≤D＜1, and the optimized-type of the flexible support vector machine of many labels is as follows:

$\underset{w}{\min }\frac{1}{2}{\left|\right|w\left|\right|}^2+C\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\left|l_{i,r}\right|{\mathrm{\ξ}}_i$

$s.t.{\∀}_{i=1}^n:l_{i,r}(w^T{x}_i+\hat{b})\≥{\left|l_{i,r}\right|}^2-\left|l_{i,r}\right|{\mathrm{\ξ}}_i,{\mathrm{\ξ}}_i\≥0$

W wherein,

Be respectively coefficient and the biasing of the flexible support vector machine classifiers of many labels (SVM-MSM), C is constant, ξ _iIt is slack variable.

Compared with prior art, the present invention has following beneficial effect:

(1) proposition of novelty takes full advantage of the information in many Label space in the classification mode of many labels one-to-many (One Vs All), to reduce noise to the impact of classification based training process, improves the classify accuracy of many labelings identifying sorter.

(2) a kind of novel distance metric method of definition in many Label space, be used for weighing under specific class object distance between points in many Label space, this distance metric method has fully taken into account mutual relationship and the interdependent degree between label and the label.

(3) invented flexible support vector machine (the Support Vector Machine with Multi-label Soft Membership of many labels, SVM-MSM), can be applied to each neighborhood of a point information in many Label space in the classification based training process by flexible tag along sort.

Description of drawings

Fig. 1 is the process flow diagram based on the sorting technique of the flexible support vector machine of many labels of the embodiment of the invention.

Embodiment

In order to make purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, is not intended to limit the present invention.

On the contrary, the present invention contain any by claim definition in substituting of making of marrow of the present invention and scope, modification, equivalent method and scheme.Further, in order to make the public the present invention is had a better understanding, in hereinafter details of the present invention being described, detailed some specific detail sections of having described.There is not for a person skilled in the art the description of these detail sections can understand the present invention fully yet.

With reference to figure 1, be depicted as the process flow diagram based on the sorting technique of the flexible support vector machine of many labels of the embodiment of the invention, it may further comprise the steps:

S01, a kind of novel distance metric method of definition in many Label space is used for weighing under specific class object distance between points in many Label space;

S02, to the neighborhood of each some definition in many Label space, certain neighborhood of a point is included in the nearest several points of decentering point under the novel distance metric method under specific class object;

S03 in conjunction with the neighborhood information of each sample point in many Label space, utilizes a kind of new flexible support vector machine classifier of many labels that proposes to carry out many labeling training.

The embodiment of the invention proposes the flexible support vector machine (Support Vector Machine with Multi-label Soft Membership, SVM-MSM) of a kind of many labels.Many labels training set is expressed as

Each point in many labels training set

All be marked with diversified label, the label dictionary of whole many labels training set has formed many Label space of S dimension

As any one label T _r(1≤r≤S) is during as the target of two classification, and remaining label will form the label characteristics space of a S-1 dimension

Each point in many labels training set

At the visual angle

In proper vector be expressed as x _i, the label vector representation in label dictionary is d _i=(d _{I, 1}, d _{I, 2}..., d _{I, S}) ', be d wherein _{I, r}{ 0,1}, 1≤r≤S represent r label T in the dictionary to ∈ _rWhether at I _iMiddle appearance.For each label T _rWith each some I _i=(x _i, d _i), use y _{I, r}Expression I _iTag along sort, y _{I, r}=2d _{I, r}-1.

A kind of novel distance metric method that the embodiment of the invention defines in many Label space is used for weighing under specific class object distance between points in many Label space.In the classification mode of many labels one-to-many (One Vs All), as a label T _rWhen being used as class object, remaining label will form the label characteristics space of a S-1 dimension in the label dictionary

In the space

The classification similarity of the point that middle distance is nearer is also higher.Use t _{I, r}Expression I _iIn the space

In proper vector, t _{I, r}=(d _{I, 1}..., d _{I, r-1}, d _{I, r+1}..., d _{I, S}) '.Yet, use formula || t _{I, r}-t _{J, r}|| _pDirectly measure I _iAnd I _j

In distance in most of the cases be irrational because be separate between this method hypothesis label, and ignored the mutual relationship that may exist between the label.In reality, exist various relations between the label, some label often occurs together, and some label but occurs never simultaneously.

By estimating For I _iAnd I _j

In the impact of distance label T is discussed _rAnd T _k(k ∈ 1 ..., and r-1, r+1 ..., the relation between S}).When | d _{I, k}-d _{J, k}|=0 o'clock, | d _{I, k}-d _{J, k}| for I _iAnd I _j

In the impact of distance also be 0; When | d _{I, k}-d _{J, k}|=1 o'clock, | d _{I, k}-d _{J, k}| for I _iAnd I _j

In the impact of distance depend on label T _rAnd T _kBetween the degree of association.| d _{I, k}-d _{J, k}|=1 He | d _{I, r}-d _{J, r}| value between relation as shown in the formula described:

When

And $d_{i,k}=1\⇒d_{i,r}=1$

Or

And $d_{i,k}=1\⇒d_{i,r}=0$ The time

(1)

When

And $d_{i,k}=1\⇒d_{i,r}=0$

Or

And $d_{i,k}=1\⇒d_{i,r}=1$ The time

Definition

Formula (1) has been described label T _rAnd T _kBetween four kinds of special relationships.In practice, as label T _r

With

In when being evenly distributed, T _kFor T _rNot one and have distinctive label; As label T _r

With

In the time pockety, T _kFor T _rBe one and have distinctive label.As given d _{I, k}During=0or1, d _{I, r}The conditional probability of=0or1 is as follows:

$P_{10}\stackrel{\mathrm{\Δ}}{=}P(d_{i,r}=1|d_{i,k}=0)=1-P_{00}$

$P_{01}\stackrel{\mathrm{\Δ}}{=}P(d_{i,r}=0|d_{i,k}=1)=1-P_{11}$

Can find out from formula (1) (2), work as P ₀₀P ₁₁Perhaps P ₁₀P ₀₁Value when larger, | d _{I, k}-d _{J, k}|=1 releases | d _{I, r}-d _{J, r}|=1 probability is also larger, | d _{I, k}-d _{J, k}| for I _iAnd I _j In the impact of distance also larger.Work as P ₀₀P ₀₁Perhaps P ₁₀P ₁₁Value when larger, | d _{I, k}-d _{J, k}|=1 releases | d _{I, r}-d _{J, r}|=0 probability is also larger, | d _{I, k}-d _{J, k}| for I _iAnd I _j

In the impact of distance also less.

P ₀₀·P ₁₁+P ₁₀·P ₀₁+P ₀₀·P ₀₁+P ₁₀·P ₁₁＝1。With each label T _rDegree of association vector be labeled as g _r, g _r=(g _{R, 1}..., g _{R, r-1}, g _{R, r+1}..., g _{R, S}) '.Each element representation label T of vector _rThe degree of association with other labels.Define degree of association element g with following formula _{R, k}(k ∈ 1 ..., and r-1, r+1 ..., S}): g _{R, k}=P ₀₀P ₁₁+ P ₁₀P ₀₁

With sample point in the space

In proper vector and each label T _rDegree of association vector combines, and has defined a kind of novel distance metric method in many Label space, is shown below: dis _r(I _i, I _j)=|| (t _{I, r}-t _{J, r}) ⊙ g _r|| _p⊙ represents Hadamard (Hadamard) product between the vector, I _iIn the space

The neighborhood of this new distance metric method definition of middle usefulness does not comprise I _iOneself is expressed as

I _iWith its neighborhood

The classification results similarity of middle data point is high, and the classification results similarity at non-Neighborhood Number strong point is low.Neighborhood

Big or small u represent I _iIn the space

In the most contiguous neighbours number of ordering,

In order to utilize the information that comprises in many Label space, a kind of new flexible support vector machine (SVM-MSM) of many labels of the embodiment of the invention, this vector machine is put I with each _iTag along sort y _{I, r}Span { 1 ,+1} has expanded in the flexible scope [1 ,+1] from two points.The embodiment of the invention is put I with each _iFlexible tag along sort be labeled as l _{I, r}, l _{I, r}Value not only depend on I _iTag along sort y _{I, r}, also depend on I _iIn the space

In the most contiguous neighbours tag along sort of ordering.l _{I, r}Shown in being defined as follows:

D is constant, and 0≤D＜1.The optimized-type of the flexible support vector machine of many labels is as follows:

$\underset{w}{\min }\frac{1}{2}{\left|\right|w\left|\right|}^2+C\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\left|l_{i,r}\right|{\mathrm{\ξ}}_i$

$s.t.{\∀}_{i=1}^n:l_{i,r}(w^T{x}_i+\hat{b})\≥{\left|l_{i,r}\right|}^2-\left|l_{i,r}\right|{\mathrm{\ξ}}_i,{\mathrm{\ξ}}_i\≥0$

W wherein,

Be respectively coefficient and the biasing of the flexible support vector machine classifiers of many labels (SVM-MSM), C is constant, ξ _iIt is slack variable.By lagrange's method of multipliers, we can obtain the dual problem of this problem:

$\underset{\mathrm{\λ}}{\max }-\frac{1}{2}\underset{i,j=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\λ}}_i{\mathrm{\λ}}_j{l}_{i,r}{l}_{j,r}{x}_i^T{x}_j+\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\λ}}_i{\left|l_{i,r}\right|}^2$

$s.t.\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\λ}}_i{l}_{i,r}=0;$ ${\∀}_{i=1}^n:0\≤{\mathrm{\λ}}_i\≤C$

λ wherein _iIt is Lagrange multiplier.

The above only is preferred embodiment of the present invention, not in order to limiting the present invention, all any modifications of doing within the spirit and principles in the present invention, is equal to and replaces and improvement etc., all should be included within protection scope of the present invention.

Claims (1)

1. the sorting technique based on the flexible support vector machine of many labels is characterized in that, may further comprise the steps:

At first, a kind of novel distance metric method of definition in many Label space is used for weighing under specific class object distance between points in many Label space, and described novel distance metric method is: many labels training set is expressed as Each point in the training set

All be marked with diversified label, the label dictionary of whole training set has formed many Label space of S dimension

Each point in many labels training set At the visual angle In proper vector be expressed as x _i, the label vector representation in label dictionary is d _i=(d _{I, 1}, d _{I, 2}..., d _{I, S}) ', be d wherein _{I, r}{ 0,1}, 1≤r≤S represent r label T in the dictionary to ∈ _rWhether at I _iY is used in middle appearance simultaneously _{I, r}Expression I _iTag along sort, y _{I, r}=2d _{I, r}-1, in the classification mode of many labels one-to-many OneVs All, as a label T _rWhen being used as class object, remaining label will form the label characteristics space of a S-1 dimension in the label dictionary

Use t _{I, r}Expression I _iIn the space

In proper vector, t _{I, r}=(d _{I, 1}..., d _{I, r-1}, d _{I, r+1}..., d _{I, S}) ',

Definition

As given d _{I, k}During=0or 1, d _{I, r}The conditional probability of=0or1 is as follows:

$P_{10}\stackrel{\mathrm{\Δ}}{=}P(d_{i,r}=1|d_{i,k}=0)=1-P_{00}$

$P_{01}\stackrel{\mathrm{\Δ}}{=}P(d_{i,r}=0|d_{i,k}=1)=1-P_{11}$

With each label T _rDegree of association vector be labeled as g _r,

g _r＝(g _r，1，...，g _r，r-1，g _r，r+1，...，g _r，S)′，

Each element representation label T of vector _rWith the degree of association of other labels,

Degree of association element g _{R, k}(k ∈ 1 ..., and r-1, r+1 ..., S}) be defined as follows shown in the formula: g _{R, k}=P ₀₀P ₁₁+ P ₁₀P ₀₁, with sample point in the space

In proper vector and each label T _rDegree of association vector combines, and obtains that a kind of novel distance metric method is defined as follows shown in the formula in many Label space: dis _r(I _i, I _j)=|| (t _{I, r}-t _{J, r}) ⊙ g _r|| _p, wherein ⊙ represents the Hadamard Hadamard product between the vector;

Then, to the neighborhood of each some definition in many Label space, certain neighborhood of a point is included in the nearest several points of decentering point under the novel distance metric method under specific class object, and the method for expressing of described neighborhood is: I _iIn the space

The neighborhood of this new distance metric method definition of middle usefulness does not comprise I _iOneself is expressed as

I _iWith its field

The classification results similarity of middle data point is high, and the classification results similarity at non-Neighborhood Number strong point is low, neighborhood

Big or small u represent I _iIn the space

In the most contiguous neighbours number of ordering,

At last, in conjunction with the neighborhood information of each sample point in many Label space, a kind of new flexible support vector machine classifiers of many labels that utilization proposes carry out many labelings training, and the method for building up of the described flexible support vector machine classifiers of many labels newly is: with each some I _iTag along sort y _{I, r}Span from two points 1 ,+1} has expanded in the flexible scope [1 ,+1], and each the some I _iFlexible tag along sort be labeled as l _{I, r}, l _{I, r}Value not only depend on I _iTag along sort y _{I, r}, also depend on I _iIn the space

In the most contiguous neighbours tag along sort of ordering, l _{I, r}Shown in being defined as follows:

D is constant, and 0≤D＜1, and the optimized-type of the flexible support vector machine of many labels is as follows:

$\underset{w}{\min }\frac{1}{2}{\left|\right|w\left|\right|}^2+C\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\left|l_{i,r}\right|{\mathrm{\ξ}}_i$

$s.t.{\∀}_{i=1}^n:l_{i,r}(w^T{x}_i+\hat{b})\≥{\left|l_{i,r}\right|}^2-\left|l_{i,r}\right|{\mathrm{\ξ}}_i,{\mathrm{\ξ}}_i\≥0$

W wherein,

Be respectively coefficient and the biasing of the flexible support vector machine classifier SVM-MSM of many labels, C is constant, ξ _iIt is slack variable.

Images