CN109947938A - Multi-label classification method, system, readable storage medium and computer device - Google Patents

Abstract

The present invention provides a multi-label classification method, system, readable storage medium and computer equipment. The method includes: converting a multi-label classification problem into two types of classification problems, each of which is corresponding to a label in the multi-label data set. ;Construct a semi-supervised multi-label learning model according to the preset feature selection and the correlation between the two types of classifiers; use a preset algorithm to solve the semi-supervised multi-label learning model to obtain the model parameters of the semi-supervised multi-label classification and marker correlation matrix; according to the model parameters and marker correlation matrix, predict the marker set to which the unknown marker belongs. The multi-label classification method in the present invention adopts the semi-supervised multi-label learning method of joint label correlation and feature selection, which not only effectively utilizes a large number of unlabeled multi-label samples, but also automatically obtains the correlation between labels through learning. , and dimensionality reduction for high-dimensional data is beneficial to obtain a multi-label classifier with better generalization performance.

Description

Multi-label classification method, system, readable storage medium and computer device

technical field

The present invention relates to the technical field of multi-label learning, and in particular, to a multi-label classification method, system, readable storage medium and computer device.

Background technique

Machine learning specializes in how computers simulate or realize human learning behavior, and is an important research field in computer science. Supervised learning is the most studied and widely used learning framework in machine learning, which assumes that each learning object belongs to only one label.

But in the real world, a token is often not enough to accurately describe some complex semantic objects. It can be seen that complex learning objects with multiple labels at the same time are ubiquitous, and traditional supervised learning methods are difficult to deal with these complex learning objects well.

For this reason, multi-label learning comes into being, and its task is to learn the known multi-label data set to predict the label set to which the unknown sample belongs. Among them, semi-supervised multi-label learning is a typical type of multi-label learning, which uses a small number of labeled samples and a large number of unlabeled samples to train to obtain a multi-label classifier.

However, in the prior art, although the current semi-supervised multi-label learning method considers the correlation between labels, it cannot estimate the label correlation and does not consider multi-label feature selection, and some semi-supervised multi-label learning methods consider multi-label feature selection. , but cannot automatically mine and exploit label correlation through learning, resulting in poor generalization performance of the trained multi-label classifier.

SUMMARY OF THE INVENTION

Based on this, the purpose of the present invention is to provide a multi-label classification method, system, readable storage medium and computer equipment to solve the poor generalization performance of the multi-label classifier trained by the semi-supervised multi-label learning method in the prior art technical issues.

A multi-label classification method according to an embodiment of the present invention includes:

Convert the multi-label classification problem into a two-class classification problem, each of which corresponds to a label in the multi-label data set;

According to the preset feature selection and the correlation between the two types of classifiers, construct a semi-supervised multi-label learning model;

Using a preset algorithm to solve the semi-supervised multi-label learning model, to obtain the model parameters and label correlation matrix of the semi-supervised multi-label classification;

According to the model parameters of the semi-supervised multi-label classification and the label correlation matrix, the label set to which the unknown label belongs is predicted.

In addition, according to a kind of multi-label classification method of the above-mentioned embodiment of the present invention, can also have the following additional technical features:

Further, the step of converting the multi-label classification problem into a two-class classification problem includes:

The multi-label classification problem is transformed into Q two-class classifiers, and the discriminant function of the q-th two-class classifier is defined as

where Q represents the number of markers in the multi-label dataset, and respectively represent the weight and bias corresponding to the discriminant function of the qth classifier of the two types of classifiers.

Further, the correlation between the two types of classifiers is that the learning of each of the two types of classifiers depends on the original feature vector and other label variables at the same time.

Further, the discriminant functions of the two types of classifiers can be transformed into:

Among them, the weight matrix Deviation matrix marker correlation vector

Further, the feature selection is an item that constrains the weight matrix W w ⁱ is the ith row of W.

Further, the objective function expression of the semi-supervised multi-label learning model is:

Among them, n and Q represent the number of samples and the number of markers, respectively, Represents the label matrix corresponding to the labeled sample, the label matrix Label matrix corresponding to n _l labeled multilabel samples It is composed of the label matrix Fu corresponding to the unlabeled multi-label sample. The initial state of _Fu is an all-zero matrix, F _i represents the label vector corresponding to the _ith sample, and p _i defines the importance of the ith training sample. is the marker correlation matrix, is the regularization term to avoid overfitting, w ⁱ is the ith row of W, λ and η are balance parameters, is the multi-label data matrix, tr( ) represents the trace of the matrix, Represents a vector whose elements are all ones.

Further, the preset algorithm is an alternate iterative solution algorithm.

A multi-label classification system according to an embodiment of the present invention includes:

The problem conversion module is used to convert the multi-label classification problem into two-class classification problems, and each two-class classifier corresponds to a label in the multi-label data set;

a model building module for building a semi-supervised multi-label learning model according to preset feature selection and the correlation between the two types of classifiers;

a model solving module for solving the semi-supervised multi-label learning model by using a preset algorithm to obtain model parameters and a label correlation matrix of the semi-supervised multi-label classification;

The marker prediction module predicts the marker set to which the unknown marker belongs according to the model parameters of the semi-supervised multi-marker classification and the marker correlation matrix.

Another method of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned method is implemented

Another method of the present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the above-mentioned method when the processor executes the program.

The multi-label classification method in the present invention adopts the semi-supervised multi-label learning method of joint label correlation and feature selection. When designing the multi-label classifier, not only the use of a large number of unlabeled data sets, but also the label correlation is considered. and multi-label feature selection, which not only effectively utilizes a large number of unlabeled multi-label samples, but also automatically obtains the correlation between labels through learning. In addition, dimensionality reduction for high-dimensional data is beneficial to obtain better generalization performance multi-label classifier.

Description of drawings

1 is a flowchart of a multi-label classification method in a first embodiment of the present invention;

Fig. 2 is the flow chart of the multi-label classification system in the second embodiment of the present invention;

FIG. 3 is a structural diagram of a computer device in a third embodiment of the present invention.

Description of main component symbols:

problem conversion module 11 Model building blocks 12 Model Solving Module 13 Tag prediction module 14 memory 10 processor 20 Computer program 30

The following specific embodiments will further illustrate the present invention in conjunction with the above drawings.

Detailed ways

In order to facilitate understanding of the present invention, the present invention will be described more fully hereinafter with reference to the related drawings. Several embodiments of the invention are presented in the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical," "horizontal," "left," "right," and similar expressions are used herein for illustrative purposes only.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

Please refer to FIG. 1, which shows the multi-label classification method in the first embodiment of the present invention, including steps S01 to S04.

In step S01, the multi-label classification problem is converted into a two-class classification problem, and each two-class classifier corresponds to a label in the multi-label data set.

Specifically, the steps of transforming a multi-label classification problem into a two-class classification problem include:

The multi-label classification problem is transformed into Q two-class classifiers, and the discriminant function of the q-th two-class classifier is defined as where Q represents the number of markers in the multi-label dataset, and respectively represent the weight and bias corresponding to the discriminant function of the qth classifier of the two types of classifiers.

Step S02, constructing a semi-supervised multi-label learning model according to the preset feature selection and the correlation between the two types of classifiers.

The correlation between the two types of classifiers is: the learning of each of the two types of classifiers depends on the original feature vector and other label variables at the same time.

Understandably, since the learning of the classifier corresponding to each tag depends on the original feature vector and other tag variables at the same time, two types of classifiers that are independent of each other can be learned at the same time, and multi-tag classification and tag correlation can also be simultaneously learned. study. Based on this, the corresponding discriminant function of the qth two-class classifier can be transformed into:

Among them, the weight matrix Deviation matrix marker correlation vector

In addition, in order to reduce the dimension of the feature space of high-dimensional data, the feature selection is: the term that constrains the weight matrix W w ⁱ is the ith row of W.

Specifically, the purpose of this step S02 is to unify the semi-supervised multi-label classification model parameters, label correlation and semi-supervised multi-label feature selection into the same model framework for joint learning to construct a semi-supervised multi-label learning model.

Wherein, the objective function of the semi-supervised multi-label learning model is expressed as:

Among them, n and Q represent the number of samples and the number of markers, respectively, Represents the label matrix corresponding to the labeled sample, the label matrix Label matrix corresponding to n _l labeled multilabel samples It is composed of the label matrix Fu corresponding to the unlabeled multi-label sample. The initial state of _Fu is an all-zero matrix, F _i represents the label vector corresponding to the _ith sample, and p _i defines the importance of the ith training sample. is the marker correlation matrix, is the regularization term to avoid overfitting, w ⁱ is the ith row of W, and λ and η are the balance parameters.

Based on this, the model can be further expressed as:

in, is the multi-label data matrix, tr( ) represents the trace of the matrix, Represents a vector whose elements are all ones.

Step S03, using a preset algorithm to solve the semi-supervised multi-label learning model, to obtain the model parameters and label correlation matrix of the semi-supervised multi-label classification.

Wherein, the preset algorithm is an alternate iterative solution algorithm.

The specific alternate iterative solution algorithm includes steps (1) to (3):

Step (1): Fix C and F, find W and b

When C and F are given, W and b can be obtained by optimally solving the following objective function.

Since ||W|| ₂ , 1 is not differentiable, this objective function is a non-smooth convex problem. The present invention transforms it into the following equivalent constrained smooth convex optimization problem, and uses the Nesterov-based accelerated projected gradient method to solve it.

where Θ={θ∈E ^(d+1)×Q |θ=(W ^T , b) ^T , ||W|| _2,1 ≤r, W∈R ^d×Q , b∈R ^Q } is Convex set, r is l ₂ , 1 the radius of the sphere.

make represents the search point of the t-th iteration, where α _t is an adjustment parameter.

for definition

in, Represents the function g( ) with respect to θ at the point upper derivative, is strongly convex with respect to θ. Therefore, the approximate solution of Θ _t+1 in the t+1th iteration can be calculated according to the following equation.

Therefore, w _t+1 can be calculated as follows:

Among them, Ω={W∈R ^d×Q |||W|| _2,1 ≤r] is a closed convex set, and π _Ω (·) is the Euclidean projection onto the convex set Ω.

b _t+1 can be calculated as follows:

which is

Step (2): Fix W, b and F, and find C

When W, b and F are given, C can be obtained by optimally solving the following objective function.

Each component in C can be obtained by optimizing the following objective function:

Step (3): Fix W, b and C, and find F

After optimizing W, b and C, it is possible to calculate Therefore, in each iterative solution process, the label value corresponding to the unlabeled sample in the label matrix F can be adjusted according to the formula defined below.

Step S04, according to the model parameters of the semi-supervised multi-label classification and the label correlation matrix, predict the label set to which the unknown label belongs.

For example, given an unknown sample x _u , its discriminant function value on the qth marker is: The predicted sign vector is: h(x _u )=sign(f ₁ (x _u ), . . . , f _Q (x _u )), where sign(·) is a sign function.

In summary, the multi-label classification method in the above-mentioned embodiments of the present invention adopts the semi-supervised multi-label learning method of joint label correlation and feature selection, and not only considers the use of a large number of unlabeled data sets while designing the multi-label classifier, Label correlation and multi-label feature selection are also considered, which not only effectively utilizes a large number of unlabeled multi-label samples, but also automatically obtains the correlation between labels through learning. In addition, dimensionality reduction for high-dimensional data is beneficial to Obtain a multi-label classifier with better generalization performance.

Another aspect of the present invention also provides a multi-label classification system, please refer to FIG. 2, which shows the multi-label classification system in the third embodiment of the present invention, including:

The problem conversion module 11 is used to convert the multi-label classification problem into two-class classification problems, and each two-class classifier corresponds to a label in the multi-label data set;

Model building module 12, for constructing a semi-supervised multi-label learning model according to preset feature selection and the correlation between the two types of classifiers;

Model solving module 13, for adopting preset algorithm to solve described semi-supervised multi-label learning model, to obtain the model parameters and label correlation matrix of described semi-supervised multi-label classification;

The marker prediction module 14 predicts the marker set to which the unknown marker belongs according to the model parameters of the semi-supervised multi-marker classification and the marker correlation matrix.

Further, the problem conversion module 11 can also be used to convert the multi-label classification problem into Q two-class classifiers, and the discriminant function of the q-th two-class classifier is defined as:

where Q represents the number of markers in the multi-label dataset, and respectively represent the weight and bias corresponding to the discriminant function of the qth classifier of the two types of classifiers.

Further, the correlation between the two types of classifiers is that the learning of each of the two types of classifiers depends on the original feature vector and other label variables at the same time.

Further, the discriminant functions of the two types of classifiers can be transformed into:

Among them, the weight matrix Deviation matrix marker correlation vector

Further, the feature selection is: an item that constrains the weight matrix W w ⁱ is the ith row of W.

Further, the objective function expression of the semi-supervised multi-label learning model is:

Among them, n and Q represent the number of samples and the number of markers, respectively, Represents the label matrix corresponding to the labeled sample, the label matrix Label matrix corresponding to n _l labeled multilabel samples It is composed of the label matrix Fu corresponding to the unlabeled multi-label sample. The initial state of _Fu is an all-zero matrix, F _i represents the label vector corresponding to the _ith sample, and p _i defines the importance of the ith training sample. is the marker correlation matrix, is the regularization term to avoid overfitting, w ⁱ is the ith row of W, λ and η are balance parameters, is the multi-label data matrix, tr( ) represents the trace of the matrix, Represents a vector whose elements are all ones.

Further, the preset algorithm is an alternate iterative solution algorithm.

To sum up, the multi-label classification system in the above-mentioned embodiments of the present invention adopts the semi-supervised multi-label learning method of joint label correlation and feature selection, and not only considers the use of a large number of unlabeled data sets while designing the multi-label classifier, Label correlation and multi-label feature selection are also considered, which not only effectively utilizes a large number of unlabeled multi-label samples, but also automatically obtains the correlation between labels through learning. In addition, dimensionality reduction for high-dimensional data is beneficial to Obtain a multi-label classifier with better generalization performance.

Another method of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements the above-mentioned multi-label classification method

Another method of the present invention further proposes a computer device, please refer to FIG. 3 , which shows the computer device in the third embodiment of the present invention, including a memory 10 , a processor 20 , and a computer device stored in the memory 20 and available in the processor 10 . A computer program 30 running on the processor 20 implements the multi-label classification method described above when the processor 20 executes the program 30.

Those skilled in the art will appreciate that logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing logical functions, may be embodied in in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or Used in conjunction with these instruction execution systems, apparatus or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in conjunction with an instruction execution system, apparatus, or apparatus.

More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as it may be, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

The above-mentioned embodiments only represent several embodiments of the present invention, and the descriptions thereof are specific and detailed, but should not be construed as limiting the scope of the patent of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present invention, some modifications and improvements can be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be governed by the appended claims.

Claims (10)

1. A multi-label classification method, comprising:

converting the multi-label classification problem into two classification problems, wherein each two classifier corresponds to one label in the multi-label data set;

constructing a semi-supervised multi-label learning model according to the preset feature selection and the correlation between the two types of classifiers;

solving the semi-supervised multi-label learning model by adopting a preset algorithm to obtain model parameters and a label correlation matrix of the semi-supervised multi-label classification;

and predicting a label set to which unknown labels belong according to the model parameters and the label correlation matrix of the semi-supervised multi-label classification.

2. The multi-label classification method according to claim 1, wherein said step of converting the multi-label classification problem into a two-class classification problem comprises:

converting the multi-label classification problem into Q two types of classifiers, wherein the discriminant function of the Q two types of classifiers is defined as

Wherein Q represents the number of markers in the multi-marker dataset,andand respectively representing the weight and the deviation corresponding to the discriminant functions of the qth classifiers and the two classifiers.

3. The multi-label classification method according to claim 2, characterized in that the correlation between the two classes of classifiers depends on the original feature vectors and other label variables simultaneously for the learning of each of the two classes of classifiers.

4. The multi-label classification method according to claim 3, characterized in that the discriminant functions of the two classes of classifiers are convertible into:

wherein the weight matrixDeviation ofMatrix arrayMark up correlation vector

5. The multi-label classification method according to claim 4, characterized in that the features are selected as terms constraining a weight matrix WwⁱRow i of W.

6. The multi-label classification method according to claim 1, characterized in that the objective function expression of the semi-supervised multi-label learning model is:

s.t.c_qq＝1，q＝1，2，...，Q

wherein n and Q respectively represent the number of samples and the number of marks,a marking matrix representing the correspondence of marked samples, the marking matrixFrom n to_lMarking matrix corresponding to marked multi-marked samplesMark matrix corresponding to unmarked multi-marked sampleComposition of，The initial state is an all-zero matrix, F_i·Representing the marker vector, p, corresponding to the ith sample_iThe importance of the ith training sample is defined,in order to mark the correlation matrix, the correlation matrix is marked,in order to avoid over-fitting the regularization term,wⁱrow i of W, λ and η are balance parameters,in order to be a multi-label data matrix,tr (-) denotes the trace of the matrix,representing a vector with elements all 1.

7. The multi-label classification method according to claim 1, characterized in that the preset algorithm is an alternating iterative solution algorithm.

8. A multi-label classification system, comprising:

the problem conversion module is used for converting the multi-label classification problem into two types of classification problems, and each two types of classifiers corresponds to one label in the multi-label data set;

the model building module is used for building a semi-supervised multi-label learning model according to the preset feature selection and the correlation between the two types of classifiers;

the model solving module is used for solving the semi-supervised multi-label learning model by adopting a preset algorithm so as to obtain model parameters and a label correlation matrix of the semi-supervised multi-label classification;

and the mark prediction module predicts a mark set to which unknown marks belong according to the model parameters of the semi-supervised multi-mark classification and the mark correlation matrix.

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.

10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1-7 when executing the program.