CN104102705A - Digital media object classification method based on large margin distributed learning - Google Patents

Abstract

The invention discloses a digital media object classification method based on large-interval distribution learning. In order to overcome the noise problem of digital media object category labels, the classification problem of digital media objects is finally formalized into a Convex quadratic optimization problem, and according to whether to use nonlinear kernel function and the characteristics of the training digital media object library itself, two optimization algorithms based on dual coordinate descent and average stochastic gradient descent are given. The situation is optional. If the user chooses the nonlinear kernel function, DCD is selected as the optimization algorithm during training; if the user chooses the linear kernel function, and there are many samples or features in the training digital media object library, ASGD is selected as the optimization algorithm during training, otherwise it is still Choose DCD as the optimization algorithm.

Description

A Digital Media Object Classification Method Based on Large Margin Distribution Learning

technical field

The invention relates to a digital media object classification method, in particular to a digital media object classification method based on large interval distribution learning.

Background technique

The current human society has entered the stage of digitalization in an all-round way. Currently, images, texts, videos, audios and other media used to disseminate information are recorded and processed in the form of binary codes. These encoded images, texts, videos, audios Collectively referred to as Digital Media Objects. Digital media objects have been widely used in all walks of life, such as remote sensing measurement and control, Internet sites, digital TV, telephone communications, etc. These industries accumulate a large amount of data every day, so as the amount of data continues to expand, how to effectively organize and manage digital media objects becomes more and more important, and the core issue is the classification of digital media objects. Scientific classification can not only facilitate the storage of these digital media objects; in subsequent services such as digital media retrieval, it can also provide faster and better retrieval results. In the classification task of digital media objects, each digital media object will have a corresponding category label. These category labels are usually manually marked by humans, so some noise will inevitably be introduced. Traditional classification methods based on large intervals, such as support vector machines (hereinafter abbreviated as SVM), are sensitive to noise because they only consider the interval of a single sample, and are not suitable for directly classifying digital media objects. Based on this finding, the present invention proposes a digital media object classification method based on large-interval distribution learning, which avoids sensitivity to noise by utilizing the information of the entire interval distribution instead of the interval of a single sample The problem of digital media object classification is solved.

Contents of the invention

Purpose of the invention: Considering that the category marks of digital media objects usually contain a lot of noise, the present invention proposes a noise-insensitive digital media object classification method based on the idea of large interval distribution learning. By making full use of the information of the entire interval distribution, the method maximizes the interval mean and minimizes the interval variance, avoids sensitivity to noise, and solves the problem of digital media object classification well.

Technical solution: a digital media object classification method based on large-interval distribution learning. First, the user prepares a digital media object library, in which each digital media object has a category mark, which is the training data. Next, convert the training digital media object into a feature representation, specifically, input the training digital media object into a feature extraction algorithm to obtain a feature vector of the digital media object. There are many methods for feature extraction of digital media objects. One method can be used to correspond to a feature. For example, for an image, its brightness can be used as a feature of the object, and contrast can be used as another feature. Record the total number of features as d, then each digital media object corresponds to a vector in the d-dimensional Euclidean space. Then, all the feature vectors corresponding to the training digital media objects and their category marks are input into the training algorithm of the classification model, and the classification model can be obtained after the training. In the prediction stage, the user inputs the digital media object to be predicted into the classification model, and the classification model can output its predicted category label. When training the classification model, in order to overcome the noise problem of the digital media object category label, the present invention is based on the idea of large interval distribution learning, and proposes a digital media object classification method LDM that is not sensitive to noise, by maximizing the interval mean while minimizing interval variance, and finally formalize the classification problem of digital media objects into a convex quadratic optimization problem, and according to whether to use the nonlinear kernel function and the characteristics of the training digital media object library itself (such as the number of samples, feature sparsity, etc.), give Two optimization algorithms based on dual coordinate descent (abbreviated as DCD below) and based on average stochastic gradient descent (abbreviated as ASGD below) have been proposed, and users can choose according to the actual situation. If the user chooses the nonlinear kernel function, DCD is selected as the optimization algorithm during training; if the user chooses the linear kernel function, and there are many samples or features in the training digital media object library, ASGD is selected as the optimization algorithm during training, otherwise it is still Choose DCD as the optimization algorithm.

Beneficial effects: Compared with the prior art, the present invention makes full use of the interval distribution information of the training digital media object library, and by maximizing the interval mean value while minimizing the interval variance, it overcomes the noise problem of the category label in the digital media object classification problem, and at the same time It also maintains the original advantages of SVM, and finally achieved a good classification effect.

Description of drawings

Fig. 1 is a principle flow chart of the present invention;

Fig. 2 is a flow chart of the present invention;

Fig. 3 is the flowchart of training classification model according to DCD optimization algorithm;

Fig. 4 is a flow chart of training a classification model according to the ASGD optimization algorithm.

Detailed ways

Below in conjunction with specific embodiment, further illustrate the present invention, should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention, after having read the present invention, those skilled in the art will understand various equivalent forms of the present invention All modifications fall within the scope defined by the appended claims of the present application.

As shown in Figure 1, the digital media object classification method based on large interval distribution learning, first, the user prepares a digital media object library, and for each digital media object in it, obtains the corresponding category by labeling or crowdsourcing mark to form the training data. Next, convert the training digital media object into a feature representation, specifically, input the training digital media object into a feature extraction algorithm to obtain a feature vector of the digital media object. Then, all the feature vectors corresponding to the training digital media objects and their category marks are input into the training algorithm of the classification model, and the classification model can be obtained after the training. In the prediction phase, the user inputs the digital media objects to be predicted in the test digital media object library into the classification model, and the classification model outputs classification results.

The main flow of the present invention is shown in Figure 2. Step 1 is the starting action, and step 2 obtains the feature vector matrix of all training digital media objects and the class label vector Where X is a d×m real number matrix, the i-th column corresponds to the digital media object x _i , and y is an m-dimensional real number vector. Step 3 accepts user input, which includes selection of optimization algorithm, weight coefficients λ ₁ , λ ₂ , C of interval variance, interval mean and overall loss, and kernel function parameters (no parameters if linear kernel is selected). Step 4 makes a judgment based on the user's input. If DCD is selected as the optimization algorithm, then go to step 5, and the detailed description is shown in Figure 3; if ASGD is selected as the optimal algorithm, then go to step 6, and the detailed description is shown in Figure 4 shown. Step 7 uses the trained classification model to classify the digital media objects without class labels, step 8 outputs the classification result, and finally ends in step 9.

Fig. 3 illustrates how to train the classification model according to the DCD optimization algorithm, step 50 is the start action. In step 51, the kernel matrix G is calculated based on the eigenvector matrix X. The kernel function used here is specified by the user. Common ones include RBF kernel, polynomial kernel, Sigmoid kernel, linear kernel, etc., and each digital media object corresponds to G in G. A certain row and a certain column. In step 52, the solution β of the optimization problem is initialized as a vector of all 0s, and matrix H and vector p are calculated according to formula (1):

where Y is a diagonal matrix with y as the diagonal element, and e is an m-dimensional full 1 vector. The matrix H contains the information of the interval variance, and the vector p is also related to the interval mean, and they are also the quadratic and first-order items in the final objective function to be optimized. Step 53 judges whether β has converged, based on whether a certain norm (usually 2-norm) of the difference between the current β and the previous round of β is smaller than a preset threshold. If β has converged, then go to step 56, output β, and the training ends; otherwise, go to step 54. Steps 54 and 55 are the core parts of DCD. Since the objective function after the formalization of LDM is a convex quadratic function, and the constraints are decoupled upper and lower bound constraints, there is an advantage in choosing DCD as the optimization algorithm. Each time a variable is selected , keeping other variables unchanged, then only optimizing this variable is a problem of taking the minimum value of a one-dimensional quadratic function on a specified interval, and this problem has an analytical solution. Specifically, assuming that the current solution is β, the i-th dimension is randomly selected as the optimization variable, and the other dimensions are fixed, then there is the following update formula

${\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; new</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; H\&beta;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; ]</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

where [Hβ+β] _i is the i-th dimension of the vector Hβ+β, and h _ii is the i-th element on the diagonal of the matrix H. Step 54 randomly selects β _i as the optimization variable, step 55 updates β _i according to formula (2), and then returns to step 53 for iteration until convergence.

Fig. 4 illustrates how to train the classification model according to the ASGD optimization algorithm, step 60 is the start action. Step 61 initializes the solution w of the optimization problem as a vector of all 0s. Step 62 judges whether w has converged, based on whether a certain norm (usually 2-norm) of the difference between the current w and w of the previous round is smaller than a preset threshold. If w has converged, go to step 66, output w, and the training ends; otherwise, go to step 63. Step 63, step 64 and step 65 are the core parts of ASGD. The core idea of ASGD is to use the unbiased estimate of the gradient of the objective function to replace the gradient as the direction of descent, which can avoid the problem of time-consuming calculation of the gradient when the amount of data is large. , because unbiased estimates of the gradient are generally easy to compute. For SVM, ASGD only needs to randomly sample one sample in each round to obtain an unbiased estimate of the gradient of its objective function. LDM additionally introduces the interval mean and interval variance on the basis of it. The unbiased estimate of the interval mean gradient is obtained by randomly sampling a The sample can be obtained, and the unbiased estimation of the interval variance gradient needs to randomly sample two samples, which is step 63 . Assuming that the randomly sampled samples are x _i and x _j respectively, the unbiased estimate of the gradient of the objective function can be obtained through formula (3),

Among them, λ ₁ , λ ₂ , and C are the weight coefficients of interval variance, interval mean and overall loss respectively, and the set is the subscript set of samples with loss, which is step 64 . Then set the step size ηt=1/t, and update w according to formula (4) just like gradient descent.

${\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&eta;t</span>}<span\; class="notranslate">\; \&dtri;</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

This is step 65, after which it turns back to step 62 to iterate until convergence.

Claims (3)

1. the digital media object sorting technique based on large-spacing Distributed learning, is characterized in that:

First, what a is first set up and comprise digital media object message digit library of media objects as training data, each digital media object in described digital media object storehouse is with classification mark;

Then, convert training digital media object to character representation, specifically, training digital media object is input in feature extraction algorithm, obtain the proper vector of digital media object;

Then, all training digital media object characteristic of correspondence vectors and classification mark thereof are all inputted to the training algorithm into disaggregated model, after having trained, obtain disaggregated model; At forecast period, user is by digital media object input disaggregated model to be predicted, and disaggregated model is the classification mark of exportable its prediction;

When train classification models, by maximize margin average simultaneous minimization interval variance, the classification problem form of digital media object changes into a protruding double optimization problem the most at last, and according to the feature of whether using Non-linear Kernel function and training digital media object storehouse itself, provided and based on dual coordinates, declined respectively and the realization based on two kinds of optimizing algorithms of mean random Gradient Descent, user can select voluntarily according to actual conditions; If user selects Non-linear Kernel function, while training, select DCD as optimizing algorithm; If user selects linear kernel function, and training digital media object storehouse sample is a lot of or feature is very sparse, selects ASGD as optimizing algorithm, otherwise still select DCD as optimizing algorithm while training.

2. the digital media object sorting technique based on large-spacing Distributed learning as claimed in claim 1, is characterized in that:

According to DCD optimizing algorithm train classification models step, be:

Step 51, calculates nuclear matrix G based on eigenvectors matrix X, and each digital media object is corresponding certain a line and a certain row in G;

Step 52, is initialized as full 0 vector by the optimum solution β of optimization problem, presses (1) formula compute matrix H and vectorial p:

Wherein Y be take the diagonal matrix that y is diagonal entry, and e is that m ties up complete 1 vector;

Step 53, judges whether β restrains, and whether certain norm according to being the difference of current β and last round of β of judgement is less than predefined threshold value; If β restrains, go to step 56, output β, training finishes; Otherwise go to step 54;

Step 54, establishing current solution is β, chooses at random i dimension β _ias optimized variable, other dimension immobilizes,

Step 55, upgrades β according to (2) formula _i,

New formula more

${\mathrm{\&beta;}}_i^{\mathrm{new}}=\min (\max ({\mathrm{\&beta;}}_i-{[\mathrm{H\&beta;}+\mathrm{\&beta;}]}_i/h_{\mathrm{ii}},0),C),---\left(2\right)$

Go back to afterwards that step 53 is carried out iteration until convergence;

Step 56, output β, training finishes.

3. the digital media object sorting technique based on large-spacing Distributed learning as claimed in claim 1, is characterized in that:

According to the step of ASGD optimizing algorithm train classification models, be:

Step 61, is initialized as full 0 vector by the optimum solution w of optimization problem;

Step 62, judges whether w restrains, and basis for estimation is whether certain norm of the difference of current w and last round of w is less than predefined threshold value; If w restrains, go to step 66, output w, training finishes; Otherwise go to step 63;

Step 63, from training data, stochastic sampling goes out two digital media object characteristic of correspondence vector x _iand x _j;

Step 64, through type (3) just can obtain target function gradient without inclined to one side estimation,

Wherein, C is the weight coefficient of the overall loss that sets in advance of user, set it is the indexed set of lossy sample;

Step 65, arranges step-length η t=1/t, by formula (4), upgrades w,

$w_{t+1}=w_t-\mathrm{\&eta;t}\&dtri;g(w,x_i,x_j)---\left(4\right)$

, go back to afterwards that step 62 is carried out iteration until convergence;

Step 66, output w, training finishes.