CN104102705A - Digital media object classification method based on large margin distributed learning - Google Patents

Abstract

The invention discloses a digital media object classification method based on large margin distributed learning and aims to solve the problem that marking of classes of digital media causes noise. The form of the classification problem of the digital media objects is finally reduced into a convex secondary optimization problem by maximizing a margin average and minimizing a margin variance; two optimization algorithms which are based on dual coordinate descent and average stochastic gradient descent respectively are achieved according to whether or not to use a nonlinear kernel function and according to features of a training digital media object base; users can make selections on their own according to actual conditions; if the users select the nonlinear kernel function, the DCD (dual coordinate descent) is selected as the optimization algorithm for training; if the users select a linear kernel function, the training digital media object base provide many samples or has few features, the ASGD (average stochastic gradient descent) is selected as the optimization algorithm for training, and if not, the DCD is still selected as the optimization algorithm.

Description

A kind of digital media object sorting technique based on large-spacing Distributed learning

Technical field

The present invention relates to a kind of digital media object sorting technique, particularly a kind of digital media object sorting technique based on large-spacing Distributed learning.

Background technology

Instantly human society has entered the digitizing stage comprehensively, the media such as the image that is used at present diffusing information, text, video, audio frequency all record, process with binary-coded form, and image, text, video, audio frequency after these codings are referred to as digital media object.Digital media object because of its have figure, literary composition, sound, as and luxuriant three-dimensional performance feature, be widely used in all trades and professions, as remote sensing observing and controlling, internet site, Digital Television, telephone communication etc.These industries all can accumulate a large amount of data every day, therefore, along with the continuous expansion of data volume, how digital media object is carried out to organization and administration effectively and become more and more important, and its key problem is exactly the classification of digital media object.The classification of science both can facilitate for storing these digital media object; After service as in Digital Media retrieval, also can provide more quickly the result for retrieval of better effects if.In the classification task of digital media object, each digital media object can have a corresponding classification mark, and these classification marks are normally undertaken by people that manual mark obtains, and therefore inevitably can introduce some noises.Traditional sorting technique based on large-spacing, as support vector machine (all brief note is SVM below), because it has considered the interval of single sample, therefore more responsive to noise ratio, be not suitable for being directly used for digital media object to classify.Based on this discovery, the present invention proposes a kind of digital media object sorting technique based on large-spacing Distributed learning, the method is by utilizing whole information spaced apart, rather than the interval of single sample, therefore avoid the sensitivity to noise, solved well the problem of digital media object classification.

Summary of the invention

Goal of the invention: the classification mark of considering digital media object contains many noises conventionally, the present invention is based on the thought of large-spacing Distributed learning, has proposed a kind of digital media object sorting technique to insensitive for noise.The method is by making full use of whole information spaced apart, and maximize margin average simultaneous minimization interval variance, has avoided the sensitivity to noise, has solved well the problem of digital media object classification.

Technical scheme: a kind of digital media object sorting technique based on large-spacing Distributed learning, first, user is first ready to a digital library of media objects, and wherein each digital media object is with classification mark, and these are exactly training data.Then, convert training digital media object to character representation, specifically, training digital media object is input in feature extraction algorithm, obtain the proper vector of digital media object.The feature extracting method of digital media object has a variety of, can be by the corresponding feature of method, and for example, for piece image, its brightness can be used as a feature of this object, and contrast can be used as another one feature.Remember that total Characteristic Number is d, so just each digital media object has been corresponded to a vector in d dimension Euclidean space.Then all training digital media object characteristic of correspondence vectors and classification mark thereof are all inputted to the training algorithm into disaggregated model, after having trained, just can obtain disaggregated model.At forecast period, user is by digital media object input disaggregated model to be predicted, and disaggregated model is the classification mark of exportable its prediction.When train classification models, in order to overcome the noise problem of digital media object classification mark, the present invention is based on the thought of large-spacing Distributed learning, a kind of digital media object sorting technique LDM to insensitive for noise is proposed, by maximize margin average simultaneous minimization interval variance, the classification problem form of digital media object changes into a protruding double optimization problem the most at last, and according to whether using the feature in Non-linear Kernel function and training digital media object storehouse itself (as number of samples, the sparse property of feature etc.), provided and based on dual coordinates, declined (all brief note is DCD below) respectively and the realization based on two kinds of optimizing algorithms of mean random Gradient Descent (all brief note is ASGD below), user can select voluntarily according to actual conditions.If user selects Non-linear Kernel function, while training, select DCD as optimizing algorithm; If user selects linear kernel function, and training digital media object storehouse sample is a lot of or feature is very sparse, selects ASGD as optimizing algorithm, otherwise still select DCD as optimizing algorithm while training.

Beneficial effect: compared with prior art, the present invention makes full use of the information spaced apart in training digital media object storehouse, by maximize margin average simultaneous minimization interval variance, overcome the noise problem of classification mark in digital media object classification problem, also keep the original advantage of SVM simultaneously, finally obtained good classifying quality.

Accompanying drawing explanation

Fig. 1 is principle of the invention process flow diagram;

Fig. 2 is process flow diagram of the present invention;

Fig. 3 is according to the process flow diagram of DCD optimizing algorithm train classification models;

Fig. 4 is according to the process flow diagram of ASGD optimizing algorithm train classification models.

Embodiment

Below in conjunction with specific embodiment, further illustrate the present invention, should understand these embodiment is only not used in and limits the scope of the invention for the present invention is described, after having read the present invention, those skilled in the art all fall within the application's claims limited range to the modification of the various equivalent form of values of the present invention.

As shown in Figure 1, the digital media object sorting technique based on large-spacing Distributed learning, first, user is first ready to a digital library of media objects, for each digital media object wherein, by mark or mass-rent method, obtain corresponding classification mark, form training data.Then, convert training digital media object to character representation, specifically, training digital media object is input in feature extraction algorithm, obtain the proper vector of digital media object.Then all training digital media object characteristic of correspondence vectors and classification mark thereof are all inputted to the training algorithm into disaggregated model, after having trained, just can obtain disaggregated model.At forecast period, user is by the digital media object input disaggregated model to be predicted in test digital media object storehouse, disaggregated model output category result.

Main flow process of the present invention as shown in Figure 2.Step 1 is origination action, and step 2 obtains the eigenvectors matrix of all training digital media object with classification label vector wherein X is the real number matrix of d * m, and i is listed as corresponding digital media object x _i, y is the real number vector of m dimension.Step 3 is accepted user's input, and user's input comprises the selection of optimizing algorithm, the weight coefficient λ of interval variance, interval average and overall loss ₁, λ ₂, C and kernel functional parameter (if selecting printenv of linear kernel).Step 4 makes a decision according to user's input, if select DCD as optimizing algorithm, goes to step 5, and it describes in detail as shown in Figure 3; If select ASGD as optimizing algorithm, go to step 6, it describes in detail as shown in Figure 4.Step 7 is used the disaggregated model training to not having the digital media object of classification mark to classify, and step 8 output category result, finally ends at step 9.

How Fig. 3 explanation is according to DCD optimizing algorithm train classification models, and step 50 is for starting action.In step 51, based on eigenvectors matrix X, calculate nuclear matrix G, kernel function used is specified by user here, common are RBF core, polynomial kernel, Sigmoid core, linear kernel etc., and each digital media object is corresponding certain a line and a certain row in G.In step 52, the solution β of optimization problem is initialized as to full 0 vector, presses (1) formula compute matrix H and vectorial p:

Wherein Y be take the diagonal matrix that y is diagonal entry, and e is that m ties up complete 1 vector.The information that contains interval variance in matrix H, vectorial p is also relevant with interval average, and simultaneously they are also quadratic term in the objective function that finally will optimize and item once.Step 53 judges whether β restrains, and whether certain norm (conventionally selecting 2-norm) according to being the difference of current β and last round of β of judgement is less than predefined threshold value.If β restrains, go to step 56, output β, training finishes; Otherwise go to step 54.Step 54 and step 55 are cores of DCD, because the objective function after LDM formalization is Convex quadratic function, constraint is the bound constraint of uncoupling, therefore select DCD to have individual benefit as optimizing algorithm, choose a variable at every turn, keep other variable constant, so only optimize this variable and be exactly the problem that an one dimension quadratic function is got minimum value between designation area, this problem has analytic solution.Specifically, establishing current solution is β, chooses at random i dimension as optimized variable, and other dimension immobilizes, and has so following more new formula

${\mathrm{\β}}_i^{\mathrm{new}}=\min (\max ({\mathrm{\β}}_i-{[\mathrm{H\β}+\mathrm{\β}]}_i/h_{\mathrm{ii}},0),C),---\left(6\right)$

Wherein [H β+β] _ithe i dimension of vectorial H β+β, h _iii element on matrix H diagonal line.Step 54 is chosen β at random _ias optimized variable, step 55 is upgraded β according to (2) formula _i, go back to afterwards that step 53 is carried out iteration until convergence.

How Fig. 4 explanation is according to ASGD optimizing algorithm train classification models, and step 60 is for starting action.Step 61 is initialized as full 0 vector by the solution w of optimization problem.Step 62 judges whether w restrains, and basis for estimation is whether certain norm (conventionally selecting 2-norm) of the difference of current w and last round of w is less than predefined threshold value.If w restrains, go to step 66, output w, training finishes; Otherwise go to step 63.Step 63, step 64 and step 65 are cores of ASGD, the core concept of ASGD is without partially estimating, to substitute gradient as descent direction with target function gradient, in the time of can avoiding like this data volume very large, the problem that compute gradient is quite consuming time because gradient without partially estimating, be in general all easy to calculate.For SVM, ASGD is every take turns only need sample of stochastic sampling just can obtain its target function gradient without inclined to one side estimation, LDM has additionally introduced interval average and interval variance on its basis, wherein interval average gradient just can obtain by sample of stochastic sampling without partially estimating, interval variance gradient without estimation partially, need two samples of stochastic sampling, step 63 that Here it is.Suppose that sample that stochastic sampling goes out is for being respectively x _iand x _j, be exactly through type (3) just can obtain target function gradient without inclined to one side estimation,

λ wherein ₁, λ ₂, C is respectively the weight coefficient of interval variance, interval average and overall loss, set the indexed set of lossy sample, step 64 that Here it is.Step-length η t=1/t is set afterwards, the same with Gradient Descent just passable by formula (4) renewal w,

$w_{t+1}=w_t-\mathrm{\ηt}\▿g(w,x_i,x_j)---\left(8\right)$

Step 65 that Here it is, goes back to that step 62 is carried out iteration afterwards until convergence.

Claims (3)

1. the digital media object sorting technique based on large-spacing Distributed learning, is characterized in that:

First, what a is first set up and comprise digital media object message digit library of media objects as training data, each digital media object in described digital media object storehouse is with classification mark;

Then, convert training digital media object to character representation, specifically, training digital media object is input in feature extraction algorithm, obtain the proper vector of digital media object;

Then, all training digital media object characteristic of correspondence vectors and classification mark thereof are all inputted to the training algorithm into disaggregated model, after having trained, obtain disaggregated model; At forecast period, user is by digital media object input disaggregated model to be predicted, and disaggregated model is the classification mark of exportable its prediction;

When train classification models, by maximize margin average simultaneous minimization interval variance, the classification problem form of digital media object changes into a protruding double optimization problem the most at last, and according to the feature of whether using Non-linear Kernel function and training digital media object storehouse itself, provided and based on dual coordinates, declined respectively and the realization based on two kinds of optimizing algorithms of mean random Gradient Descent, user can select voluntarily according to actual conditions; If user selects Non-linear Kernel function, while training, select DCD as optimizing algorithm; If user selects linear kernel function, and training digital media object storehouse sample is a lot of or feature is very sparse, selects ASGD as optimizing algorithm, otherwise still select DCD as optimizing algorithm while training.

2. the digital media object sorting technique based on large-spacing Distributed learning as claimed in claim 1, is characterized in that:

According to DCD optimizing algorithm train classification models step, be:

Step 51, calculates nuclear matrix G based on eigenvectors matrix X, and each digital media object is corresponding certain a line and a certain row in G;

Step 52, is initialized as full 0 vector by the optimum solution β of optimization problem, presses (1) formula compute matrix H and vectorial p:

Wherein Y be take the diagonal matrix that y is diagonal entry, and e is that m ties up complete 1 vector;

Step 53, judges whether β restrains, and whether certain norm according to being the difference of current β and last round of β of judgement is less than predefined threshold value; If β restrains, go to step 56, output β, training finishes; Otherwise go to step 54;

Step 54, establishing current solution is β, chooses at random i dimension β _ias optimized variable, other dimension immobilizes,

Step 55, upgrades β according to (2) formula _i,

New formula more

${\mathrm{\β}}_i^{\mathrm{new}}=\min (\max ({\mathrm{\β}}_i-{[\mathrm{H\β}+\mathrm{\β}]}_i/h_{\mathrm{ii}},0),C),---\left(2\right)$

Go back to afterwards that step 53 is carried out iteration until convergence;

Step 56, output β, training finishes.

3. the digital media object sorting technique based on large-spacing Distributed learning as claimed in claim 1, is characterized in that:

According to the step of ASGD optimizing algorithm train classification models, be:

Step 61, is initialized as full 0 vector by the optimum solution w of optimization problem;

Step 62, judges whether w restrains, and basis for estimation is whether certain norm of the difference of current w and last round of w is less than predefined threshold value; If w restrains, go to step 66, output w, training finishes; Otherwise go to step 63;

Step 63, from training data, stochastic sampling goes out two digital media object characteristic of correspondence vector x _iand x _j;

Step 64, through type (3) just can obtain target function gradient without inclined to one side estimation,

Wherein, C is the weight coefficient of the overall loss that sets in advance of user, set it is the indexed set of lossy sample;

Step 65, arranges step-length η t=1/t, by formula (4), upgrades w,

$w_{t+1}=w_t-\mathrm{\ηt}\▿g(w,x_i,x_j)---\left(4\right)$

, go back to afterwards that step 62 is carried out iteration until convergence;

Step 66, output w, training finishes.