Background technology
Classifier design is the research emphasis in the computer patterns identification field, because sorter is the basic tool as pattern identification research.Generally speaking, present main two critical problems of classifier algorithm: the popularization ability of (1) sorter; (2) new samples is carried out the spent time of branch time-like.
In general, the popularization ability of sorter is exactly the ability of sorter prediction unknown sample classification, the i.e. height of classification accuracy rate.Based on Statistical Learning Theory and the theoretical svm classifier device of structural risk minimization; Because it has advantages such as globally optimal solution and excellent popularization ability; Become the most successful in recent years sorter, be widely used in various fields such as artificial intelligence, machine learning, Neurobiology, medical science.Its initial motivation is exactly two types of set of data samples for linear separability, finds out the largest interval that can separate these two types of samples.But in practical application, many grouped datas are linear inseparable.For this reason; People have introduced the notion of kernel function; The data of luv space are mapped in higher-dimension even the infinite dimension nuclear space through nuclear; And these are mapped to data in the nuclear space and can be regarded as that linear separability or approximately linear can divide, thereby have solved for the inseparable classification of Data problem of linearity.Though successfully expanded the range of application of linear SVM based on the SVM of kernel method, and the excellent popularization ability is arranged, it still has following three big shortcomings:
1) difficulty selected of kernel function and parameter thereof: a lot of kernel functions are arranged at present, and like polynomial kernel function, batten kernel function, basic kernel function etc. radially, each kernel function has parameter separately again.Kernel function and parameters of choice thereof have very important effect for the popularization ability of svm classifier device.How to select suitable kernel function and parameter thereof to make the popularization ability of sorter reach and preferably remain a difficult problem that needs to be resolved hurrily.
2) need occupy very big internal memory during training classifier: need store an O (l when the training based on the SVM of kernel function
2) non-sparse nuclear matrix, wherein l is the number of training sample.This just makes SVM can not handle the classification problem of big data quantity.
3) the classification time complexity is high: the svm classifier device based on kernel function is carrying out the branch time-like to new samples, needs to calculate the kernel function between this new samples and all support vectors.Number linear dependence that so its complicated classification degree is a calcaneus branches holds vector.And the growth of the number of Steinwart proof support vector increases linear with the training sample number at least.Therefore, the sorter that big data quantity trains out, to the classification time of new data sample will be very slow, this just makes it can not apply in some real-time mode recognizing systems, such as pedestrian detecting system and aircraft vision navigation system.
In order to overcome the above-mentioned defective based on kernel function SVM, the method for some structural classification devices in original data space has been carried.Fu has proposed the linear svm classifier device of a mixing based on the joint distribution model of training data and label.But, because the joint distribution of data is difficult to estimate that when the DATA DISTRIBUTION of reality did not conform to the probability model of hypothesis, the popularization ability of sorter will reduce greatly.Li has proposed a piecewise linear classifier---many convex lines property perceptron based on the notion of " the protruding branch ".But because this sorter do not introduce the notion at soft interval, so it can not handle those two types of grouped datas that mutual overlapping is arranged, so just greatly limitations range of application.
Summary of the invention
The present invention provides a kind of mode identification method---H-MLSVMs with mixed linear svm classifier device of hierarchical structure, and it has the hierarchical tree structure, and each node place all corresponding the linear svm classifier device of a mixing.With respect to other classical SVM method; This sorting technique has and based on the SVM method of kernel function popularization ability about the same; Do not select difficulty, take very shortcoming such as big internal memory and do not exist kernel function and parameter thereof; Particularly have the classification time complexity with rank such as linear SVM, make this sorting technique divide time-like more effective new samples.
To achieve these goals, the technical scheme below the present invention has adopted:
Mode identification method with mixed linear svm classifier device of hierarchical structure, the sorter at its each node place all corresponding an iterative process.Iterative process mainly comprises following two steps each time:
(1) uses the neighbour's geometric relationship of training data and the error in classification of current H-MLSVMs sorter, upgrade the training weight of each sample;
(2) utilize the sample training that has weight to go out the heavy linear svm classifier device of a cum rights, this sorter is embedded among the current H-MLSVMs, and upgrades the error in classification of sorter, turn back to the first step.
Wherein, H-MLSVMs root node sorter is to obtain through the linear SVM that trains equal weight.Above-mentioned iterative process until among the H-MLSVMs number of all node place sorters reach a given integer, perhaps when the increasing degree of accuracy on the test data set is within enough little scope, stop.
Furtherly, described step (1) specifically comprises following substep:
At first, select " training centre's sample " through the neighbour's geometric relationship of training data and the error in classification of current H-MLSVMs sorter, promptly near the zone this " training centre's sample " is the most serious zone of the wrong branch of current H-MLSVMs sorter;
Secondly, according to each sample and the distance between the last step selected " training centre's sample ", calculated the training weight of each sample;
At last, in order to prevent to train weight to be partial to a certain type, in all kinds of, will train weight to do normalization and handle.
Furtherly, described step (2) specifically comprises following substep:
At first, the training sample that has weight above the utilization trains the heavy linear svm classifier device of a cum rights;
Secondly, combine " training centre's sample " to belong to the sorter at father and mother's node place of node, obtain a mixed linear svm classifier device;
Once more, with the sorter of this mixed linear svm classifier device as node place, " training centre's sample " place;
At last, upgrade the error in classification of sorter.
The invention has the beneficial effects as follows:
1, sorting algorithm proposed by the invention is a kind of method in common, and the arbitrary classification data type all is suitable for.
2, the present invention proposes a kind of self-adaptation adjustment weight strategy; Can automatically adjust the training weight of each sample according to current error in classification; The method that weight is set than classics; Owing to not only considered error in classification, and incorporated the geometric relationship of data, make this weight method to set up to noise point robust more.
3, the present invention has adopted a kind of sorter of hierarchical tree structure; Wherein each node place all exists a mixed linear svm classifier device; Its basic thought is exactly that layering ground is separated the positive sample of mistake branch from negative sample, and the negative sample that perhaps mistake is divided is separated from positive sample.Linear svm classifier device than classics has greatly improved the popularization ability to the inseparable data of linearity; Svm classifier device than classics based on kernel function; The popularization ability of this H-MLSVMs sorter can be similar to the popularization ability that reaches nuclear svm classifier device; And overcome nuclear SVM in training process kernel function and selection of parameter thereof difficulty, consume shortcoming such as internal memory; The more important thing is to make the classification time complexity that needs reduce greatly, thereby can be used in the real-time mode recognizing system.
Embodiment
Through embodiment the present invention is carried out concrete description below; Only be used for the present invention is further specified; Can not be interpreted as the qualification to protection domain of the present invention, the technician in this field can make some nonessential improvement and adjustment to the present invention according to the content of foregoing invention.
Fig. 1 is the training process flow diagram of H-MLSVMs sorter of the present invention, and it is a kind of mode identification method with mixed linear svm classifier device of hierarchical structure.Fig. 2 is that the two-dimentional example of sorter training process of the present invention is showed.Fig. 3 is the process flow diagram of H-MLSVMs sorter to the assorting process of new samples.The hardware and the programming language of the concrete operation of method of the present invention do not limit, and can accomplish with any language, and other mode of operation repeats no more for this reason.
Embodiments of the invention adopt a Pentium 4 computing machine with 3.2G hertz central processing unit and 1G byte of memory also to work out the working routine of H-MLSVMs sorter with the Matlab language, have realized method of the present invention.
Mode identification method with mixed linear svm classifier device of hierarchical structure of the present invention mainly comprises following two steps: the renewal of the renewal of training sample weight and hierarchical structure sorter H-MLSVMs.
Before introducing concrete steps, below we introduce earlier the meaning of the symbol that will use.
S: the training sample set, promptly
I: the node in the expression H-MLSVMs sorter.I=[0] represents root node, as
Its division of fruit, [0,1] and [0 ,-1] is its left child's node and right child's node just.Similarly, if node [0,1] divides again, then [0,1,1] and [0,1 ,-1] is its left child's node and right child's node just.If a node no longer divides, then this node just is called leafy node.The number of element is being represented the number of plies of node I in H-MLSVMs among the I.Shown in Fig. 2 (f), node [0,1,1,1] is at the 4th layer of H-MLSVMs;
F
(t): in the process of training H-MLSVMs sorter, the linear SVM that the cum rights that is calculated during the t time iteration is heavy;
F
I: at the sorter (shown in Fig. 2 (f)) at node I place, wherein the leafy node place does not have sorter;
S
I: be divided into node I place sample set.Shown in Fig. 2 (f), all training samples the most all are divided into the root node place, that is:
Sorter f at the root node place
[0]With S set
[0]Be divided into S set
[0,1]And S
[0 ,-1]Sorter f at [0,1] node place
[0,1]Again with S
[0,1]Be divided into S
[0,1,1]And S
[0,1 ,-1]Similarly, the set at other node places also can obtain.We can find out that the set at leaf node place is non-intersect in twos, and their union has constituted whole training samples.If among the note leafy node I last element be I (| I|), then I (| I|) determined S set
IThe category attribute of middle sample.I (| I|)=1 (I (| I|)=-1) represents S
IMiddle sample is divided into positive type (negative type) by the H-MLSVMs sorter;
The number of all node place sorters among the M:H-MLSVMs;
F
t: the set of node place sorter among the H-MLSVMs after the t time iteration, shown in Fig. 2 (f), F
4={ f
[0], f
[0,1], f
[0,1,1], f
[0 ,-1] }
Wherein
Be illustrated in H-MLSVMs after the t time iteration
Sorter is to sample x
iPredicted value.
Red circle is represented positive sample among Fig. 2, blue fork expression negative sample.The training process and the classification process of H-MLSVMs sorter are described with reference to figure 2 below, and concrete steps are described below:
Initial H-MLSVMs: all training samples all are divided into the root node place, that is:
Training sample with equal weight trains a linear svm classifier device f
(1)(shown in Fig. 2 (a)), and give root node place sorter, i.e. f with it
[0]=f
(1)f
[0]With S set
[0]Be divided into S set
[0,1]And S
[0 ,-1], S wherein
[0,1]={ x|f
[0](x)>=0, x ∈ S
[0], S
[0 ,-1]={ x|f
[0](x)<0, x ∈ S
[0], can find out S
[0,1]In the negative sample that is divided by mistake is arranged in a large number, S
[0 ,-1]In the positive sample that is divided by mistake is arranged in a large number.This moment H-MLSVMs F
1Structure such as Fig. 2 (f) shown in, F
1Predicted value to training sample is V
(1)=F
1(S
[0])=f
[0](S
[0]).
Get into the cycle stage below, it mainly contains following two steps
(1) uses the neighbour's geometric relationship of training data and the error in classification of current H-MLSVMs sorter, upgrade the training weight of each sample.It mainly comprises:
A) find out " training centre's sample " (shown in Fig. 2 (b) Smalt triangle)
Neighbours, it is defined as N
ε(x
i)={ x
j|| | x
j-x
i||<ε, y
i=y
i, (x
j, y
j) ∈ S}, the general value of ε does
B) according to the distance calculation sample weights of sample, promptly apart from " training centre's sample "
C) train weight to be partial to a certain type in order to prevent; To train weight to do normalization processing
shown in Fig. 2 (b) in all kinds of, the size of mark is being represented the size of sample weights.
Can find out that from top computation process we are with sorter F
1Mistake divides the sample in the most serious zone to give very big weight.
(2) utilize the sample training that has weight to go out the heavy linear svm classifier device of a cum rights, this sorter is embedded among the current H-MLSVMs, and upgrade the error in classification of sorter.It mainly comprises:
A) shown in Fig. 2 (b), heavy with cum rights
Training sample train the heavy linear svm classifier device f of cum rights
(2)(rhodo colo(u)r streak among Fig. 2 (b));
B) suppose x
c∈ S
I, S set promptly is described
IThe middle wrong sample that divides is the most serious.Make I
1=I (1:|I|-1), I
2=I (| I|), I then
1Be father and mother's node of I.Datum node I
1Place's sorter obtains a mixed linear svm classifier device.If I (| I|)=1, then this mixed linear svm classifier device does
If I (| I|)=-1, then this mixed linear svm classifier device does
In Fig. 2 (b), because I=[0,1], so mixed linear svm classifier device is min{f
(2), f
[0];
C) the linear svm classifier device min{f that obtains above the general
(2), f
[0]Give the node sorter f that [0,1] is located
[0,1], f
[0,1]Further with S set
[0,1]Be divided into S
[0,1,1]And S
[0,1 ,-1], S wherein
[0,1,1]={ x|f
[0,1](x)>=0, x ∈ S
[0,1], S
[0,1 ,-1]={ x|f
[0,1](x)<0, x ∈ S
[0,1], can find out, with f
[0,1]S set
[0,1]A middle part is by F
1The wrong negative sample that divides is separated.The H-MLSVMs of this moment is F
2=F
1∪ { f
[0,1], can find out F
2Compare F
1Had more a sorter f of node place
[0,1]Thereby, improved classification accuracy rate.Shown in its structure such as Fig. 2 (f);
D) calculate F
2To the predicted value of training sample, promptly
Return (1).
Repeat to be similar to the process of step (1) (2), we have obtained H-MLSVMs sorter F
3(shown in Fig. 2 (c)) and F
4(shown in Fig. 2 (d)), its structure is respectively shown in Fig. 2 (f).Above-mentioned iterative process until among the H-MLSVMs number of all node place sorters reach a given integer, perhaps when the increasing degree of accuracy on the test data set is within enough little scope, stop.In test, we adopt first kind of iteration stopping method.Fig. 2 (e) is the classification boundaries through H-MLSVMs sorter after 25 iteration.
A sample needs H-MLSVMs to divide time-like when newly arriving; Let this new sample arrive certain leafy node from the root node of hierarchical structure sorter H-MLSVMs always; Judge its classification according to the functional value that calculates, concrete classification process is as shown in Figure 3.
Because F
MIn comprised the sorter at all node places among the H-MLSVMs, so among Fig. 2 "
" indicate that x has arrived leafy node, export its functional value and get final product, if its functional value more than or equal to zero, it be positive type, otherwise is negative class." f
[I, 1]∈ F
M(f
[I ,-1]∈ F
M) " mean that x does not also arrive leafy node, need with sorter " f
[I, 1](f
[I ,-1]) " further divide.Shown in sample x (pentagram) among Fig. 2 (c), the path that first three step is walked in its assorting process is shown in Fig. 2 (f) arrow.