CN102708384B - Bootstrapping weak learning method based on random fern and classifier thereof - Google Patents

Abstract

The invention provides a bootstrapping weak learning method based on a random fern and a classifier thereof, which belong to the technical field of computer graphic identification. In graphic identification, positive and negative patterns are generally judged by adopting the mean value distances of weighting and Gaussian probability distribution of a weak classifier. Or classification trees are taken as weak learners, nodes are partitioned by using the partitioning rule of error metric reduction maximization, and the weak classifiers are improved into strong classifiers. However, the weak learning methods have the defects of low convergence rate, poor accuracy and low computing efficiency. Through steps for selecting image characteristics, constructing a random fern and a weak learning method based on a random fern, establishing a weak classifier and a result classifier and the like, the image mode identification problems of complexity of an imaging environment and strict requirement on the operand can be well solved, a rapidly-converging and efficient bootstrapping weak learning method is realized, and a high-accuracy classifier for real-time processing is obtained. The method and the classifier are mainly applied to identification occasions in various modes.

Description

Learning method and sorter thereof a little less than a kind of bootstrapping based on random fern

Technical field

The invention belongs to computer graphic image mode identification technology, particularly machine learning, computer vision technique.

Background technology

Along with the development of computer technology, studying complicated information process becomes possibility.An important form of information process is exactly pattern-recognition, i.e. the identification of main body to environment and object, and assorting process is the basic task of pattern-recognition.At present, Boosting bootstrapping sorting technique (mainly referring to AdaBoost self-adaptation Bootload here), because of its actual application ability in detection and Identification, obtains applying more and more widely in pattern-recognition and machine learning task, as medical image analysis, optical character identification, speech recognition, handwriting recognition, recognition of face, fingerprint recognition, iris recognition, document classification, Photogrammetry and Remote Sensing, internet search engine, credit scoring etc.In the training process of Boosting bootstrapping sorter, different weak learning methods will determine training speed, convergence and the accuracy rate of Boosting bootstrapping sorter.Weak learning method is determining of the selection based on feature and threshold value mainly.

Leshem in traffic management information system, utilizes weak learner to train road traffic data AdaBoost algorithm application, and predicted link magnitude of traffic flow situation, obtains good effect.Lin by RealAdaBoost algorithm application in CBIR system, by the classification phrase to image, train, reach and fall low noise effect, experiment shows to increase than KNN (K-nearest Neighbor) sorting algorithm accuracy.The people such as Dai in area image retrieval, by using AdaBoost Weak Classifier to feature repetition training, obtain having the strong classifier compared with mistake minute rate by AdaBoost algorithm application, thereby return to more accurate Query Result.In order to solve different Fusion Features classification problems, the people such as Yin have proposed a kind of improved boosting algorithm, use a Weak Classifier only some feature sets to be trained, finally according to weight, these Weak Classifiers are combined into a strong classifier, the method has obtained good effect in Handwritten Digit Recognition.The sample weights update method that the people such as Viola have proposed, reduced, and mis-classification sample weights is constant by correct classification samples weight.What AdaBoost will solve in the process of learning training is each sample distribution problem of taking turns sample training collection, and wherein the processing of the renewal of the weight of positive negative sample and wrong minute rate is most important.Sample is carried out to two class divisions, to guarantee that sample accuracy rate that Weak Classifier is got is greater than the accuracy rate of random conjecture.The people such as Li Chuan have proposed the improvement AdaBoost algorithm for target detection problems, have adopted new parametric solution method, and the weighting parameters of Weak Classifier not only divides rate relevant with mistake, also relevant with its recognition capability that aligns sample.It is the AdaBoost algorithm distributing based on gaussian probability that the people such as Kim have proposed eigenwert, by the distribution of eigenwert and the mean distance of gaussian probability distribution, differentiates positive negative sample.The people such as Xie Hongyue propose a kind of AdaBoost sample threshold and biasing computing method, the method calculates corresponding sample threshold according to the size of sample weights, and be used for distinguishing the sample of correct classification and mis-classification, thereby make Weak Classifier divide accuracy, be greater than 50%.Clockwise sun waits people to propose the Adaboost detecting device based on learning a little less than many threshold values, the method adopts classification tree as weak learner, with greedy method, by error, estimate and reduce maximized criteria for classifying partitioning site, and generate thus Weak Classifier, then adopt RAB or GAB method, on the training set of data-oriented and label, these Weak Classifiers are promoted to strong classifier.But these weak learning methods or speed of convergence are slow, or accuracy rate is not high enough, or counting yield is low.

In view of the deficiency of above method, learning method a little less than the bootstrapping of the present invention's proposition based on random fern, the method fast convergence rate, counting yield is high, and the bootstrapping sorter accuracy rate finally obtaining is high.Multi-features local binary feature (LBP) and class Lis Hartel that the inventive method adopts are levied (haar-like), and object regional area is measured and encoded; Then, the sample input using sample characteristic of correspondence value as random fern, random fern is counted all codings; Then,, according to the sample distribution of random fern record, select best coding as differentiation eigenwert (threshold value) corresponding to this feature; Finally, join in bootstrapping algorithm frame using the random fern Weak Classifier based on this Coding pattern features obtaining as anticipation function, all the other processing procedures are consistent with self-adaptation bootstrapping algorithm, finally obtain a strong classifier.Therefore, the inventive method can solve well imaging circumstances complexity and operand is required to strict image model identification problem.In addition, the inventive method is suitable for off-line and the training of online bootstrapping sorter simultaneously.

Summary of the invention

The object of this invention is to provide learning method and sorter thereof a little less than a kind of bootstrapping based on random fern, it can realize Fast Convergent and the weak learning method of bootstrapping efficiently, is processed in real time and sorter that accuracy rate is high.

It is that the technical scheme that goal of the invention adopts is that the present invention realizes: learning method and sorter thereof a little less than a kind of bootstrapping based on random fern, and described method comprises following content:

(1) select characteristics of image and the random fern of structure

The multi-features local binary feature (LBP) and the class Lis Hartel that adopt are levied (haar-like), these features are measured the gradient direction of image-region, and quantize 8 kinds of possible codings of output, as shown in Figure 1, Figure 2 and Figure 3, they are rectangular characteristic.Specifically, first respectively according to horizontal direction, vertical direction and angular direction is divided rectangular area for two parts equally, be labeled as respectively white and black, then, calculate white rectangle pixel value and that deduct black rectangle pixel value and, if this value is greater than 0, is encoded to 1, otherwise is 0.Therefore, for a feature, 3 directions obtain the binary coding of 3, and 2 ³=8 kinds of possible values.

As shown in Figure 4, the formation that is different from general random fern, here, random fern is reduced to and only comprises a feature, be the value that the value of fern equals comprised this feature, and in fern, the probability distribution of each coding determine number and mark thereof by follow-up this received sample of encoding.

(2) the weak learning method based on random fern

For bootstrapping sorting technique, it is in the nature binary classification, and therefore, sample set is comprised of positive sample and negative sample two class samples.According to the method for (1), in different positions, with different sizes and the ratio of width to height, and the iterations required according to Bootload, generate at random feature and the corresponding fern thereof of respective amount.

Weak learning method based on random fern is divided into following two aspects:

(i) random fern training

It to the training process of random fern, is also the process of selecting to differentiate eigenwert.Input sample set, calculates all sample characteristic of correspondence values to each fern, and this fern is recorded positive sample that each Coding pattern features is corresponding and the number of negative sample.If

for the number of the positive sample that receives of coding j,

for the number of the negative sample that receives of coding j, P ^jfor the probability of the corresponding positive sample of coding j, in fern, the probability calculation of each coding is:

$P^j=N_p^j/(N_p^j+N_N^j),$

J=0 wherein, 1 ..., M-1, here M=8.The differentiation eigenwert J of this fern is for making P ^jmaximum j value,

That is:

$J=\underset{j}{\arg \max }\left(P^j\right),$

Here J is the threshold value of this random fern.

(ii) random fern is evaluated

According to random fern, train determined threshold value, random fern to the evaluation method of sample is: if the eigenwert of sample equals the threshold value of this random fern, predict that so this sample is positive sample, and output+1, otherwise be predicted as negative sample, output-1.The techniqueflow chart of the method as shown in Figure 5.

Owing to adopting such Coding pattern features probability distribution as evaluation model, therefore this random fern structure can remain higher resolving ability.If add more rectangular characteristic type in above learning framework, increase corresponding random fern coding figure place, its resolution characteristic can further be strengthened.Meanwhile, the computing method of combining image integrogram, can calculate the eigenwert of sample rapidly, thereby realize sorter training fast and identifying.

(3) build Weak Classifier

For off-line Bootload, the building process of its Weak Classifier is as follows.

Iteration is each time carried out to following step:

1. according to set being formed by random fern of the method construct of (1);

2. input sample set, according to the method for (2), each random fern is calculated all coding probability and determines its threshold value;

3. each random fern carries out classification of assessment to sample set respectively according to its threshold value;

4. the Weak Classifier of selecting the random fern of wherein classification error rate minimum to produce and obtain as this iteration, as the anticipation function input of Bootload;

5. all the other processing procedures are identical with self-adaptation Bootload.

For online Bootload, sample set only comprises a sample at every turn, so the building process of its Weak Classifier is as follows.

Iteration is each time carried out to following step:

1. according to the method for feature selecting, to each iteration, to each feature selecting device, according to the method for (1), for it, generate a feature pool (comprising a plurality of random characters) and corresponding random fern set;

2. to each sample input, according to the method for (2), each random fern recalculated all coding probability and upgrades its threshold value, having upgraded corresponding Weak Classifier;

3. each random fern carries out classification of assessment to this sample;

4. in random fern set, the random fern of selection sort error rate minimum also preserves as the Weak Classifier of this selector switch;

5. the random fern of classification error rate maximum is deleted from random fern set, generate at random simultaneously and add a new fern;

6. all the other processing procedures are identical with online Bootload.

The operation of above feature pool must corresponding carrying out with the operation of random fern set.It should be noted that, these Weak Classifiers that comprise in online Bootload possibility after each iteration is identical also may be different, only after the online training examples of inputting is abundant, these weak minute devices will progressively settle out, that is to say, each selector switch will progressively select to obtain classifying random fern feature accurately.

(4) result sorter

Had in (3) and built the Weak Classifier obtaining, can generate final result sorter, and this result sorter has been required strong classifier.

For off-line Bootload, establish h _t(x) be the Weak Classifier that the acquistion of t ratation school is arrived, α _tweight for this Weak Classifier.Therefore, through T, training in rotation gets T optimum Weak Classifier, finally forms a strong classifier H ^strong(x):

$H^{\mathrm{strong}}\left(x\right)=\mathrm{sign}\left(\underset{t=1}{\overset{T}{\mathrm{\Σ}}}{\mathrm{\α}}_t{h}_t\left(x\right)\right)$

For online Bootload, establish be n the Weak Classifier that selector switch obtains, α _nweight for this Weak Classifier.Therefore, through N selection, obtain N optimum Weak Classifier, finally form a strong classifier H ^strong(x):

$H^{\mathrm{strong}}\left(x\right)=\mathrm{sign}\left(\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_n{h}_n^{\mathrm{sel}}\left(x\right)\right)$

The h that more than forms result sorter _t(x) and

all to be obtained by the weak learning method based on random fern, their described features of all corresponding (1).

Therefore,, after the processing of above (1)-(4), can be processed in real time and sorter that classification accuracy is high.

The invention has the beneficial effects as follows:

It is complicated and operand is required to strict image model identification problem that the inventive method can solve imaging circumstances well, realizes Fast Convergent and the weak learning method of bootstrapping efficiently, processed in real time and sorter that accuracy rate is high.In addition, the inventive method is suitable for off-line and the training of online bootstrapping sorter simultaneously.

Accompanying drawing explanation

Fig. 1 is horizontal direction image feature of the present invention;

Fig. 2 is vertical direction image feature of the present invention;

Fig. 3 is diagonal angle of the present invention directional image feature;

Fig. 4 is the random fern of the present invention and Coding pattern features thereof;

Fig. 5 is the weak learning method process flow diagram that the present invention is based on random fern;

Embodiment

Method of the present invention is suitable for off-line and the training of online bootstrapping sorter simultaneously, can be used for various pattern-recognitions and computer vision field problem, comprise object video tracking, medical image analysis, optical character identification, handwriting recognition, recognition of face, fingerprint recognition, document classification, Photogrammetry and Remote Sensing etc.

With object video, be tracked as example: at the initial frame of following the tracks of constantly, by extracting corresponding positive and negative sample in the initial target location obtaining and location about thereof, and the method training classifier proposing according to the present invention.And in tracing process, in the region of search centered by last time definite target location, the sorter obtaining with this training carries out evaluation of classification to each position of this region of search, position with the highest position of evaluation of classification confidence value as current goal, thus realize the tracking of object video.

The inventive method can be passed through any computer programming language (as C language) programming and realize, and the categorizing system software based on this method can be realized real-time detection and Identification application in any PC or embedded system.

Claims (1)

1. a learning method a little less than the bootstrapping based on random fern, described method comprises following content:

(1) select characteristics of image and the random fern of structure

The multi-features local binary feature and the class Lis Hartel that adopt are levied, and these features are measured the gradient direction of image-region, and quantize 8 kinds of possible codings of output, and they are rectangular characteristic; Specifically, first respectively according to horizontal direction, vertical direction and angular direction is divided rectangular area for two parts equally, be labeled as respectively white and black, then, calculate white rectangle pixel value and that deduct black rectangle pixel value and, if this value is greater than 0, is encoded to 1, otherwise is 0;

Random fern is reduced to and only comprises a feature, and the value of fern equals the value of comprised this feature, and in fern, the probability distribution of each coding determines number and mark thereof by follow-up this received sample of encoding;

(2) the weak learning method based on random fern

Weak learning method based on random fern is divided into following two aspects:

(I) random fern training

It to the training process of random fern, is also the process of selecting to differentiate eigenwert; Input sample set, calculates all sample characteristic of correspondence values to each fern, and this fern is recorded positive sample that each Coding pattern features is corresponding and the number of negative sample, establishes

for the number of the positive sample that receives of coding j, for the number of the negative sample that receives of coding j, P ^jfor the probability of the corresponding positive sample of coding j, in fern, the probability calculation of each coding is:

J=0 wherein, 1 ..., M-1, here M=8; The differentiation eigenwert J of this fern is for making P ^jmaximum j value, that is:

Here J is the threshold value of this random fern;

(ii) random fern is evaluated

According to random fern, train determined threshold value, random fern to the evaluation method of sample is: if the eigenwert of sample equals the threshold value of this random fern, predict that so this sample is positive sample, and output+1, otherwise be predicted as negative sample, Shu Chu – 1;

(3) build Weak Classifier

For off-line Bootload, the building process of its Weak Classifier, carries out following step to iteration each time:

1. according to set being formed by random fern of the method construct of (1);

2. input sample set, according to the method for (2), each random fern is calculated all coding probability and determines its threshold value;

3. each random fern carries out classification of assessment to sample set respectively according to its threshold value;

4. the Weak Classifier of selecting the random fern of wherein classification error rate minimum to produce and obtain as this iteration, as the anticipation function input of Bootload;

5. all the other processing procedures are identical with self-adaptation Bootload;

For online Bootload, sample set only comprises a sample at every turn, so the building process of its Weak Classifier, and iteration is each time carried out to following step:

1. according to the method for feature selecting, to each iteration, to each feature selecting device, according to the method for (1), for it, generate a feature pool that comprises a plurality of random characters and corresponding random fern set;

2. to each sample input, according to the method for (2), each random fern recalculated all coding probability and upgrades its threshold value, having upgraded corresponding Weak Classifier;

3. each random fern carries out classification of assessment to this sample;

4. in random fern set, the random fern of selection sort error rate minimum also preserves as the Weak Classifier of this selector switch;

5. the random fern of classification error rate maximum is deleted from random fern set, generate at random simultaneously and add a new fern;

6. all the other processing procedures are identical with online Bootload;

(4) result sorter

For off-line Bootload, establish h _t(x) be the Weak Classifier that the acquistion of t ratation school is arrived, α _tweight for this Weak Classifier; Therefore, through T, training in rotation gets T optimum Weak Classifier, finally forms a strong classifier H ^strong(x):

For online Bootload, establish

be n the Weak Classifier that selector switch obtains, α _nfor the weight of this Weak Classifier, therefore, through N selection, obtain N optimum Weak Classifier, finally form a strong classifier H ^strong(x):

The h that more than forms result sorter _t(x) and

all to be obtained by the weak learning method based on random fern, their described features of all corresponding (1).

Images