CN102708384A - Bootstrapping weak learning method based on random fern and classifier thereof - Google Patents

Abstract

The invention provides a bootstrapping weak learning method based on a random fern and a classifier thereof, which belong to the technical field of computer graphic identification. In graphic identification, positive and negative patterns are generally judged by adopting the mean value distances of weighting and Gaussian probability distribution of a weak classifier. Or classification trees are taken as weak learners, nodes are partitioned by using the partitioning rule of error metric reduction maximization, and the weak classifiers are improved into strong classifiers. However, the weak learning methods have the defects of low convergence rate, poor accuracy and low computing efficiency. Through steps for selecting image characteristics, constructing a random fern and a weak learning method based on a random fern, establishing a weak classifier and a result classifier and the like, the image mode identification problems of complexity of an imaging environment and strict requirement on the operand can be well solved, a rapidly-converging and efficient bootstrapping weak learning method is realized, and a high-accuracy classifier for real-time processing is obtained. The method and the classifier are mainly applied to identification occasions in various modes.

Description

A kind of based on learning method and sorter thereof a little less than the bootstrapping of fern at random

Technical field

The invention belongs to the computer graphic image mode identification technology, particularly machine learning, computer vision technique.

Background technology

The development of Along with computer technology, the complicated information process of research becomes possibility.An important form of information process is exactly pattern-recognition, and promptly main body is to the identification of environment and object, and assorting process then is the basic task of pattern-recognition.At present, Boosting bootstrapping sorting technique (mainly referring to AdaBoost self-adaptation bootstrapping method here) obtains application more and more widely, like medical image analysis because of its actual application ability in detection and Identification in pattern-recognition and machine learning task; Optical character identification, speech recognition, handwriting recognition, recognition of face; Fingerprint recognition, iris recognition, document classification; Photogrammetry and Remote Sensing, internet search engine, credit scoring etc.In the training process of Boosting bootstrapping sorter, different weak learning methods will determine training speed, convergence and the accuracy rate of Boosting bootstrapping sorter.Weak learning method is mainly based on the selection of characteristic and confirming of threshold value.

Leshem in traffic management information system, utilizes weak learner to train the road traffic data AdaBoost algorithm application, and predicted link magnitude of traffic flow situation, obtains good effect.Lin in the CBIR system, trains the RealAdaBoost algorithm application through the classification phrase to image, reach the reduction anti noise, and experiment shows than KNN (K-nearest Neighbor) sorting algorithm accuracy and increases.People such as Dai in the area image retrieval, through using the AdaBoost Weak Classifier to the characteristic repetition training, obtain having the strong classifier than mistake branch rate with the AdaBoost algorithm application, thereby return accurate more Query Result.In order to solve different character integrated classification problem; People such as Yin have proposed a kind of improved boosting algorithm; Use a Weak Classifier only some feature sets to be trained; Finally according to weight these Weak Classifiers are combined into a strong classifier, this method has obtained effect preferably in Handwritten Digit Recognition.The sample weights update method that people such as Viola have proposed reduced by correct classification samples weight, and the mis-classification sample weights is constant.What AdaBoost will solve in the process of learning training is each sample distribution problem of taking turns the sample training collection, and wherein the processing of renewal of the weight of positive negative sample and wrong branch rate is most important.Sample is carried out two types of divisions, so that guarantee that sample accuracy rate that Weak Classifier gets is greater than the accuracy rate of conjecture at random.People such as Li Chuan have proposed the improvement AdaBoost algorithm to the target detection problem, have adopted new parametric solution method, and the weighting parameters of Weak Classifier is not only relevant with mistake branch rate, and is also relevant with its recognition capability that aligns sample.People such as Kim have proposed eigenwert and have been based on the AdaBoost algorithm that gaussian probability distributes, and differentiate positive negative sample through the distribution of eigenwert and the mean distance of gaussian probability distribution.People such as Xie Hongyue propose a kind of AdaBoost sample threshold and biasing computing method; This method calculates the corresponding sample threshold value according to the size of sample weights; And be used for distinguishing the sample of correct classification and mis-classification, thereby make Weak Classifier divide accuracy greater than 50%.People such as clockwise sun propose based on the Adaboost detecting device of learning a little less than many threshold values; This method adopts classification tree as weak learner; Estimate the maximized criteria for classifying of minimizing with the method for greediness with error and divide node; And generate Weak Classifier thus, adopting RAB or GAB method on the training set of giving given data and label, these Weak Classifiers to be promoted then is strong classifier.But these weak learning methods or speed of convergence are slow, or accuracy rate is not high enough, or counting yield is low.

In view of the deficiency of above method, the present invention proposes based on learning method a little less than the bootstrapping of fern at random, this method fast convergence rate, and counting yield is high, and the bootstrapping sorter accuracy rate that finally obtains is high.The characteristics of image that the inventive method adopts merges local binary characteristic (LBP) levies (haar-like) with a class Lis Hartel, and the object regional area is measured and encoded; Then, as the sample input of fern at random, fern is counted all codings at random with sample characteristic of correspondence value; Then, according to the sample distribution of the record of fern at random, selects the differentiation eigenwert (threshold value) of best coding as this characteristic correspondence; At last, the Weak Classifier of fern at random based on this eigenwert coding that obtains is joined in the bootstrapping algorithm frame as anticipation function, all the other processing procedures are consistent with self-adaptation bootstrapping algorithm, finally obtain a strong classifier.Therefore, the inventive method can solve the complicated and image model identification problem strict to operand of imaging circumstances well.In addition, the inventive method is suitable for off-line and the training of online bootstrapping sorter simultaneously.

Summary of the invention

The purpose of this invention is to provide a kind ofly based on learning method and sorter thereof a little less than the bootstrapping of fern at random, it can realize convergence fast and the weak learning method of bootstrapping efficiently, is handled in real time and sorter that accuracy rate is high.

The present invention realizes it being that the technical scheme that goal of the invention adopts is: a kind of based on learning method and sorter thereof a little less than the bootstrapping of fern at random, said method comprises following content:

(1) selects characteristics of image and construct fern at random

The characteristics of image that adopts merges local binary characteristic (LBP) and a type Lis Hartel is levied (haar-like); These characteristics are measured the gradient direction of image-region; And quantize 8 kinds of possible codings of output, like Fig. 1, Fig. 2 and shown in Figure 3, they are rectangular characteristic.Specifically, earlier respectively according to horizontal direction, vertical direction and diagonal are divided the rectangular area equally and are two parts; Be labeled as white and black respectively, then, calculate the white rectangle pixel value with deduct the black rectangle pixel value with; If this value greater than 0, then is encoded to 1, otherwise be 0.Therefore, for a characteristic, 3 directions obtain 3 binary coding, and promptly 2 ³=8 kinds of possible values.

As shown in Figure 4; The formation that is different from the general random fern, here, fern is reduced to and only comprises a characteristic at random; Be the value of the value of fern this characteristic of equaling to be comprised, and the probability distribution of each coding will be by the number and the mark decision thereof of follow-up should coding received sample in the fern.

(2) based on the weak learning method of fern at random

For the bootstrapping sorting technique, it is in the nature binary classification, and therefore, sample set is made up of positive sample and two types of samples of negative sample.According to the method for (1),, with different sizes and the ratio of width to height, and the iterations required, generate the characteristic and the corresponding fern thereof of respective amount at random according to the bootstrapping method in different positions.

Be divided into following two aspects based on the weak learning method of fern at random:

(i) fern training at random

To the training process of fern at random also is the process of selecting to differentiate eigenwert.The input sample set calculates all sample characteristic of correspondence values to each fern, the positive sample that this each eigenwert coding of fern record is corresponding and the number of negative sample.If

Be the number of the positive sample that receives of coding j,

Be the number of the negative sample that receives of coding j, P ^jBe the probability of the corresponding positive sample of coding j, then the probability calculation of each coding is in the fern:

$P^j=N_p^j/(N_p^j+N_N^j),$

J=0 wherein, 1 ..., M-1, M=8 here.The differentiation eigenwert J of this fern is for making P ^jMaximum j value,

That is:

$J=\underset{j}{\arg \max }\left(P^j\right),$

Here J is this threshold value of fern at random.

(ii) fern is estimated at random

Train determined threshold value according to fern at random, fern to the evaluation method of sample is at random: if the eigenwert of sample equals this threshold value of fern at random, predict that so this sample is positive sample, and output+1, otherwise be predicted as negative sample, output-1.The techniqueflow chart of this method is as shown in Figure 5.

Because adopt such eigenwert coding probability distribution as evaluation model, therefore this structure of fern at random can remain higher resolving ability.If in above learning framework, add more rectangular characteristic type, promptly increase accordingly fern coding figure place at random, its resolution characteristic can further be strengthened.Simultaneously, the computing method of combining image integrogram can be calculated the eigenwert of sample apace, thereby realize sorter training fast and identifying.

(3) make up Weak Classifier

For off-line bootstrapping method, the building process of its Weak Classifier is following.

Iteration is each time carried out following step:

1. the set of forming by fern at random according to one of the method construct of (1);

2. import sample set, according to the method for (2) to each at random fern calculate all coding probability and confirm its threshold value;

3. each at random fern respectively sample set is estimated classification according to its threshold value;

4. the Weak Classifier of selecting the minimum fern at random of wherein classification error rate to produce and obtain as this iteration is promptly imported as the anticipation function of bootstrapping method;

5. all the other processing procedures are identical with self-adaptation bootstrapping method.

For online bootstrapping method, sample set only comprises a sample at every turn, so the building process of its Weak Classifier is following.

Iteration is each time carried out following step:

1. according to the method for feature selecting,,, be that it generates feature pool (comprising a plurality of random characters) and the set of corresponding random fern according to the method for (1) promptly to each feature selecting device to each iteration;

2. to the input of each sample, according to the method for (2), to each at random fern recomputate all coding probability and upgrade its threshold value, promptly upgraded corresponding Weak Classifier;

3. each at random fern this sample is estimated classification;

4. the minimum fern at random of selection sort error rate also preserves as the Weak Classifier of this selector switch in fern is gathered at random;

5. the deletion from fern set at random of the classification error rate is maximum fern at random generates at random simultaneously and adds a new fern;

6. all the other processing procedures are identical with online bootstrapping method.

The operation of above feature pool must corresponding carrying out with the operation of fern set at random.Need to prove; These Weak Classifiers that comprised in online bootstrapping method possibility after each iteration is identical also maybe be different; Only after the online training examples of input is abundant; These weak devices that divide will progressively settle out, and that is to say that each selector switch will progressively select to obtain the fern characteristic at random of classifying accurately.

(4) sorter as a result

Had to make up the Weak Classifier that obtains in (3), then can generate final sorter as a result, and this as a result sorter be required strong classifier.

For off-line bootstrapping method, establish h _t(x) be the Weak Classifier that the acquistion of t ratation school is arrived, α _tWeight for this Weak Classifier.Therefore, training in rotation gets T optimum Weak Classifier through T, forms a strong classifier H at last ^Strong(x):

$H^{\mathrm{strong}}\left(x\right)=\mathrm{sign}\left(\underset{t=1}{\overset{T}{\mathrm{\Σ}}}{\mathrm{\α}}_t{h}_t\left(x\right)\right)$

For online bootstrapping method, establish

Be n the Weak Classifier that selector switch obtains, α _nWeight for this Weak Classifier.Therefore, obtain N optimum Weak Classifier, form a strong classifier H at last through N selection ^Strong(x):

$H^{\mathrm{strong}}\left(x\right)=\mathrm{sign}\left(\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_n{h}_n^{\mathrm{sel}}\left(x\right)\right)$

More than form the h of sorter as a result _t(x) and

All be by obtain their all corresponding (1) described characteristics based on the weak learning method of fern at random.

Therefore, through after the processing of above (1)-(4), can be handled in real time and sorter that classification accuracy is high.

The invention has the beneficial effects as follows:

The inventive method can solve the complicated and image model identification problem strict to operand of imaging circumstances well, realizes convergence fast and the weak learning method of bootstrapping efficiently, is handled in real time and sorter that accuracy rate is high.In addition, the inventive method is suitable for off-line and the training of online bootstrapping sorter simultaneously.

Description of drawings

Fig. 1 is a horizontal direction image characteristic of the present invention;

Fig. 2 is a vertical direction image characteristic of the present invention;

Fig. 3 is a diagonal characteristics of image of the present invention;

Fig. 4 is the present invention's fern and eigenwert coding thereof at random;

Fig. 5 the present invention is based on the weak learning method process flow diagram of fern at random;

Embodiment

Method of the present invention is suitable for off-line and the training of online bootstrapping sorter simultaneously, can be used for various pattern-recognitions and computer vision field problem, comprises the object video tracking; Medical image analysis, optical character identification, handwriting recognition; Recognition of face; Fingerprint recognition, document classification, Photogrammetry and Remote Sensing etc.

Be tracked as example with object video: at the initial frame of following the tracks of constantly, through extracting corresponding positive and negative sample in initial target location and the location about thereof that obtains, and the method training classifier that proposes according to the present invention.And in tracing process; Target location confirming with last time is in the region of search at center; The sorter that obtains with this training carries out evaluation of classification to each position of this region of search; With the position of the highest position of evaluation of classification confidence value, thereby realize the tracking of object video as current goal.

The inventive method can be passed through any computer programming language (like the C language) programming and realize, can in any PC or embedded system, realize real-time detection and Identification application based on the categorizing system software of this method.

Claims (1)

1. one kind based on learning method and sorter thereof a little less than the bootstrapping of fern at random, and said method comprises following content:

(1) selects characteristics of image and construct fern at random

The characteristics of image that adopts merges the local binary characteristic to be levied with a type Lis Hartel, and these characteristics are measured the gradient direction of image-region, and quantize 8 kinds of possible codings of output, and they are rectangular characteristic.Specifically, earlier respectively according to horizontal direction, vertical direction and diagonal are divided the rectangular area equally and are two parts; Be labeled as white and black respectively, then, calculate the white rectangle pixel value with deduct the black rectangle pixel value with; If this value greater than 0, then is encoded to 1, otherwise be 0;

Fern is reduced to and only comprises a characteristic at random, i.e. the value of the value of fern this characteristic of equaling to be comprised, and the probability distribution of each coding will be by the number and the mark decision thereof of follow-up should coding received sample in the fern;

(2) based on the weak learning method of fern at random

Be divided into following two aspects based on the weak learning method of fern at random:

(i) fern training at random

To the training process of fern at random also is the process of selecting to differentiate eigenwert.The input sample set calculates all sample characteristic of correspondence values to each fern, and the positive sample that this each eigenwert coding of fern record is corresponding and the number of negative sample are established

Be the number of the positive sample that receives of coding j,

Be the number of the negative sample that receives of coding j, P ^jBe the probability of the corresponding positive sample of coding j, then the probability calculation of each coding is in the fern:

$P^j=N_p^j/(N_p^j+N_N^j),$

J=0 wherein, 1 ..., M-1, M=8 here.The differentiation eigenwert J of this fern is for making P ^jMaximum j value, that is:

$J=\underset{j}{\arg \max }\left(P^j\right),$

Here J is this threshold value of fern at random;

(ii) fern is estimated at random

Train determined threshold value according to fern at random, fern to the evaluation method of sample is at random: if the eigenwert of sample equals this threshold value of fern at random, predict that so this sample is positive sample, and output+1, otherwise be predicted as negative sample, output-1;

(3) make up Weak Classifier

For off-line bootstrapping method, the building process of its Weak Classifier, carry out following step to iteration each time:

1. the set of forming by fern at random according to one of the method construct of (1);

2. import sample set, according to the method for (2) to each at random fern calculate all coding probability and confirm its threshold value;

3. each at random fern respectively sample set is estimated classification according to its threshold value;

4. the Weak Classifier of selecting the minimum fern at random of wherein classification error rate to produce and obtain as this iteration is promptly imported as the anticipation function of bootstrapping method;

5. all the other processing procedures are identical with self-adaptation bootstrapping method;

For online bootstrapping method, sample set only comprises a sample at every turn, so the building process of its Weak Classifier, and iteration is each time carried out following step:

1. according to the method for feature selecting,,, be that it generates feature pool (comprising a plurality of random characters) and the set of corresponding random fern according to the method for (1) promptly to each feature selecting device to each iteration;

2. to the input of each sample, according to the method for (2), to each at random fern recomputate all coding probability and upgrade its threshold value, promptly upgraded corresponding Weak Classifier;

3. each at random fern this sample is estimated classification;

4. the minimum fern at random of selection sort error rate also preserves as the Weak Classifier of this selector switch in fern is gathered at random;

5. the deletion from fern set at random of the classification error rate is maximum fern at random generates at random simultaneously and adds a new fern;

6. all the other processing procedures are identical with online bootstrapping method;

(4) sorter as a result

For off-line bootstrapping method, establish h _t(x) be the Weak Classifier that the acquistion of t ratation school is arrived, α _tWeight for this Weak Classifier.Therefore, training in rotation gets T optimum Weak Classifier through T, forms a strong classifier H at last ^Strong(x):

$H^{\mathrm{strong}}\left(x\right)=\mathrm{sign}\left(\underset{t=1}{\overset{T}{\mathrm{\Σ}}}{\mathrm{\α}}_t{h}_t\left(x\right)\right)$

For online bootstrapping method, establish

Be n the Weak Classifier that selector switch obtains, α _nBe the weight of this Weak Classifier, therefore, obtain N optimum Weak Classifier, form a strong classifier H at last through N selection ^Strong(x):

$H^{\mathrm{strong}}\left(x\right)=\mathrm{sign}\left(\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_n{h}_n^{\mathrm{sel}}\left(x\right)\right)$

More than form the h of sorter as a result _t(x) and

All be by obtain their all corresponding (1) described characteristics based on the weak learning method of fern at random.

Images