CN104992191A - Image classification method based on deep learning feature and maximum confidence path - Google Patents

Abstract

An image classification method based on a deep learning feature and a maximum confidence path belongs to the field of mode identification. The method comprises the steps of: training a convolutional neural network on a large enough image library; extracting an image feature by means of the trained convolutional neural network; calculating a mean vector of each class; performing iteration clustering on the mean vector that represents each class by means of a spectrum clustering algorithm so as to construct a visual tree; training svm for each non-leaf node of the tree; and for a given test image, from top to bottom, judging the probability of the test image to the corresponding node, and finding a leaf node with the biggest path probability, namely a final target class. The image feature is extracted by means of CNN, thereby achieving very good discrimination and robustness; a distance calculation formula of two classes is given out, the complexity of calculation is greatly optimized by means of derivation and the similarity of the classes is obtained, so that the visual tree is constructed by iteratively using the spectrum clustering algorithm; and the use of a visual relationship between the classes can achieve very good effects for large image classification.

Description

Based on the image classification method in the maximum confidence path of characteristic sum of degree of depth study

Technical field

The invention belongs to area of pattern recognition, especially relate to the image classification method in the maximum confidence path of characteristic sum based on degree of depth study that can be used for large-scale image classification.

Background technology

At computer vision field, Images Classification be one extremely important, be also very classical studying a question.But along with amount of images increases, image kind increases, large-scale image classification remains a very challenging task at present.Because amount of images increases, calculated amount also can increase, and the time of needs also can increase, also high to hardware requirement, if still adopt classic method to train a multi classifier to be used as final classification foundation, the series of problems such as computational complexity and accuracy will be there will be.So be necessary to design a set of new classification framework and sorting technique.

Compared with traditional Images Classification task, the difficult point of large-scale Images Classification task is: (1), when amount of images and kind increase, calculated amount also increases thereupon, higher to hardware requirement.(2) from a lot of target class, identify that a kind of target class identifies that than from a small amount of target class a kind of target class difficulty increases a lot, because during class increasing number, certainly exist a kind of phenomenon: some class is very similar, some class otherness is very large.These similar classes seriously affect the accuracy rate of classification.Existing method is mainly divided into two large classes, one class utilizes the degree of depth to learn to build degree of depth convolutional neural networks, first build model, then by a large amount of training data adjustment parameter, these class methods need a large amount of training data, and calculated amount is large, require high to program capability, the relation between classification can not be provided, lack friendly effect of visualization, can only classification results be provided; Another kind of is exactly build tree structure, adopts hierarchy classification method, and it can fine visual classifying quality, but owing to not utilizing the relation between tree construction i.e. class completely, does not namely provide a good marking mechanism.In addition, picture feature dimension is large, and specificity and robustness are good not, cause classification results undesirable.

Ning Zhou and Jianping Fan mentions and builds Visual tree and associating dictionary learning in document " Jointly Learning Visually CorrelatedDictionaries for Large-scale Visual Recognition Applications ", by building Visual tree, similar class is got together, different node learns different dictionaries, to increase the specificity that image represents.But when classifying, they do not make full use of the relation between tree node, be the maximum classification results back-propagation of every layer of selection one, as long as have classification error so above, classification will make mistakes.In addition, the dictionary specificity that the method learning arrives neither be fine, and final accuracy rate and degree of depth convolutional neural networks fall far short.Therefore present invention utilizes the advantage that in degree of depth convolutional neural networks, feature specificity is good, and construct a good marking mechanism in conjunction with the relation between class, improve classification accuracy.

Summary of the invention

The object of the invention is to the problems such as large and classification accuracy is low as classified calculating amount for Large Graph, a kind of image classification method of the maximum confidence path of characteristic sum learnt based on the degree of depth is provided.

The present invention includes following steps:

(1) the CNN network in ILSVRC2012 image library pre-training is utilized, the 7 layer model training CNN models mentioned in ImageNet Classificationwith Deep Convoutional Neural Networks according to Hinton;

(2) to any large-scale image storehouse, image is represented by all picture feature (output of the full articulamentum of last one deck of CNN) in the CNN model extraction storehouse trained in step (1);

(3) to any one class C in CNN model extraction storehouse _i, wherein sample image quantity is N _i, l opens image characteristic of correspondence and is calculate i-th class mean vector Q _i, calculate i-th class variance

(4) calculate the distance between every two classes, form a symmetrical Distance matrix D;

(5) similar matrix A is calculated according to Distance matrix D;

(6) according to similar matrix A, iteration uses spectral clustering, builds Visual tree T;

(7) to an each bunch of training SVM classifier, all SVM classifier form the tree classificator that has structure;

(8) to any test pattern, divide from the SVM classifier that the root node of tree is corresponding successively, each SVM classifier can provide a confidence score, judge that this test pattern belongs to the probability of each child node of this node, until leaf node, by in the path between leaf node with root node confidence score corresponding to the node of process be multiplied, as the value of the confidence in path, wherein root node place probability is set to 1;

(9) path owing to will judge in step (8) is too many, in order to pick up speed, all filters once at every one deck of tree, only retains the node that confidence score comes front K.

In step (3), described mean vector Q _icomputing formula be

I-th class variance computing formula be

In step (4), the formula of the distance between every two classes of described calculating is or a rear equation is derived by previous equation.

In step (5), between the class of described similar matrix A, Similarity measures formula is selection with reference to the article Clustering by Passing Messages Between Data Points of Brendan J.Frey, also simply can be taken as characteristic dimension.

In step (6), described according to similar matrix A, iteration uses spectral clustering, the concrete grammar building Visual tree T can be: first corresponding to all categories similar matrix A, use spectral clustering, form K bunch, multiple similar class is contained in each bunch of the inside, continues each bunch of corresponding similar matrix use spectral clustering ... until meet tree depth capacity restrictive condition or bunch infima species membership restrictive condition just stop cluster; The non-leaf nodes of bunch corresponding tree, is made up of multiple target class; The leaf node of tree is target class.

In step (8), the SVM of use is one-vs-rest:

(5.1), time SVM carries out dividing, the Confidence distance d of test picture to each class can be provided, by logistic function, can by this distance map in a probable value of 0 to 1, its computing formula is

(5.2) obtain test pattern by a Bayesian network and be assigned to certain target class c _iprobability, be namely rooting node to this leaf node the score of a paths of process, computing formula is

P(c _i)＝P(c _i|parent(c _i))*p(parent(c _i))

Wherein P (c _i) be target class c _ifinal score, parent (c _i) be c _ifather node;

(5.3) in order to accelerate computing velocity, avoiding traveling through all paths, setting front K the intermediate node that every layer is all chosen maximum probability.

The advantage that the present invention utilizes the degree of depth to learn, extracts the output of last full articulamentum of CNN as characteristics of image, and builds Visual tree, the sorter that training is corresponding, gives corresponding marking mechanism.The present invention has following outstanding advantages:

1. the present invention utilizes CNN to extract characteristics of image, has good identification and robustness.

2. The present invention gives the distance computing formula of two classes, take into account each sample, and greatly optimize computational complexity by deriving.And the similarity obtained further between class, thus iteration uses spectral clustering to build Visual tree.

3. The present invention gives a mechanism of giving a mark efficiently, take full advantage of the vision relation between class, experimental result shows method used in the present invention has good effect for large-scale Images Classification, and has obvious advantage in the method for current popular.

Accompanying drawing explanation

Fig. 1 is the process flow diagram that CNN of the present invention extracts feature.

Fig. 2 is the process flow diagram that the present invention judges to test picture.

Embodiment

With reference to Fig. 1 and 2, implementation step of the present invention comprises extraction characteristics of image, builds Visual tree and trains corresponding sorter, and according to marking mechanism test picture three parts that the present invention proposes.

Step 1, trains a CNN model

Download a large image library, as ImageNet2012 Images Classification match storehouse, 7 layer models mentioned in ImageNetClassification with Deep Convoutional Neural Networks with reference to Hinton train a CNN model

Step 2, extracts feature

The CNN model trained by step 1 to image zooming-out features all in experimental data base, namely in the output of last full articulamentum of CNN as the feature of image, with calculating later.

Step 3, calculates similar matrix

(3a) mean vector of each class is calculated class variance for the l pictures characteristic of correspondence of picture i-th class.

(3b) formula is utilized calculate the distance between every two classes, calculate and allly just can construct a symmetrical distance matrix apart from rear, the value on positive diagonal line is all 0.

(3c) calculate the similarity between two classes according to the distance between two classes, computing formula is selection with reference to the article Clustering by Passing Messages Between DataPoints of Brendan J.Frey, also can be taken as characteristic dimension simply, thus structure symmetrical similar matrix A.

Step 4, structure Visual tree

(4a) by the similar matrix that step 3 obtains, use spectral clustering, similar class got together, N number of class is polymerized to K bunch, is got together for each bunch by some similar classes;

(4b) judge whether to reach the condition stopping cluster, namely whether reach the maximum height of the tree of setting, in bunch, whether the number of class is less than the minimum threshold of setting; Otherwise enter (4c);

(4c) to last time, cluster generated bunch, continue to use spectral clustering, corresponding similar matrix is the submatrix of A, is namely made up of the row and column that the class in this bunch is corresponding in A;

(4d) repeat step (4b) and (4c), complete the structure of Visual tree.

Step 5, training classifier.

For each non-leaf nodes of tree, training SVM classifier, for being divided in its child node by test pattern, and provides corresponding mark.

Step 6, classification.

(6a) to given test pattern, lose to sorter corresponding to root node in Visual tree, classify, marking, provides the highest k of a mark child node.

(6b) judge whether a current k node is leaf node, if k node is all leaf node, then stops; Otherwise enter step (6c).

(6c) to each non-leaf nodes in a new k node, with the sorter of its correspondence, test picture is given a mark, be divided into child node to go, the fractional multiplication corresponding with its father node by this mark, as the final mark of this child node, the more front k selecting mark the highest in newly-generated all nodes.

(6d) repeat step (6b) and (6c), complete classification, export k target class, and the mark of correspondence.

The present invention carries out the proof of advantage and validity by following experiment

1. experiment condition:

Use for laboratory desktop computer parameter: the Tesla C2050 GPU of 3G buffer memory, CPU are 16 Inter (R) Xeon (R) X5647, and dominant frequency is 2.93GHz, inside save as 32G, operating system is Ubuntu12.04 64 systems, and experiment porch is caffe, python2.7.

The large-scale image classification method based on CNN characteristic sum maximum confidence path that use for laboratory the present invention proposes, wherein the training method of CNN sees reference document " Krizhevsky A; Sutskever I, Hinton G E.Imagenet classificationwith deep convolutional neural networks [C] //Advances in neural informationprocessing systems.2012:1097-1105. ".

1. experimental result and interpretation of result:

Table 1 is that the present invention compares with current other six popular methods on ImageNet2010 Images Classification match storehouse.Result shows the present invention very large advantage, and wherein Top1accuracy represents and provides a classification results, correct accuracy rate of classifying, and Top5accuracy represents and provides 5 classification results, wherein has the accuracy rate that correct.

Table 1

Model	Top-1 accuracy	Top-5 accuracy
			Sparse coding [1]	52.9％	71.8％
SIFT+FV [2]	54.3％	74.3％
			JDL+AP Clustering [3]	38.9％	N/A
Fisher Vector [4]	45.7％	65.9％
			NEC [5]	52.9％	71.8％
Visual forest [6]	41.1％	N/A
			The present invention	61.2％	81.7％

List of references:

[1]Berg,A.,Deng,J.,Fei-Fei,L.:Large scale visual recognition challenge2010.www.image-net.org(2010)。

[2]Sánchez,J.,Perronnin,F.:High-dimensional signature compression forlarge-scale image classification.In:Computer Vision and Pattern Recognition(CVPR),2011IEEE Conference on,pp.1665-1672.IEEE,(2011)。

[3]Zhou,N.,Fan,J.:Jointly learning visually correlated dictionaries forlarge-scale visual recognition applications.Pattern Analysis and MachineIntelligence,IEEE Transactions on 36,715-730(2014)。

[4]Perronnin,F.,Akata,Z.,Harchaoui,Z.,Schmid,C.:Towards good practicein large-scale learning for image classification.In:Computer Vision and PatternRecognition(CVPR),2012IEEE Conference on,pp.3482-3489.IEEE,(2012)。

[5]Lin,Y.,Lv,F.,Zhu,S.,Yang,M.,Cour,T.,Yu,K.,Cao,L.,Huang,T.:Large-scale image classification:fast feature extraction and svm training.In:Computer Vision and Pattern Recognition(CVPR),2011IEEE Conference on,pp.1689-1696.IEEE,(2011)。

[6]Fan,J.,Zhang,J.,Mei,K.,Peng,J.,Gao,L.:Cost-sensitive learning ofhierarchical tree classifiers for large-scale image classification and novelcategory detection.Pattern Recognition(2014)。

The present invention mainly solves because image category is many in large-scale image classification problem, the problem that the classification accuracy that data volume causes greatly is low and computational complexity is large.Key step of the present invention is: 1) training convolutional neural networks in an enough large image library.2) the convolutional neural networks model extraction characteristics of image trained is utilized.3) mean vector of each class is calculated.4) spectral clustering is utilized to carry out iteration cluster, in order to build Visual tree to the mean vector representing each class.5) for each non-leaf nodes training svm of tree.6) to given test pattern, top-down, judge the probability of test picture to corresponding child node, the leaf node finding path probability maximum is final target class.The present invention may be used for large-scale image classification.

Claims (6)

1., based on the image classification method in the maximum confidence path of characteristic sum of degree of depth study, it is characterized in that comprising the following steps:

(1) the CNN network in ILSVRC2012 image library pre-training is utilized, the 7 layer model training CNN models mentioned in ImageNetClassification with Deep Convoutional Neural Networks according to Hinton;

(2) to any large-scale image storehouse, image is represented by all picture feature (output of the full articulamentum of last one deck of CNN) in the CNN model extraction storehouse trained in step (1);

(3) to any one class C in CNN model extraction storehouse _i, wherein sample image quantity is N _i, l opens image characteristic of correspondence and is calculate i-th class mean vector Q _i, calculate i-th class variance

(4) calculate the distance between every two classes, form a symmetrical Distance matrix D;

(5) similar matrix A is calculated according to Distance matrix D;

(6) according to similar matrix A, iteration uses spectral clustering, builds Visual tree T;

(7) to an each bunch of training SVM classifier, all SVM classifier form the tree classificator that has structure;

(8) to any test pattern, divide from the SVM classifier that the root node of tree is corresponding successively, each SVM classifier can provide a confidence score, judge that this test pattern belongs to the probability of each child node of this node, until leaf node, by in the path between leaf node with root node confidence score corresponding to the node of process be multiplied, as the value of the confidence in path, wherein root node place probability is set to 1;

(9) path owing to will judge in step (8) is too many, in order to pick up speed, all filters once at every one deck of tree, only retains the node that confidence score comes front K.

2., as claimed in claim 1 based on the image classification method in the maximum confidence path of characteristic sum of degree of depth study, it is characterized in that in step (3), described mean vector Q _icomputing formula be

I-th class variance computing formula be

3., as claimed in claim 1 based on the image classification method in the maximum confidence path of characteristic sum of degree of depth study, it is characterized in that in step (4), the formula of the distance between every two classes of described calculating is or $dis(C_i,C_j)=sqrt({\left|\right|Q_i-Q_j\left|\right|}^2+{\mathrm{\σ}}_i^2+{\mathrm{\σ}}_j^2),$ A rear equation is derived by previous equation.

4., as claimed in claim 1 based on the image classification method in the maximum confidence path of characteristic sum of degree of depth study, it is characterized in that, in step (5), between the class of described similar matrix A, Similarity measures formula is selection with reference to the article Clustering by Passing Messages Between Data Points of Brendan J.Frey, also simply can be taken as characteristic dimension.

5. as claimed in claim 1 based on the image classification method in the maximum confidence path of characteristic sum of degree of depth study, it is characterized in that in step (6), described according to similar matrix A, iteration uses spectral clustering, the concrete grammar building Visual tree T is: first corresponding to all categories similar matrix A, use spectral clustering, form K bunch, multiple similar class is contained in each bunch of the inside, continue each bunch of corresponding similar matrix use spectral clustering ... until meet tree depth capacity restrictive condition or bunch infima species membership restrictive condition just stop cluster, the non-leaf nodes of bunch corresponding tree, is made up of multiple target class, the leaf node of tree is target class.

6., as claimed in claim 1 based on the image classification method in the maximum confidence path of characteristic sum of degree of depth study, it is characterized in that in step (8), the SVM of use is one-vs-rest:

(5.1), time SVM carries out dividing, the Confidence distance d of test picture to each class can be provided, by logistic function, can by this distance map in a probable value of 0 to 1, its computing formula is

(5.2) obtain test pattern by a Bayesian network and be assigned to certain target class c _iprobability, be namely rooting node to this leaf node the score of a paths of process, computing formula is

P(c _i)＝P(c _i|parent(c _i))*p(parent(c _i))

Wherein P (c _i) be target class c _ifinal score, parent (c _i) be c _ifather node;

(5.3) in order to accelerate computing velocity, avoiding traveling through all paths, setting front K the intermediate node that every layer is all chosen maximum probability.