CN102034116B - Commodity image classifying method based on complementary features and class description - Google Patents

Abstract

The invention discloses a commodity image classifying method based on complementary features and class description, comprising the following steps of: firstly, taking the classified image as a training sample; secondly, carrying out the resolution compression on all marking image class and test pictures by using a picture batch editing tool; thirdly, extracting the complementary features of tower type gradient direction histograms and tower type key word histograms of the pictures in various marking images; fourthly, extracting the features of the tower type gradient direction histograms and the tower type key word histograms of images of commodities to be classified; fifthly, constructing a class descriptor of each marking image class; and sixthly, classifying the obtained feature vectors by using a nearest neighbor classification algorithm, calculating the distance between the images of the commodities to be classified and various marking image class descriptors and using the image class with the shortest distance as the classification result. According to the invention, two complementary features can be fully utilized and the classification result is more accurate by using the nearest neighbor classification algorithm based on the image-class distance.

Description

Commodity image classification method based on complementary characteristic and class description

Technical field

What the present invention relates to is a kind of method of commodity classification of images, specifically a kind of commodity classification of images algorithm of the distance algorithm based on complementary characteristic and improved image-class.

Background technology

With popularizing and development of internet, ecommerce has progressed into brand-new epoch, and the quantity sharp increase of e-commerce website a collection of well-known e-commerce website both at home and abroad occurred, such as Amazon, ebay, Taobao etc.E-commerce website need to be by marking to make things convenient for the user to search for to the online sales commodity.Under the present circumstances, these marks only illustrate the essential information (metamessage) of commodity, such as the title of commodity, the place of production, size, price etc., are difficult to reflect the complete characterization of commodity.As: Women's Leather Shoes are round end or tip, and T-shirt is Jewel neck or V-type neckline, and the playshoes shoestring is velcro type or thin shoelace-type etc.; These features all are the interested potential informations of user's possibility, but because lack further mark, the user can only could obtain these information by browsing the commodity picture.If the picture classification filtrator is set in the website, can make things convenient for undoubtedly the user to browse.If by manually finishing the mark of these potential interest informations, at commodity amount and kind scale very large e-commerce website all, waste time and energy very much beyond doubt.

Content-based image classification (content-based image classification) is that the visual signature according to image carries out semantic classification to image.The focus of Image Classification Studies was scene classification (scene classification) and the object classification (object classification) of natural image in recent years, the main supervised learning method that adopts is by realizing classification to low-level image feature modeling and middle semantic analysis.Test pattern database Caltech 101 and Caltech 256 commonly used reached 101 classes and 256 classes in the Research Literature at present.Different from the natural image in these storehouses, the commodity image that provides on the e-commerce website generally is more satisfactory picture, has less background interference, and target is more single; These characteristics make content-based commodity Images Classification more easily obtain desirable classification accuracy rate, for a kind of novel commodity classification method provides possibility.

Summary of the invention

The present invention is directed to the proposition of above problem, and develop a kind of commodity image classification method based on complementary characteristic and class description.The technical solution used in the present invention is as follows:

A kind of commodity image classification method based on complementary characteristic and class description is characterized in that the extraction of complementary characteristic and the distance calculating method of improved image-class, specifically describes following steps:

Step 1, get the image of classified as training sample;

Step 2, the tower-type gradient direction histogram that extracts picture in each marking image class and tower keyword histogram complementary characteristic, wherein tower progression is L (L=0,1...n);

Tower-type gradient direction histogram and the tower keyword histogram feature of step 3, extraction commodity image to be sorted, wherein tower progression is L (L=0,1...n);

Step 4, then calculate the class descriptor of each image class, namely represent tower-type gradient direction histogram and the tower keyword histogram feature of each image class, wherein tower progression is L;

Formula is tried to achieve under the class descriptor utilization of image class:

The category feature descriptor of image class is { HGC ^(l), HWC ^(l), l=0,1 ..., L-1

${\mathrm{HGC}}^{\left(l\right)}={\mathrm{HGC}}_j^{\left(l\right)}|\underset{j}{\min }d({\mathrm{HGC}}_j^{\left(l\right)},{\mathrm{HGq}}^{\left(l\right)})$

${\mathrm{HWC}}^{\left(l\right)}={\mathrm{HWC}}_j^{\left(l\right)}|\underset{j}{\min }d({\mathrm{HWC}}_j^{\left(l\right)},{\mathrm{HWq}}^{\left(l\right)})$

l＝0，1，...，L-1，j＝0，1...，N _C

Wherein: HGq ^(l), HWq ^(l)Represent respectively commodity image to be sorted the i level (i=0,1 ..., L-1) tower-type gradient histogram and tower keyword histogram, The i level of j width of cloth image in the difference presentation video class (l=0,1 ..., L-1) tower-type gradient histogram and tower keyword histogram; In addition, L is tower decomposed class, N _CSum for the picture that contains in this image class.

Distance between step 5, calculating commodity image to be sorted and the class descriptor, namely calculate respectively tower-type gradient direction Histogram map distance and tower keyword histogram distance between commodity image to be sorted and each the image class, adopt the chi-square distance calculating method, two kinds of distances calculating are carried out linear weighted function, with linear weighted function apart from the image class of minimum as classification results.

Also comprised the step of using picture batch edit tool that the resolution of all marking image classes and commodity image to be sorted is compressed before step 2 and step 4, wherein the marking image class is identical with the resolution of commodity compression of images to be sorted.

Linear weighted function coefficient in the step 5 obtains by the cross validation mode.

Owing to having adopted technique scheme, it is apparent that the commodity image classification method based on complementary characteristic and class description provided by the invention is compared its advantage with prior art; At first, aspect image characteristics extraction and description, adopt dense sample mode, formed two kinds of Multistage tower-type architectural features with complementary characteristic: tower-type gradient direction histogram and tower keyword histogram, and by the final feature representation of fusion linear feature acquisition.This feature is described the shape facility of both having considered image, considered again the local distribution information of image, the tower structure that consists of by the image space Multiresolution Decomposition and characteristic weighing merge can be more complete, Description Image characteristic information neatly, thereby improve the Images Classification performance.(2) aspect classifier design, the improvement arest neighbors sorting algorithm described based on the image category feature has been proposed, realize the commodity Images Classification by computed image to the distance of class (rather than image is to image).

Description of drawings

Fig. 1 is the process flow diagram of sorting technique of the present invention;

Fig. 2 is commodity picture library schematic diagram in the embodiment of the invention;

Fig. 3 is CSAA best result class accuracy table in the embodiment of the invention;

Fig. 4 is the classification accuracy rate figure under the different marker samples in the embodiment of the invention;

Fig. 5 is the average mark class testing time under the different marker samples in the embodiment of the invention.

Embodiment

Shown in 1, should based on the commodity image classification method of complementary characteristic and class description, specifically describe following steps:

Step 1, get the image of classified as training sample;

Step 2, the tower-type gradient direction histogram that extracts picture in each marking image class and tower keyword histogram complementary characteristic, wherein tower progression is L (L=0,1...n is natural number);

Tower-type gradient direction histogram and the tower keyword histogram feature of step 3, extraction commodity image to be sorted, wherein tower progression is L (L=0,1...n is natural number);

Step 4, then calculate the class descriptor of each image class, namely represent tower-type gradient direction histogram and the tower keyword histogram feature of each image class; Wherein tower progression is L;

Distance between step 5, calculating commodity image to be sorted and the class descriptor, namely calculate respectively tower-type gradient direction Histogram map distance and tower keyword histogram distance between commodity image to be sorted and each the image class, adopt the chi-square distance calculating method, two kinds of distances calculating are carried out linear weighted function, with linear weighted function apart from the image class of minimum as classification results.

Calculating distance in the step 5 is specially: calculating and calculating respectively the tower level is 0,1...L-1 the time commodity image gradient direction histogram to be sorted and each image class in the chi-square distance of each width of cloth picture gradient orientation histogram, with the picture gradient orientation histogram of the respective distances minimum gradient orientation histogram as the corresponding column level of this image class.Calculating respectively the tower level is 0,1...L-1 the time commodity image keyword histogram to be sorted and each image class in the histogrammic chi-square distance of each width of cloth picture keyword, with the image keyword histogram of the respective distances minimum image keyword histogram as the corresponding column level of this image class.

In order to reduce the processing time, before step 2 and step 4, also comprise the step of using picture batch edit tool that the resolution of all marking image classes and commodity image to be sorted is compressed, wherein marking image class identical with the resolution of commodity compression of images to be sorted (the present embodiment is take resolution 100 * 100 as example).

In addition, the linear weighted function coefficient in the step 5 obtains by the cross validation mode.

Concrete computation process is as follows:

Described tower-type gradient direction histogram is set up mode following (take L=3 as example): as:

(1) gradient direction with image pixel is divided into K interval, and the size in each interval is 360/K;

(2) gradient of the whole image of calculating forms the corresponding gradient orientation histogram H0 with the gradient magnitude weighting;

(3) image being carried out the space quaternary tree decomposes, soon image is divided into 4 rectangular elements into onesize (or approximate size), calculate respectively the gradient orientation histogram of the gradient magnitude weighting of each rectangular element, from left to right, connect successively from top to bottom 4 unit gradient orientation histograms, form gradient orientation histogram H1;

(4) image being carried out the secondary quaternary tree decomposes, be about to the rectangular element that image is divided into 16 onesize (or approximate size), calculate respectively the gradient orientation histogram of the gradient magnitude weighting of each unit, from left to right, connect successively from top to bottom 16 unit gradient orientation histograms, form gradient orientation histogram H2;

(5) connect successively H0, H1, H2 carries out normalized with whole histogram " energy " (L2 norm) to proper vector, forms the tower-type gradient direction histogram H of image.

For example: be made as 20 if the gradient direction quantized interval is counted K, the tower-type gradient direction histogram is sequentially connect by 3 gradient orientation histogram proper vectors and forms.Do not carry out spatial division during tower level 1=0, as 1 unit compute gradient direction histogram, its dimension is 20 with whole image; During tower level 1=1 image is carried out quad-tree partition, image is divided into 4 rectangular element compute gradient direction histograms, its dimension is 20 * 4=80; Be 16 rectangular element compute gradient direction histograms with picture breakdown during tower level 1=2, its dimension is 20 * 16=320, and the final histogram that forms is 1=0, the sequential combination of 1,2 each gradient orientation histogram, and its dimension is 20+80+320=420.Histogram is carried out normalized, can further remove the impact of illumination variation.

Described tower keyword histogram is set up mode following (take L=3 as example):

(1) adopt dense sampling (dense sample) mode, sampling interval is made as 8 pixels, and each block of pixels of 16 * 16 uses the sift descriptor to form the proper vector of 128 dimensions.

(2) image descriptor with all training images carries out forming some cluster centres behind the K mean quantization, and namely K=500 got in the vision keyword, then has 500 vision keywords, and namely word bag size is 500;

(3) calculate successively each sift descriptor of commodity image to be sorted to the Euclidean distance of each cluster centre, the cluster centre of corresponding Euclidean distance minimum is vision keyword corresponding to this sift descriptor; Add up the appearance frequency of vision keyword in the commodity image to be sorted, form vision keyword histogram H0;

(4) image being carried out the space quaternary tree decomposes, be about to the rectangular element that image is divided into 4 sizes identical (or approximate size), calculate respectively the vision keyword histogram of each unit, from left to right, connect successively from top to bottom 4 unit vision keyword histograms, form one-level vision keyword histogram H1;

(5) image being carried out the secondary quaternary tree decomposes, be about to image and be divided into 16 rectangular elements that size is identical, calculate respectively the vision keyword histogram of each unit, from left to right, connect successively from top to bottom 16 unit vision keyword histograms, form secondary vision keyword histogram H2;

(6) connect H0, H1, H2 forms a series of vision keyword histograms that represent at feature space and represents from the low resolution to the high resolving power.With whole histogram " energy " (L2 norm) proper vector is carried out normalized, form the tower vision keyword histogram H of image.

Set word bag size K=500, then the final tower keyword histogram that forms: 500+500 * 4+500 * 16=82500.

Chi-square distance calculating method between the described image histogram is as follows:

The chi-square Furthest Neighbor is adopted in the calculating of distance between the image histogram, and chi-square Furthest Neighbor computing formula is as follows:

$d(s1,s2)=1-\exp (-{\mathrm{\χ}}^2(s1,s2)),{\mathrm{\χ}}^2(s1,s2)=\underset{j}{\mathrm{\Σ}}\frac{{\left(s_1\right[j]-s_2[j\left]\right)}^2}{s_1\left[j\right]+s_2\left[j\right]}$

Wherein, the chi-square distance between two histogram s1 of d (s1, s2) expression and the s2.s ₁[j], s ₁[j] represents respectively the value of two histogram s1 and j element of s2.

Described class descriptor building method is as follows:

If HGq ^(l), HWq ^(l)Represent respectively commodity image q to be sorted the i level (i=0,1 ..., L-1) tower-type gradient histogram and tower keyword histogram,

The i level of j width of cloth image among the difference presentation video class C (l=0,1 ..., L-1) tower-type gradient histogram and tower keyword histogram, then the category feature descriptor { HGC of image class C ^(l), HWC ^(l), l=0, and 1 ..., L-1 should meet the following conditions:

${\mathrm{HGC}}^{\left(l\right)}={\mathrm{HGC}}_j^{\left(l\right)}|\underset{j}{\min }d({\mathrm{HGC}}_j^{\left(l\right)},{\mathrm{HGq}}^{\left(l\right)})$

${\mathrm{HWC}}^{\left(l\right)}={\mathrm{HWC}}_j^{\left(l\right)}|\underset{j}{\min }d({\mathrm{HWC}}_j^{\left(l\right)},{\mathrm{HWq}}^{\left(l\right)})$

l＝0，1，...，L-1，j＝0，1...，N _C

Wherein, L is tower decomposed class, N _CSum for the picture that contains among the image class C

Distance calculating method between described commodity image to be sorted and the class descriptor is as follows:

(1) histogram of different resolution has different impacts to classification performance, thus compute histograms apart from the time different weight coefficients should be set.In general, with respect to the low resolution histogram, the high resolving power histogram is larger on the impact of classification performance.The weight of tower keyword histogram and tower-type gradient histogram l level is made as

(l=0,1 ..., L-1, L are tower decomposed classes, the present invention gets L=3):

$d(\mathrm{HGC},\mathrm{HGq})=\underset{l=0}{\overset{L-1}{\mathrm{\Σ}}}\frac{1}{2^{L-1}}d({\mathrm{HGC}}_j^{\left(l\right)},{\mathrm{HGq}}^{\left(l\right)})$

$d(\mathrm{HWC},\mathrm{HWq})=\underset{l=0}{\overset{L-1}{\mathrm{\Σ}}}\frac{1}{2^{L-1}}d({\mathrm{HWC}}_j^{\left(l\right)},{\mathrm{HWq}}^{\left(l\right)})$

(2) distance calculating method between commodity image to be sorted and the class descriptor is as follows:

d(HC，Hq)＝α·d(HGC，HGq)+(1-α)d(HWC，HWq)

D (HGC wherein, HGq) and d (HWC, HWq) represent respectively with tower-type gradient direction histogram feature and the commodity image q to be sorted of tower keyword histogram feature calculating and the distance between the image class C, the distance between the commodity image q to be sorted and image class C behind the Fusion Features is carried out in d (HC, Hq) expression.The value of linear weighted function factor alpha is determined by the method for five retransposings checking.By the selection of α, obtain the character representation of tool separating capacity.

The acquisition methods of described linear weighted function factor alpha is as follows:

(1) the α initial value is made as 0;

(2) adopt five retransposing verification methods to try to achieve average classification accuracy rate;

(3)α＝α+0.01

(4) if (2) are returned in α＜=1,

If α＞1, the α that the classification accuracy rate of on average classifying is the highest finishes as final linear weighted function coefficient.

Described five retransposing verification methods are as follows:

(1) all marking images is divided into 5 parts;

(2) in turn will be wherein 4 parts do 1 part of training and do test, the record sort accuracy;

(3) record 5 subseries as a result the average of accuracy as to the estimation of the average classification accuracy rate of algorithm.

Described arest neighbors sorting algorithm is as follows:

Final classification results

By formula

$\hat{C}=\underset{C}{\min d}(\mathrm{HC},\mathrm{Hq})$ Determine

The distance between the commodity image q to be sorted and image class C behind the Fusion Features is carried out in d (HC, Hq) expression.

The present invention can take full advantage of two kinds of complementary characteristics, and uses the improvement arest neighbors sorting algorithm based on image-class distance, so that classification results is more accurate.

Below in conjunction with accompanying drawing embodiments of the invention are elaborated: the present embodiment is implemented under take technical solution of the present invention as prerequisite, provided detailed embodiment and concrete operating process, but protection scope of the present invention is not limited to following embodiment.

The image data base that adopts in the present embodiment is Tomasik B, Thiha P and Tumbull D " Tagging products using image classification " (32nd international ACM SIGIR) use 5 class image libraries ( Http:// www.sccs.swarthmore.edu/users/09/btomasil/tagging-produc ts.html).These images all are that resolution is about 280 * 280 from the commodity image of eBay and the download of Amazon.com website, and example is shown in Figure 2.This paper experiment is configuring Intel Pentium CPU 2.66GHz, and 1GBRAM carries out on the computing machine of operation Windows XP operating system and MATLAB7.1 software.

As shown in Figure 1, this example comprises following steps:

Step 1. random 5,25,50,75 width of cloth images of selecting form some marking image classes as training sample in given picture library.Other image is as commodity image to be sorted.

Step 2. use picture in batches edit tool (such as Batch Image Resizer 2.88) with the resolution compression of the picture of all marking image classes and test library for being 100 * 100.

Step 3. extract tower-type gradient direction histogram (K=80, L=3) and the tower keyword histogram feature (K=500, L=3) of picture in each marking image class.

Step 4 is extracted tower-type gradient direction histogram (K=80, L=3) and the tower keyword histogram feature (K=500, L=3) of commodity image to be sorted.

Step 5. calculate the class descriptor of each image class, namely represent tower-type gradient direction histogram and the tower keyword histogram feature of each image class.

Step 6, use the arest neighbors sorting algorithm to classify the proper vector of above acquisition, by calculating the distance between commodity image to be sorted and the class descriptor, obtain tower-type gradient direction histogram between commodity image to be sorted and each the image class and the card side's distance between the tower keyword histogram descriptor, with linear weighted function apart from the image class of minimum as classification results.The linear weighted function coefficient obtains by the cross validation mode.

The topmost index of classification of assessment performance is classification accuracy rate and classification speed.

Because every class picture number may have larger difference in the image measurement storehouse, the weight of using the computing method (correct classified image number accounts for the ratio of all images number) of overall classification accuracy rate (Overall Accuracy, OA) can cause the less classification of picture number to take is less; So adopt the class size to adjust the computing method of accuracy (Class-Size-Adjusted Accuracy, CSAA), be shown below:

$\mathrm{CSAA}=\frac{1}{C}\underset{i=1}{\overset{C}{\mathrm{\Σ}}}\frac{P_i}{N_i}$

Wherein, C presentation video classification number, P _iRepresent the number of categories that the i class is correct, N _iThe sum that represents i class image.As in the classification of short sleeved blouse and long sleeve blouse, suppose that 100 width of cloth long sleeve blouses have the classification of 90 width of cloth correct, and have the classification of 30 width of cloth correct in 50 width of cloth short sleeved blouses, then overall classification accuracy rate OA=(90+30)/(100+50)=80%; And class size adjustment classification accuracy rate CSAA=1/2 * (90/100+30/50)=75%.

Classification speed adopts the average mark class testing time to go interpretive classification speed.

Test result such as Fig. 3, Fig. 4 and shown in Figure 5, can find out from above test result:

(1) there is larger difference in different classification task classification accuracy rates.Such as the classification of long sleeves and cotta, be just to reach 90% at 5 o'clock in number of training, increase the number of training classification accuracy rate and move closer to 99%; And the classification accuracy rate of velcro and shoestring is the highest by only 70%.

(2) in general, be higher than based on the histogrammic classification accuracy rate of tower-type gradient based on the histogrammic classification accuracy rate of tower keyword.And both Fusion Features classification accuracy rate 1～3 percentage point raising has been arranged.

(3) method that proposes with respect to Tomasik B etc. all is improved to some extent based on the classification accuracy rate of two kinds of Fusion Features.Based on especially crew neck, V-type neck bring up to 70% and 74% by 66%, 67% respectively with 3 classification of blouse and 2 classification best result class accuracy of velcro and shoestring.Its reason is that (a) the present invention has adopted the complementary characteristics of image that differentiation power is more arranged, and (b) the present invention has designed more rational nearest neighbor classifier based on the image class description, better promotes performance by computed image to the distance acquisition of class.

(4) from classification speed, raising along with the marker samples number, the average mark class testing time has and approaches linear increasing more slowly, illustrate that the test duration depends primarily on commodity Characteristic of Image leaching process to be sorted, and the extraction of class descriptor and match time affect less.When every class reference numerals reaches 75, the average mark class testing time based on tower-type gradient histogram, tower keyword histogram, two kinds of Fusion Features is respectively 0.2s, 0.56s and 0.76s, can both reach the requirement of real-time, wherein at classification speed obvious advantage be arranged based on the histogrammic method of tower-type gradient.

This paper uses complementary characteristics of image and has realized the automatic classification of 2～3 class commodity images based on the improvement arest neighbors sorting algorithm of class description, and accuracy reaches 70%-99%, and can reach the requirement of real-time;

The above; only be the better embodiment of the present invention; but protection scope of the present invention is not limited to this; anyly be familiar with those skilled in the art in the technical scope that the present invention discloses; be equal to replacement or change according to technical scheme of the present invention and inventive concept thereof, all should be encompassed within protection scope of the present invention.

Claims (3)

1. the commodity image classification method based on complementary characteristic and class description is characterized in that the extraction of complementary characteristic and the distance calculating method of improved image-class, specifically describes following steps:

Step 1, get the image of classified as training sample;

Step 2, the tower-type gradient direction histogram that extracts image in each image class and tower keyword histogram feature, wherein tower decomposed class is L, L is set to 3;

Tower-type gradient direction histogram and the tower keyword histogram feature of step 3, extraction commodity image to be sorted, wherein tower decomposed class is L, L is set to 3;

Step 4, then calculate the class descriptor of each image class, namely represent tower-type gradient direction histogram and the tower keyword histogram feature of each image class;

Formula is tried to achieve under the class descriptor utilization of image class:

The class descriptor of image class is { HGC ^(l), HWC ^(l), }

${\mathrm{HGC}}^{\left(l\right)}={\mathrm{HGC}}_j^{\left(l\right)}|\underset{j}{\min }d({\mathrm{HGC}}_j^{\left(l\right)},{\mathrm{HGq}}^{\left(l\right)})$

${\mathrm{HWC}}^{\left(l\right)}={\mathrm{HWC}}_j^{\left(l\right)}|\underset{j}{\min }d({\mathrm{HWC}}_j^{\left(l\right)},{\mathrm{HWq}}^{\left(l\right)})$

l=0,1,…,L-1,j=0,1…,N _C-1

Wherein: HGq ^(l), HWq ^(l)Represent respectively commodity image to be sorted the l level (l=0,1 ..., L-1) tower-type gradient direction histogram and tower keyword histogram,

L level tower-type gradient direction histogram and the tower keyword histogram of j width of cloth image in the difference presentation video class; In addition, L is tower decomposed class, N _CSum for the picture that contains in this image class;

With

Distance is calculated the chi-square distance method that adopts in the step 5;

Distance between the class descriptor of step 5, calculating commodity image to be sorted and each image class, namely calculate respectively tower-type gradient direction histogram and tower keyword histogram distance in the class descriptor of commodity image tower-type gradient direction histogram to be sorted and tower keyword histogram and each image class, adopt the chi-square distance calculating method, chi-square Furthest Neighbor computing formula is as follows:

$d(s1,s2)=\underset{j}{\mathrm{\Σ}}\frac{{\left(s_1\right[j]-s_2[j\left]\right)}^2}{s_1\left[j\right]+s_2\left[j\right]}$

Wherein, the chi-square distance between two histogram s1 of d (s1, s2) expression and the s2, s1[j], s2[j] represent respectively the value of two histogram s1 and j element of s2; Two kinds of distances calculating are carried out linear weighted function, with linear weighted function apart from the image class of minimum as classification results.

2. sorting technique according to claim 1, it is characterized in that also comprising before step 2 the step of using image batch edit tool that the resolution of the image in all image classes and commodity image to be sorted is compressed, wherein the image class is identical with the resolution of commodity compression of images to be sorted.

3. sorting technique according to claim 1 is characterized in that the linear weighted function coefficient in the step 5 obtains by the cross validation mode.

Images