CN103544501A - Indoor and outdoor scene classification method based on Fourier transformation - Google Patents

Abstract

The invention provides an indoor and outdoor scene classification method based on Fourier transformation, and relates to a method for classifying indoor and outdoor scene images. The method aims to solve the problems that frequency domain characteristics are not taken into consideration in an existing indoor and outdoor scene classification method, and classification accuracy is poor. The indoor and outdoor scene classification method based on Fourier transformation includes the following steps of segmenting scene images, obtaining an edge orientation histogram (EOH), obtaining a color orientation histogram (COH), calculating a characteristic vectors of the images, carrying out Fourier transformation of the images, carrying out secondary segmentation of the images, and classifying indoor and outdoor scenes. According to the method, classification accuracy is improved from 91% to 92%, and the method can be used in an indoor and outdoor picture scene classification method.

Description

Indoor and outdoor scene classification method based on Fourier transform

Technical field

The present invention relates to a kind of method of the indoor and outdoor scene image of classifying.

Background technology

In image is processed, it is particularly important that indoor and outdoor scene classification all seems in a lot of fields, such as the application of content-based image recovery, digital library, autonomous robot and digital photography etc.Because the object of some indoor pictures also can be found in outdoor picture, the structure of the buildings in outdoor picture is similar to doors structure simultaneously, this has just brought very large difficulty to indoor and outdoor scene classification problem, and certain methods is by being used form, test data and the semantic knowledge of different sorters, feature, training automatically to classify.

Color and textural characteristics are the features that indoor and outdoor scene classification problem is the most often used, in order further to improve classification accuracy, some post-processing technologies have obtained concern, refusal threshold value is as a kind of post-processing technology, its use can be used for improving classification accuracy effectively, except choosing suitable feature, select a good sorter no less important.Edge and color direction histogram (Edge and Color Orientation Histogram, ECOH) by Wonjun Kim, equaled to propose for 2010, the method is passed through edge and the color direction histogram of pixel in statistical picture, thereby has effectively improved classification accuracy.

Outdoor scene contains the textures such as sky mostly compared with flat feature, the complicacy of interior space object and there is more edge and linear information, therefore texture variations is inevitable larger, be reflected to frequency domain space, the frequency change of outdoor scene will be more violent, and indoor scene can relatively flat, yet existing method is only considered the space domain characteristic such as color and texture edge, for scene classification, have equally the but not consideration of frequency field feature of vital role, classification accuracy is lower.

Summary of the invention

The object of the invention is, in order to solve existing indoor and outdoor scene classification method owing to not considering frequency field feature, to cause the problem that classification accuracy is lower, a kind of indoor and outdoor scene classification method based on Fourier transform is provided.

Indoor and outdoor scene classification method based on Fourier transform, according to following steps, carry out:

One, cutting apart of scene image: image is divided into BLK1, BLK2, BLK3, BLK4 and BLK5 five bulks;

Two, ask edge histogram EOH: to dividing each bulk obtaining in step 1, ask edge histogram EOH;

Three, ask color histogram COH: to dividing each bulk obtaining in step 1, ask color histogram COH;

Four, the eigenvector of computed image: the color histogram COH trying to achieve in the edge histogram EOH trying to achieve in step 2 and step 3 is merged, each bulk is assigned weight, obtain the eigenvector of image, i.e. ECOH method;

Five, the Fourier transform of image: each bulk is carried out respectively to Fourier transform, the piece after Fourier transform is carried out to translation transformation, low frequency part is positioned at center;

Six, again cutting apart of image: each bulk is divided into 32 fritters again, calculates amplitude the summation of each fritter, as a dimension of eigenvector, dividing mode is: BLK1 is divided into 16 of every row, totally 2 row, BLK2, BLK3, BLK4 and BLK5 are divided into 2 of every row, totally 16 row;

Seven, indoor and outdoor scene classification: on ECOH method basis, add Fourier spectrum feature, comprehensive spectrum signature and ECOH method, calculate the final vector of image, pie graph sheet scene characteristic of division space, finally in svm classifier device, carry out the classification of indoor and outdoor scene, described feature space form is as follows:

$F={\mathrm{<figure-callout\; id="1"\; label="F"\; filenames="HDA0000403006440000012.png,HDA0000403006440000013.png"\; state="\{\{state\}\}">F</figure-callout>}}_1^{\mathrm{ECOH}},F_1^O,{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA0000403006440000012.png,HDA0000403006440000013.png"\; state="\{\{state\}\}">F</figure-callout>}}_2^{\mathrm{ECOH}},F_2^O...F_5^{\mathrm{ECOH}},F_{5,}^O$

Wherein $F_i^O=F_{i1}^O,F_{i2}^O...F_{i32}^O,1\&le;i\&le;5.$

The present invention comprises following beneficial effect:

The present invention proposes a kind of spectrum information that utilizes after Fourier transform for the method for indoor and outdoor scene classification, spectrum information is combined with ECOH method, image is divided into five bulks at first, extracting section after each is cut apart color, texture and spectrum information is wherein as feature, the part of each picture segmentation has independently eigenvector, therefore by this stack features, can represent this width picture, for feature extraction phases, we have verified that frequency information can be used in classification indoor and outdoor picture, finally uses support vector machine to classify.After comparing with other sorting techniques, the present invention makes classification accuracy rate bring up to 92% from 91%.

Accompanying drawing explanation

Fig. 1 be picture cut apart at first figure;

Fig. 2 be BLK1 again cut apart figure;

Fig. 3 be BLK2, BLK3, BLK4 and BLK5 again cut apart figure;

Fig. 4 is that ECOH method is found best C and gamma figure;

Fig. 5 is that ECOH+ECFH method is found best C and gamma figure.

Embodiment

Technical solution of the present invention is not limited to following cited embodiment, also comprises the combination in any between each embodiment.

Embodiment one: the indoor and outdoor scene classification method based on Fourier transform of present embodiment, according to following steps, carry out:

One, cutting apart of scene image: image is divided into BLK1, BLK2, BLK3, BLK4 and BLK5 five bulks;

Two, ask edge histogram EOH: to dividing each bulk obtaining in step 1, ask edge histogram EOH;

Three, ask color histogram COH: to dividing each bulk obtaining in step 1, ask color histogram COH;

Four, the eigenvector of computed image: the color histogram COH trying to achieve in the edge histogram EOH trying to achieve in step 2 and step 3 is merged, each bulk is assigned weight, obtain the eigenvector of image, i.e. ECOH method;

Five, the Fourier transform of image: each bulk is carried out respectively to Fourier transform, the piece after Fourier transform is carried out to translation transformation, low frequency part is positioned at center;

Six, again cutting apart of image: each bulk is divided into 32 fritters again, calculates amplitude the summation of each fritter, as a dimension of eigenvector, dividing mode is: BLK1 is divided into 16 of every row, totally 2 row, BLK2, BLK3, BLK4 and BLK5 are divided into 2 of every row, totally 16 row;

Seven, indoor and outdoor scene classification: on ECOH method basis, add Fourier spectrum feature, comprehensive spectrum signature and ECOH method, calculate the final vector of image, pie graph sheet scene characteristic of division space, finally in svm classifier device, carry out the classification of indoor and outdoor scene, described feature space form is as follows:

$F={\mathrm{<figure-callout\; id="1"\; label="F"\; filenames="HDA0000403006440000012.png,HDA0000403006440000013.png"\; state="\{\{state\}\}">F</figure-callout>}}_1^{\mathrm{ECOH}},F_1^O,{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA0000403006440000012.png,HDA0000403006440000013.png"\; state="\{\{state\}\}">F</figure-callout>}}_2^{\mathrm{ECOH}}{,F}_2^O...F_5^{\mathrm{ECOH}},F_5^O,$

Wherein $F_i^O=F_{i1}^O,F_{i2}^O...F_{i32}^O,1\&le;i\&le;5.$

Embodiment two: present embodiment is different from embodiment one: the image dividing mode described in step 1 is: BLK1 is topmost one of image, wide is the wide of entire image, height is 1/8 of image, BLK2, BLK3, BLK4 and BLK5 be evenly distributed in BLK1 below, wide 1/4 of the figure image width that is respectively of BLK2, BLK3, BLK4 and BLK5, height is respectively 7/8 of figure image height.Other is identical with embodiment one.

Embodiment three: present embodiment is different from embodiment one or two: the concrete steps of asking edge histogram EOH described in step 2 are: according to formula $A=(x,y)=\sqrt{P_x{(x,y)}^2+P_y{(x,y)}^2}$ And $\mathrm{\&theta;}(x,y)={\tan }^{-1}\frac{P_y(x,y)}{P_x(x,y)}$ Obtain texture amplitude A (x, y) and argument θ (x, y), the wherein P of every middle edge pixel _x(x, y) and P _y(x, y) be respectively pixel (x, y) level and perpendicular direction vector, the direction vector of pixel is quantized within the scope of 0 °～180 °, i.e. the argument of pixel and 180 deliverys, according to the argument of texture edge pixel point, by 180 degree eight equal parts, edge pixel point is divided in 8 regions, calculate be divided into pixel amplitude each region in, formula is as follows:

$E_{i,m}=\underset{\mathrm{\&theta;}(x,y)\&Element;m}{\underset{(x,y)\&Element;\mathrm{BLK}}{\mathrm{\&Sigma;}}}A(x,y),1\&le;i\&le;\mathrm{5,1}\&le;m\&le;8,$ E wherein _i,mfor belong to i piece in m region amplitude and.Other is identical with embodiment one or two.

Embodiment four: present embodiment is different from one of embodiment one to three: the concrete steps of asking color histogram COH described in step 3 are: image is transformed into HSV color space by rgb color space, the span of H component is 0 °～360 °, by 360 degree 8 deciles, according to the H component value of pixel using different pixels be divided in 8 regions and ask pixel S component in each region and as a dimension, formula is as follows

wherein, s (x, y) and h (x, y) represent pixel point (x, y) is located respectively degree of saturation and form and aspect.Other is identical with one of embodiment one to three.

Embodiment five: present embodiment is different from one of embodiment one to four: the feature space form of the characteristics of image vector described in step 4 is as follows:

$F={\mathrm{\&omega;}}_1{\mathrm{<figure-callout\; id="1"\; label="F"\; filenames="HDA0000403006440000012.png,HDA0000403006440000013.png"\; state="\{\{state\}\}">F</figure-callout>}}_1^{\mathrm{ECOH}},{\mathrm{\&omega;}}_2{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA0000403006440000012.png,HDA0000403006440000013.png"\; state="\{\{state\}\}">F</figure-callout>}}_2^{\mathrm{ECOH}},...{\mathrm{\&omega;}}_5{F}_5^{\mathrm{ECOH}},$ Wherein $F_i^{\mathrm{ECOH}}=F_{i1}^E{F}_{i1}^C,F_{i2}^E{F}_{i2}^C,...F_{i8}^E{F}_{i8}^C,1\&le;i\&le;5.$ Other is identical with one of embodiment one to four.

By following examples, verify beneficial effect of the present invention:

Embodiment mono-: the present embodiment view data is from the Corel image database of Washington, DC university computer scientific and engineering institute, and wherein training sample picture has 2638 width, indoor picture 1494 width, outdoor picture 1144 width.Test sample book has 2139 width pictures, indoor picture 1583 width, outdoor picture 556 width.

The indoor and outdoor scene classification method based on Fourier transform of the present embodiment, carry out according to the following steps:

One, picture is divided into five bulks, concrete dividing mode is as Fig. 1, and wherein BLK1 be one of the top, and wide is the wide of picture in its entirety, and height is 1/8 of picture, all the other four wide be picture wide 1/4, height be picture high 7/8.

Two, according to formula $A=(x,y)=\sqrt{P_x{(x,y)}^2+P_y{(x,y)}^2}$ And $\mathrm{\&theta;}(x,y)={\tan }^{-1}\frac{P_y(x,y)}{P_x(x,y)}$ Obtain texture amplitude A (x, y) and the argument θ (x, y) of every middle edge pixel, P _x(x, y) and P _y(x, y) be respectively pixel (x, y) level and perpendicular direction vector, the direction vector of pixel is quantized in 0 to 180 scope, be argument and 180 deliverys of pixel, according to the argument of texture edge pixel point, by 180 degree eight equal parts, edge pixel point is divided in 8 regions, and calculate be divided into pixel amplitude each region in.Thereby obtain EOH, formula is as follows:

e wherein _i,mfor belonging in m region, the amplitude of i piece and.

Three, image is transformed into HSV color space by rgb color space, because the span of H component is 0～360, and the degree of saturation of S representation in components color, therefore similar to the 1st step (asking EOH process), by 360 degree 8 deciles, according to the H component value of pixel using different pixels be divided in 8 bin and ask pixel S component in each bin and as a dimension, calculate the H value that is divided in every part with.Formula is as follows:

wherein, s (x, y) and h (x, y) represent pixel point (x, y) is located respectively degree of saturation and form and aspect.

Four, by EOH and COH, merged, and different bulks is distributed to different weights, obtain the final eigenvector of every width picture, form is as follows:

$F={\mathrm{\&omega;}}_1{\mathrm{<figure-callout\; id="1"\; label="F"\; filenames="HDA0000403006440000012.png,HDA0000403006440000013.png"\; state="\{\{state\}\}">F</figure-callout>}}_1^{\mathrm{ECOH}},{\mathrm{\&omega;}}_2{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA0000403006440000012.png,HDA0000403006440000013.png"\; state="\{\{state\}\}">F</figure-callout>}}_2^{\mathrm{ECOH}},...{\mathrm{\&omega;}}_5{F}_5^{\mathrm{ECOH}}$

Wherein

can be calculated 80 (8bin*5 piece (EOH)+8bin*5(COH) piece) n dimensional vector n.

Five, each bulk (i=1～5) is carried out respectively to Fourier transform, the piece after Fourier transform is carried out to translation transformation, make low frequency part be positioned at center.The discrete Fourier transformation (FT) of image can be represented by following formula:

wherein, its spectral amplitude A (f)=| I (f) | represent the size of Fourier transform

Six, each bulk is divided into again to 32 fritters, dividing mode is: BLK1 is divided into 16 of every row, totally 2 row, as Fig. 2, BLK2, BLK3, BLK4 and BLK5 are divided into 2 of every row, totally 16 row, as Fig. 3, calculate amplitude the summation of each fritter, as a dimension of eigenvector;

Seven, respectively all pixels in every fritter are calculated to its amplitude summation, a dimension as eigenvector, the spectrum signature of image is extracted, joins in original ECOH method, therefore in every width picture, can extract 240 (ECOH(80)+32*5 pieces) dimensional feature vector.

Use MATLAB software and classify with libsvm tool box.In order to obtain result more accurately, before setting up model, SVM carried out parameter optimization.SVM model has two very important parameters C and gamma.Wherein C is penalty coefficient, i.e. the tolerance to error.C is higher, illustrates more to can't stand to occur error.C is excessive or too small, and generalization ability variation gamma is after selecting RBF function as kernel, the parameter that this function carries.Impliedly determined that data-mapping arrives the distribution after new feature space, the number impact training of d support vector and the speed of prediction.Therefore reasonably choose C and gamma value is extremely important to classifying quality.First the feature space after training is divided into 5 parts.By the mode of cross validation, respectively using 4 parts in 5 parts as training sample, a part of as test sample book in addition, thus find out optimum C and gamma value, as Fig. 4 and Fig. 5.To different c and gamma value, obtain respectively different prediction classification accuracies, according to predictablity rate, finally determine that C is that 4, gamma value is 1, different C, the predictablity rate that gamma value obtains is as following table:

The optimizing of table 2 this paper method

Optimizing is carried out on Python.Respectively ECOH method and method in this paper have been carried out to experimental verification, the result obtaining is as follows:

Table 3 method contrast accuracy rate

From result, the method for carrying out indoor and outdoor scene classification by the information in frequency domain space is feasible and is effective.The present embodiment is combined spectrum information with ECOH method, utilize support vector machine to classify.Experiment warp and the comparison of ECOH method, accuracy rate has improved 1%.Higher accuracy can be applied in the picture scene classification method of indoor and outdoor this paper method.Work from now on will be considered the spectrum signature (as wavelet transformation) of other conversion, and classification accuracy rate is further improved.

Claims (5)

1. the indoor and outdoor scene classification method based on Fourier transform, is characterized in that described method carries out according to following steps:

One, cutting apart of scene image: image is divided into BLK1, BLK2, BLK3, BLK4 and BLK5 five bulks;

Two, ask edge histogram EOH: to dividing each bulk obtaining in step 1, ask edge histogram EOH;

Three, ask color histogram COH: to dividing each bulk obtaining in step 1, ask color histogram COH;

Four, the eigenvector of computed image: the color histogram COH trying to achieve in the edge histogram EOH trying to achieve in step 2 and step 3 is merged, each bulk is assigned weight, obtain the eigenvector of image, i.e. ECOH method;

Five, the Fourier transform of image: each bulk is carried out respectively to Fourier transform, the piece after Fourier transform is carried out to translation transformation, low frequency part is positioned at center;

Six, again cutting apart of image: each bulk is divided into 32 fritters again, calculates amplitude the summation of each fritter, as a dimension of eigenvector, dividing mode is: BLK1 is divided into 16 of every row, totally 2 row, BLK2, BLK3, BLK4 and BLK5 are divided into 2 of every row, totally 16 row;

Seven, indoor and outdoor scene classification: on ECOH method basis, add Fourier spectrum feature, comprehensive spectrum signature and ECOH method, calculate the final vector of image, pie graph sheet scene characteristic of division space, finally in svm classifier device, carry out the classification of indoor and outdoor scene, described feature space form is as follows:

$F=F_1^{\mathrm{ECOH}},F_1^O,F_2^{\mathrm{ECOH}},F_2^O...F_5^{\mathrm{ECOH}},F_{5,}^O$

Wherein $F_i^O=F_{i1}^O,F_{i2}^O...F_{i32}^O,1\&le;i\&le;5.$

2. according to the indoor and outdoor scene classification method based on Fourier transform described in claim 1, it is characterized in that the image dividing mode described in step 1 is: BLK1 is one of image topmost, wide is the wide of entire image, height is 1/8 of image, BLK2, BLK3, BLK4 and BLK5 be evenly distributed in BLK1 below, wide 1/4 of the figure image width that is respectively of BLK2, BLK3, BLK4 and BLK5, height is respectively 7/8 of figure image height.

3. according to the indoor and outdoor scene classification method based on Fourier transform described in claim 2, it is characterized in that the concrete steps of asking edge histogram EOH described in step 2 are: according to formula

and

obtain texture amplitude A (x, y) and argument θ (x, y), the wherein P of every middle edge pixel _x(x, y) and P _y(x, y) be respectively pixel (x, y) level and perpendicular direction vector, the direction vector of pixel is quantized within the scope of 0 °～180 °, according to the argument of texture edge pixel point, by 180 degree eight equal parts, edge pixel point is divided in 8 regions, calculating be divided into pixel amplitude in each region and, formula is as follows:

$E_{i,m}=\underset{\mathrm{\&theta;}(x,y)\&Element;m}{\underset{(x,y)\&Element;\mathrm{BLK}}{\mathrm{\&Sigma;}}}A(x,y),1\&le;i\&le;\mathrm{5,1}\&le;m\&le;8,$

E wherein _i,mfor belong to i piece in m region amplitude and.

4. according to the indoor and outdoor scene classification method based on Fourier transform described in claim 3, it is characterized in that the concrete steps of asking color histogram COH described in step 3 are: image is transformed into HSV color space by rgb color space, the span of H component is 0 °～360 °, by 360 degree 8 deciles, according to the H component value of pixel using different pixels be divided in 8 regions and ask pixel S component in each region and as a dimension, formula is as follows

right and form and aspect.

5. according to the indoor and outdoor scene classification method based on Fourier transform described in claim 4, it is characterized in that the feature space form of the characteristics of image vector described in step 4 is as follows:

$F={\mathrm{\&omega;}}_1{F}_1^{\mathrm{ECOH}},{\mathrm{\&omega;}}_2{F}_2^{\mathrm{ECOH}},...,{\mathrm{\&omega;}}_5{F}_5^{\mathrm{ECOH}},$

Wherein $F_i^{\mathrm{ECOH}}=F_{i1}^E{F}_{i1}^C{,F}_{i2}^E{F}_{i2}^C...F_{i8}^E{F}_{i8}^C,1\&le;i\&le;5.$

Images