CN103605984A - Supergraph learning-based indoor scene classification method - Google Patents

Abstract

The invention, which relates to the indoor scene classification field, provides a supergraph learning-based indoor scene classification method. The method comprises the following steps that: a target is extracted from an image by using nearly a hundred of target detectors and a super descriptor formed by the formed target descriptor is used as a feature descriptor of the image; a supergraph of the image descriptor is constructed by using a K neighbor method and a Laplacian matrix is calculated, thereby constructing a semi-supervised learning frame; a linear regression model is constructed and is added into the semi-supervised learning frame; according to the constructed semi-supervised learning frame, marking is carried out on the part of image descriptor by combining the extracted image feature descriptor, so that the semi-supervised learning frame can predetermine a label of an unmarked image automatically and iteratively and thus the image classification is completed; and meanwhile, the linear regression model is initialized during the automatic iteration process; and according to the linear regression model, image classification is carried out on data that are added newly directly by combining the extracted image feature descriptor, so that there is no need to construct a supergraph again.

Description

Indoor scene sorting technique based on hypergraph study

Technical field

The present invention relates to indoor scene classification, especially relate to a kind of indoor scene sorting technique based on hypergraph study.

Background technology

At present, the general feature descriptor that adopts low level of indoor scene classification, mainly comprises the information such as color, texture, shape.The feature descriptor of these low levels has good effect to outdoor scene classification, yet due to the kind of object of indoor scene complexity and overlapping, thereby performance is general on indoor scene classifying quality.Development along with correlation technique, there are some improved characteristics of image descriptors to be introduced into for improving the classifying quality of image, as pyramid matching attribute ([1] S.Lazebnik, C.Schmid, and J.Ponce, " Beyond bags of features:Spatial pyramid matching for recognizing natural scene categories, " in Proc.IEEE Int.Conf.Computer Vision and Pattern Recognition, 2006, vol.2, pp.2169 – 2178), son ([2] the C.Siagian and L.Itti of global description, " Rapid biologically-inspired scene classification using features shared with visual attention, " IEEE Trans.Pattern Anal.Mach.Intell., vol.29, no.2, pp.300 – 312, Feb.2007) etc., yet these improved characteristics of image are described owing to not solving the key problem of indoor scene image, can not improve significantly the classifying quality of indoor scene.Adopt the high-level feature descriptor that comprises image, semantic, owing to having preserved a large amount of semanteme of image, can identify multiple object in indoor scene, to improving indoor scene Images Classification effect important role.

With in high-level image descriptor, early stage having researched and proposed adopts a series of image, semantic attribute to carry out Description Image information, and the method for these Description Images obtains good effect in Image Acquisition and Images Classification field.Stanford University laboratory also proposes new super descriptor ([3] L.Li that is, H.Su, E.Xing and F.Li, " Object Bank:A High-Level Image Representation for Scene Classification and Semantic Feature Sparsification; " Proceedings of the Neural Information Processing Systems (NIPS), 2010) carry out Description Image, this image descriptor has in description and on the image of the class of complex object, especially off-the-air picture, has good description effect.Yet these Images Classifications still adopt conventional full measure of supervision to classify, can not consider the global property information of all data and the relation between local data message, so performance is very general in Images Classification effect.

Summary of the invention

The object of the present invention is to provide a kind of indoor scene sorting technique based on hypergraph study.

The present invention includes following steps:

(1) use more or less a hundred target detection from image, to extract target, the more super descriptor forming according to the goal descriptor forming, as the feature descriptor of image;

(2) the image descriptor structure hypergraph of use k nearest neighbor method to all generations, and the hypergraph based on generating calculates its Laplacian Matrix, and then builds semi-supervised learning framework;

(3) build a linear regression model (LRM), and this linear regression model (LRM) is joined in semi-supervised learning framework;

(4) according to semi-supervised learning framework constructed in step (3), and the feature descriptor of the image that extracts of integrating step (1), parts of images descriptor is marked, make this semi-supervised learning frame can dope to automatic Iterative the label that does not mark image, thereby complete Images Classification, meanwhile, the linear regression model (LRM) in step (3) is initialised in automatic Iterative process;

(5) according to the linear regression model (LRM) in step (3), and the feature descriptor of the image that extracts of integrating step (1), can directly carry out Images Classification to the data that newly add, and need not again build hypergraph.

In step (2), the concrete grammar of described structure semi-supervised learning framework can be:

First calculate the Euclidean distance between any two of feature descriptor of the image of extraction, and obtain correlation matrix H with this:

$H(\mathrm{\υ},e)=\left\{\begin{array}{cc}1,& \mathrm{if\υ}\∈e\\ 0,& \mathrm{if\υ}\∉e\end{array}\right.$

Wherein υ represents the node of hypergraph, and e represents the limit of hypergraph;

And then can calculate the number of degrees δ (e) on the weight w (e) on every limit in hypergraph, the number of degrees d (υ) of each node and every super limit, can use w (e), d (υ) and δ (e) to construct its relevant diagonal matrix W, D as diagonal element simultaneously _υand D _e, according to this three diagonal matrix and correlation matrix, can calculate intermediate result Θ and be:

Applying unit matrix I deducts Θ and can obtain:

L＝I-Θ

Result of calculation L represents the Laplacian Matrix of this hypergraph; Based on this Laplacian Matrix, can construct the regularization term of semi-supervised learning framework:

Ω(f)=f ^TLf

Wherein f represents to need the label vector of predicted picture, f ^tthe transposed vector that represents vector f; And then construct semi-supervised framework, its formula is as follows:

$\arg \underset{F}{\min \mathrm{tr}}{F}^T\mathrm{LF}+\mathrm{\λtr}\left[{(F-Y)}^T(F-Y)\right]$

Wherein Y represents the matrix that image is marked, and tr represents the mark of compute matrix, and lambda parameter is a non-negative number, is controlling the balance between model complexity and experience loss function; By calculating this formula, can obtain the prediction label F of total data.

In step (3), described linear regression model (LRM), its effect is the data to newly adding, and can use this linear regression model (LRM) directly to carry out Images Classification, and need not again build hypergraph; Linear regression model (LRM) formula is as follows:

g(x)=Q ^Tx+θ

The once parameter that wherein Q is linear regression model (LRM), θ is constant term parameter; This linear regression model (LRM) is embedded in semi-supervised learning framework, and new framework is:

$\arg \underset{F,Q}{\min }\mathrm{tr}{F}^T\mathrm{LF}+\mathrm{\λtr}\left[{(F-Y)}^T(F-Y)\right]$

$+\mathrm{\α}{\left|\right|\mathrm{XQ}-F\left|\right|}_F^2+\mathrm{\γ}{\left|\right|Q\left|\right|}_F^2$

Wherein, X represents the feature descriptor of each image, and α and γ, as non-negative regular parameter, are controlling the complexity of model and the balance between experience loss function;

According to the protruding attribute of above-mentioned formula, can calculate respectively the partial derivative of parameters and try to achieve the optimum solution of F, first with J, represent this semi-supervised learning framework, the partial derivative of establishing F and Q equals 0 and obtains:

$\frac{\∂J}{\∂F}=2\mathrm{FL}+2\mathrm{\λ}(F-Y)+2\mathrm{\α}(F-\mathrm{XQ})$

$\⇒[L+(\mathrm{\λ}+\mathrm{\α})I]F=Y+\mathrm{\αXQ}$

$\frac{\∂J}{\∂Q}=2\mathrm{\α}{X}^T(\mathrm{XQ}-F)+2\mathrm{\γQ}$

$\⇒(\mathrm{\α}{X}^{T}X+\mathrm{\γI})Q=\mathrm{\α}{X}^{T}F$

The Q that second equation tried to achieve is updated in the first equation, can be in the hope of the result of F:

F=(K-αXM) ^-1Y

Wherein, intermediate result K represents L+ (λ+α) I, and intermediate result M represents (α X ^tx+ γ I) ^-1α X ^t, now by trying to achieve F substitution and asking, in the local derviation formula equation of Q, can obtain Q and be:

Q=(αX ^TX+γI) ^-1αX ^TF=MF

Obtain the parameter that Q is linear regression model (LRM), when having new data to enter, new data need not be built to hypergraph, can be directly according to formula g (x)=Q ^tx+ θ tries to achieve the label information of new data.

The present invention uses raw image data to build a hypergraph, and with semi-supervised learning framework, predict the label that does not mark image, because hypergraph itself has been preserved the information abundanter than common figure, and semi-supervised learning framework has not only been considered the attribute information of global data, also considered the local message between labeled data and unlabeled data, thereby the present invention is obtaining good effect aspect indoor scene classification simultaneously.

The beneficial effect that the present invention has is: with the image descriptor that comprises semantic information and semi-supervised learning framework, indoor scene is classified, the precision of indoor scene classification can be effectively provided.The linear regression model (LRM) simultaneously training, can accelerate the prediction of new data label.The present invention selects and Indoor Video field provides new technical method for robot path, has effectively improved the efficiency of using indoor scene art.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the embodiment of the present invention.

Fig. 2 is that the classifying quality of the present invention and other sorting techniques compares schematic diagram.In Fig. 2, the mark ratio (%) that horizontal ordinate is training data, ordinate is classification accuracy (%); Curve a represents hypergraph learning method of the present invention, and curve b represents common drawing method, and curve c represents k near neighbor method, and curve d represents Laplce's support vector machine, and curve e represents progressive direct-push support vector machine, and curve f represents common support vector machine.

Fig. 3 is the linear regression model (LRM) predicted picture label result schematic diagram that the present invention uses.In Fig. 3, the mark ratio (%) that horizontal ordinate is training data, ordinate is classification accuracy (%); Curve a represents the parameter Q that 10% training data generates, curve b represents the parameter Q that 20% training data generates, curve c represents the parameter Q that 30% training data generates, and curve d represents the parameter Q that 40% training data generates, and curve e represents the parameter Q that 50% training data generates.

Embodiment

The indoor scene sorting technique based on hypergraph study that the present invention proposes, according to Fig. 1, introduce concrete technical scheme of the present invention and implementation step:

Step 1: use more or less a hundred target detection from image, to extract target, the more super descriptor forming according to the goal descriptor forming, as the feature descriptor of image;

Step 2: use k nearest neighbor method to build hypergraph to the image descriptor of all generations, and the hypergraph based on generating calculates its Laplacian Matrix, and then construct semi-supervised learning framework;

Step 3: build a linear regression model (LRM), and this linear regression model (LRM) is joined in semi-supervised learning framework;

Step 4: according to semi-supervised learning framework constructed in step 3, and the feature descriptor of the image that extracts of integrating step one, parts of images descriptor is marked, make this semi-supervised learning frame can dope to automatic Iterative the label that does not mark image, thereby complete Images Classification.Meanwhile, the model of the linear regression in step 3 is initialised in automatic Iterative process;

Step 5: according to the model of the linear regression in step 3, and the feature descriptor of the image that extracts of integrating step one, can directly carry out Images Classification to the data that newly add, and need not again build hypergraph.

About the concrete grammar of the structure semi-supervised learning framework mentioned in step 2, first according to the feature descriptor of the image extracting, build hypergraph, and calculate its correlation matrix H:

$H(\mathrm{\υ},e)=\left\{\begin{array}{cc}1,& \mathrm{if\υ}\∈e\\ 0,& \mathrm{if\υ}\∉e\end{array}\right.$

Wherein υ represents the node of hypergraph, and e represents the limit of hypergraph.And then can calculate the weight w (e) on every limit in hypergraph, the number of degrees d (υ) of each node and the number of degrees δ (e) on every super limit, can use w (e), d (υ) and δ (e) construct its relevant diagonal matrix W, D as diagonal element simultaneously _υand D _e, according to this three diagonal matrix and correlation matrix, can calculate intermediate result Θ and be:

Applying unit matrix I deducts Θ and can obtain:

L＝I-Θ

Result of calculation L represents the Laplacian Matrix of this hypergraph.Based on this Laplacian Matrix, can construct the regularization term of semi-supervised learning framework:

Ω(f)=f ^TLf

Wherein f represents to need the label vector of predicted picture, f ^tthe transposed vector that represents vector f.And then construct semi-supervised framework, its formula is as follows:

$\arg \underset{F}{\min }\mathrm{tr}{F}^T\mathrm{LF}+\mathrm{\λtr}\left[{(F-Y)}^T(F-Y)\right]$

Wherein Y represents the matrix that image is marked, and tr represents the mark of compute matrix, and lambda parameter is a non-negative number, is controlling the balance between model complexity and experience loss function.By calculating this formula, can obtain the prediction label F of total data.

The model of the linear regression of mentioning in step 3, its effect is the data to newly adding, and can use this linear regression model (LRM) directly to carry out Images Classification, and need not again build hypergraph.The model formation of this linear regression is as follows:

g(x)=Q ^Tx+θ

The once parameter that wherein Q is linear regression model (LRM), θ is constant term parameter.This linear model is embedded in semi-supervised learning framework, and new framework is:

$\arg \underset{F,Q}{\min }\mathrm{tr}{F}^T\mathrm{LF}+\mathrm{\λtr}\left[{(F-Y)}^T(F-Y)\right]$

$+\mathrm{\α}{\left|\right|\mathrm{XQ}-F\left|\right|}_F^2+\mathrm{\γ}{\left|\right|Q\left|\right|}_F^2$

Wherein, X represents the feature descriptor of each image, and α and γ are controlling the complexity of model and the balance between experience loss function as non-negative regular parameter.

According to the protruding attribute of above-mentioned formula, can calculate respectively the partial derivative of parameters and try to achieve the optimum solution of F, first with J, represent this semi-supervised learning framework, the partial derivative of establishing F and Q equals 0 and obtains:

$\frac{\∂J}{\∂F}=2\mathrm{FL}+2\mathrm{\λ}(F-Y)+2\mathrm{\α}(F-\mathrm{XQ})$

$\⇒[L+(\mathrm{\λ}+\mathrm{\α})I]F=Y+\mathrm{\αXQ}$

$\frac{\∂J}{\∂Q}=2\mathrm{\α}{X}^T(\mathrm{XQ}-F)+2\mathrm{\γQ}$

$\⇒(\mathrm{\α}{X}^{T}X+\mathrm{\γI})Q=\mathrm{\α}{X}^{T}F$

The Q that second equation tried to achieve is updated in the first equation, can be in the hope of the result of F:

F=(K-αXM) ^-1Y

Wherein, intermediate result K represents L+ (λ+α) I, and intermediate result M represents (α X ^tx+ γ I) ^-1α X ^t, now by trying to achieve F substitution and asking, in the local derviation formula equation of Q, can obtain Q and be:

Q=(αX ^TX+γI) ^-1αX ^TF=MF

Obtain the parameter that Q is linear regression model (LRM), when having new data to enter, new data need not be built to hypergraph, can be directly according to formula g (x)=Q ^tx+ θ tries to achieve the label information of new data.

Claims (3)

1. the indoor scene sorting technique of learning based on hypergraph, is characterized in that comprising the following steps:

(1) use more or less a hundred target detection from image, to extract target, the more super descriptor forming according to the goal descriptor forming, as the feature descriptor of image;

(2) the image descriptor structure hypergraph of use k nearest neighbor method to all generations, and the hypergraph based on generating calculates its Laplacian Matrix, and then builds semi-supervised learning framework;

(3) build a linear regression model (LRM), and this linear regression model (LRM) is joined in semi-supervised learning framework;

(4) according to semi-supervised learning framework constructed in step (3), and the feature descriptor of the image that extracts of integrating step (1), parts of images descriptor is marked, make this semi-supervised learning frame can dope to automatic Iterative the label that does not mark image, thereby complete Images Classification, meanwhile, the linear regression model (LRM) in step (3) is initialised in automatic Iterative process;

(5) according to the linear regression model (LRM) in step (3), and the feature descriptor of the image that extracts of integrating step (1), can directly carry out Images Classification to the data that newly add, and need not again build hypergraph.

2. the indoor scene sorting technique of learning based on hypergraph as claimed in claim 1, is characterized in that, in step (2), the concrete grammar of described structure semi-supervised learning framework is:

First calculate the Euclidean distance between any two of feature descriptor of the image of extraction, and obtain correlation matrix H with this:

$H(\mathrm{\υ},e)=\left\{\begin{array}{cc}1,& \mathrm{if\υ}\∈e\\ 0,& \mathrm{if\υ}\∉e\end{array}\right.$

Wherein υ represents the node of hypergraph, and e represents the limit of hypergraph;

And then can calculate the number of degrees δ (e) on the weight w (e) on every limit in hypergraph, the number of degrees d (υ) of each node and every super limit, can use w (e), d (υ) and δ (e) to construct its relevant diagonal matrix W, D as diagonal element simultaneously _υand D _e, according to this three diagonal matrix and correlation matrix, can calculate intermediate result Θ and be:

Applying unit matrix I deducts Θ and can obtain:

L＝I-Θ

Result of calculation L represents the Laplacian Matrix of this hypergraph; Based on this Laplacian Matrix, can construct the regularization term of semi-supervised learning framework:

Ω(f)=f ^TLf

Wherein f represents to need the label vector of predicted picture, f ^tthe transposed vector that represents vector f; And then construct semi-supervised framework, its formula is as follows:

$\arg \underset{F}{\min }\mathrm{tr}{F}^T\mathrm{LF}+\mathrm{\λtr}\left[{(F-Y)}^T(F-Y)\right]$

Wherein Y represents the matrix that image is marked, and tr represents the mark of compute matrix, and lambda parameter is a non-negative number, is controlling the balance between model complexity and experience loss function; By calculating this formula, can obtain the prediction label F of total data.

3. the indoor scene sorting technique of learning based on hypergraph as claimed in claim 1, it is characterized in that in step (3), described linear regression model (LRM), its effect is the data to newly adding, can use this linear regression model (LRM) directly to carry out Images Classification, and need not again build hypergraph; Linear regression model (LRM) formula is as follows:

g(x)=Q ^Tx+θ

The once parameter that wherein Q is linear regression model (LRM), θ is constant term parameter; This linear regression model (LRM) is embedded in semi-supervised learning framework, and new framework is:

$\arg \underset{F,Q}{\min }\mathrm{tr}{F}^T\mathrm{LF}+\mathrm{\λtr}\left[{(F-Y)}^T(F-Y)\right]$

$+\mathrm{\α}{\left|\right|\mathrm{XQ}-F\left|\right|}_F^2+\mathrm{\γ}{\left|\right|Q\left|\right|}_F^2$

Wherein, X represents the feature descriptor of each image, and α and γ, as non-negative regular parameter, are controlling the complexity of model and the balance between experience loss function;

According to the protruding attribute of above-mentioned formula, can calculate respectively the partial derivative of parameters and try to achieve the optimum solution of F, first with J, represent this semi-supervised learning framework, the partial derivative of establishing F and Q equals 0 and obtains:

$\frac{\∂J}{\∂F}=2\mathrm{FL}+2\mathrm{\λ}(F-Y)+2\mathrm{\α}(F-\mathrm{XQ})$

$\⇒[L+(\mathrm{\λ}+\mathrm{\α})I]F=Y+\mathrm{\αXQ}$

$\frac{\∂J}{\∂Q}=2\mathrm{\α}{X}^T(\mathrm{XQ}-F)+2\mathrm{\γQ}$

$\⇒(\mathrm{\α}{X}^{T}X+\mathrm{\γI})Q=\mathrm{\α}{X}^{T}F$

The Q that second equation tried to achieve is updated in the first equation, can be in the hope of the result of F:

F=(K-αXM) ^-1Y

Wherein, intermediate result K represents L+ (λ+α) I, and intermediate result M represents (α X ^tx+ γ I) ^-1α X ^t, now by trying to achieve F substitution and asking, in the local derviation formula equation of Q, can obtain Q and be:

Q=(αX ^TX+γI) ^-1αX ^TF=MF

Obtain the parameter that Q is linear regression model (LRM), when having new data to enter, new data need not be built to hypergraph, can be directly according to formula g (x)=Q ^tx+ θ tries to achieve the label information of new data.

Images