CN104036296A - Method and device for representing and processing image - Google Patents

Abstract

The invention is suitable for the image processing field, and provides a method and a device for representing and processing an image. The method for representing and processing the image includes: extracting at least two types of local features from the image to be represented; building a mixed independent information decomposing model in united mode according to the different types of the local features extracted from the image to be represented; encoding shared information and independent information of the different types of the local features according to the mixed independent information decomposing model, summarizing codes of the local features, and obtaining multi-perspective super vector representation of the image. The obtained multi-perspective super vector representation can be used for classifier training and image retrieval. Due to the fact that the multi-perspective super vector representation is used to encode the shared information and the independent information of the different types of the local features, the method and the device for representing and processing the image can not cause redundancy on the premise of guaranteeing information integrity of the encoded image, and thereby can obviously improve encoding effects of the image.

Description

A kind of expression of image and disposal route and device

Technical field

The invention belongs to image processing field, relate in particular to a kind of expression and disposal route and device of image.

Background technology

When image (comprising the media datas such as picture and video) is classified or is retrieved, need in the image of to be sorted or retrieval, extract the topography's characteristic information for Description Image, described topography feature is encoded, so that described image is represented, be convenient to the classification of image and retrieval.Can be by becoming the descriptions such as replacement characteristic (SIFT), gradient orientation histogram (HOG) for picture, can be by descriptions such as HOG, light stream histograms (HOF) for video.

Research discovery before this, from different aspect (different visual angles) Description Image, there is certain complementarity in dissimilar local feature between them.And by thering is complementary dissimilar local feature, contribute to improve the effect of classification and the identification of image.

At present the dissimilar local feature disposal route of image is generally: graduation regional area in picture first then extracts the features such as gradient orientation histogram according to these regional areas in primitive frame.When feature is merged, comprise early stage fusion method and later stage fusion method, wherein early stage fusion method is, all local feature series connection is formed to a long proper vector and encode and polymerization, to form the picture in middle level, represents; Later stage fusion method is, list is planted to local feature and encode respectively and polymerization, form a plurality of middle levels picture method for expressing, then middle level represented to series connection or obtain classifying after score in single feature, be weighted fusion, finally middle level picture represented to input in sorter and classify.

Yet, because said method is only by the splicing of image to after local feature before encoding or coding, may cause the imperfect or redundancy too of the coding of graphics finally obtaining, the encoding efficiency of image is not good.

Summary of the invention

The object of the embodiment of the present invention is to provide a kind of expression and disposal route that comprises the image of the shared information between different local features and independent information, when image is encoded in prior art to solve, due to only, by the splicing of the image before coding or after coding, cause the imperfect or problem of redundancy too of coding of graphics.

The embodiment of the present invention is achieved in that a kind of expression and disposal route of image, and described method comprises:

In treating presentation video, extract the local feature of at least two types;

According to different types of local feature for the treatment of to extract in presentation video, to combine to set up and mix independent information decomposition model, described mixing independent information decomposition model comprises shared information and the independent information in dissimilar local feature;

According to described mixing independent information decomposition model, shared information and the independent information of dissimilar local feature to be encoded, the various visual angles super vector that obtains described image represents.

Another object of the embodiment of the present invention is to provide a kind of expression and treating apparatus of image, and described device comprises:

Extraction unit, for treating that presentation video extracts the local feature of at least two types;

Modeling unit, for according to treating different types of local feature that presentation video extracts, combines to set up and mixes independent information decomposition model, and described mixing independent information decomposition model comprises shared information and the independent information in dissimilar local feature;

Coding unit, for according to described mixing independent information decomposition model, encodes to shared information and the independent information of dissimilar local feature, and the various visual angles super vector that obtains described image represents.

In embodiments of the present invention, by extraction, treat the local feature of at least two types in presentation video, combine the mixing independent information decomposition model of setting up the shared information and the independent information that comprise dissimilar local feature, and according to described mixing independent information decomposition model, shared information and the independent information of dissimilar local feature are encoded, the various visual angles super vector that obtains image represents.The various visual angles super vector obtaining due to the present invention is expressed as encodes to shared information and the independent information of dissimilar local feature, can be so that the image information after coding be complete time, also there will not be redundancy, thereby can significantly improve the encoding efficiency of image.

Accompanying drawing explanation

Fig. 1 is the expression of image and the realization flow figure of disposal route that the specific embodiment of the invention provides;

Fig. 2 is the realization flow figure of the parameter training of the mixing independent information decomposition model that provides of the specific embodiment of the invention;

Fig. 3 is the realization flow figure that the various visual angles super vector that obtains image that the specific embodiment of the invention provides represents;

The structural representation of the treating apparatus of the picture that Fig. 4 provides for the specific embodiment of the invention.

Embodiment

In order to make object of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not intended to limit the present invention.

The description that the embodiment of the present invention is mainly used in picture or the video to carry out multiple local feature represents.Described picture and video, and other media data, the in the present invention unified image that is called.Disposal route due to image representation of the present invention, merged multiple local feature, and comprise the super vector that shared information forms by dissimilar local feature, so it can be described as expression and the disposal route of the image based on various visual angles (comprising polytype local feature) super vector.In addition, the expression of image of the present invention and disposal route obtain the various visual angles super vector of image, can be for training classifier, so that use sorter to complete fast the classification to image, also can be according to the similarity estimation function of various visual angles super vector, as Ma Shi function, kernel function etc., retrieve the image with image similarity to be found.Below by embodiment, be specifically described.

Be illustrated in figure 1 the expression of image and the implementing procedure schematic diagram of disposal route described in the embodiment of the present invention.Wherein, in step S101, in treating presentation video, extract the local feature of at least two types.

Wherein, the local feature of described image, local feature for picture, can for the constant converting characteristic of yardstick, (its English full name be Scale-invariant feature transform, its English is referred to as SIFT), (English full name is Histogram of oriented gradient to gradient orientation histogram, English referred to as HoG) etc., for the local feature of video, can be gradient orientation histogram or light stream histogram etc.

The point of interest that described SIFT feature is some local appearance based on object and with size and the irrelevant to rotation of image.For light, noise, visual angle changes slightly tolerance is also quite high.Based on these characteristics, they are highly significant and relatively easily acquisition, in the huge property data base of female number, are easy to identification object and rarely have misidentification.And it is also quite high to use SIFT feature to describe the detecting rate of covering for part object, even only need 3 above SIFT object features to be just enough to calculate position and orientation.

The core concept of described gradient orientation histogram HoG descriptor is that presentation and the shape of the object in piece image can be distributed and describe well by the direction at pixel intensity gradient or edge.Its implementation is first image to be divided into the little grid unit connected region that is called; Then gather gradient direction or the edge orientation histogram of each pixel in grid unit; Finally altogether just can constitutive characteristic descriptor these set of histograms.In order to improve degree of accuracy, these local histograms can also be carried out in the larger interval of image (block) to contrast normalization (contrast-normalized), the method, by first calculating the density of each histogram in this interval (block), is then done normalization according to this density value to each grid unit in interval.After this normalization, can obtain better stability to illumination variation and shade.

Compare with other character description method, histograms of oriented gradients (HOG) descriptor has many good qualities.First, because HoG method is to operate on the local grid unit of image, so it can keep good unchangeability to image geometry with deformation optics, these two kinds of deformation only there will be on larger space field.Secondly, under the sampling of thick spatial domain, meticulous direction sampling and stronger conditions such as indicative of local optical normalization, as long as the posture that pedestrian can be kept upright substantially, can allow that pedestrian has some trickle limb actions, these trickle actions can be left in the basket and not affect detection effect.Histograms of oriented gradients method is the pedestrian detection that is particularly suitable for doing in image.

As can be seen here, dissimilar local feature is from different aspect, i.e. different visual angles Description Image and video, and the advantage between them has complementarity with shortcoming.

In step S102, according to different types of local feature for the treatment of to extract in presentation video, to combine to set up and mix independent information decomposition model, described mixing independent information decomposition model comprises shared information and the independent information in dissimilar local feature.

For extracting two kinds of features corresponding to each regional area in given picture, these two kinds of features are together in series and have obtained comprising the new feature of information from various visual angles.For the further modeling of new feature that series connection is obtained, need to utilize the decomposition model of independent information.For two kinds of low-level image feature x, y, supposes that they meet the following conditions:

x＝W _xz+z _x+ξ _x

y＝W _yz+z _y+ξ _y

Wherein, z represents the shared information of x and two kinds of low-level image features of y, W _x, W _ybe respectively linear transformation, z _xand z _yrepresent respectively the distinctive independent information of x and y, ξ _xand ξ _yrepresenting noise, is error term, respectively Gaussian distributed.Therefore, just like lower probability model:

p(x|z)＝N(W _xz+μ _x,Ψ _x)

p(y|z)＝N(W _yz+μ _y,Ψ _y)

Edge calculation distributes, and we can obtain:

p(x)＝∫P(x|z)p(z)dz＝N(μ _x,W _x(W _x) ^T+Ψ _x)

p(y)＝∫P(y|z)p(z)dz＝N(μ _y,W _y(W _y) ^T+Ψ _y)。

Conventional independent information decomposition model can extract a part of feature that new feature the inside is shared by two kinds of features, but this model can only be processed the relation of linear dependence.In actual data, between different features, often there is nonlinear correlativity, therefore need to adopt mixing independent information decomposition model to carry out modeling to new feature.

As a mixture model, composite dependency analytical model is divided into several regional areas by feature space, and local with the modeling of independent information decomposition model at each.Because the correlativity in regional area can be approximately linear relationship, so overall nonlinear relationship just can be carried out modeling by the mixture model of local linear.Therefore, the nonlinear dependence relation of mixing independent information decomposition model of the present invention between can processing feature.

In a kind of optional embodiment, the mathematical statistical model of described mixing independent information decomposition model can be described as:

$p(x,y)=\underset{k}{\mathrm{\Σ}}{\mathrm{\ω}}_{k}p(x,y|k)=\underset{k}{\mathrm{\Σ}}{\mathrm{\ω}}_k\∫p\left(x\right|z_k,k)p\left(y\right|z_k,k)p\left(z_k\right){\mathrm{dz}}_k,$

The sequence number that wherein k is mixed number, ω _k=P (k) represents the prior probability of k mixed number, z _krepresent the shared variable of x and y in k mixed number, conditional probability normal Distribution, z represents the shared information of x and two kinds of low-level image features of y, W _x, W _ybe respectively linear transformation, z _xand z _yrepresent respectively the distinctive independent information of x and y.

Suppose z _kmeet standard Gaussian distribution, can derive:

a Gaussian distribution N (μ _k, ∑ k), its average is: ${\mathrm{\μ}}_k=\left(\begin{array}{c}{\mathrm{\μ}}_x^k\\ {\mathrm{\μ}}_y^k\end{array}\right),$ Its variance is: $\mathrm{\Σk}=\left[\begin{array}{cc}{W}_x^k{\left(W_x^k\right)}^T+{\mathrm{\Ψ}}_x^k,& W_x^k{\left(W_y^k\right)}^T\\ W_y^k{\left(W_x^k\right)}^T,& W_y^k{\left(W_y^k\right)}^T+{\mathrm{\Ψ}}_y^k\end{array}\right].$

The parameter of described mixing independent information decomposition model, under concrete scene, as the set for multiple different local features, need to carry out concrete training, to obtain parameter corresponding to specific features data, wherein, the training process of parameter is comprised to following steps as shown in Figure 2:

Step S201, initialization mixes the parameter of independent information decomposition model, the parameter of initialized mixing independent information decomposition model comprise the weights of covariance matrix, local Gaussian center of distribution, projection matrix and each local matrix.

Concrete optional, can to all features, carry out the aggregating algorithm processing of k-means, obtain image vocabulary { v _kk=1...K and corresponding to the part association correlation matrix of feature x and y with as each local Gaussian center of distribution, then each local Gaussian is carried out to single independent information decomposition, obtain the parameter corresponding to each local Gaussian, change is projection matrix the weight of each partial model is made as identical, and change is ω _k=1/k.

Step S202, according to the parameter of initialized mixing independent information decomposition model, calculate the estimation corresponding to hidden variable and the posterior probability of sample local feature.

Concrete optional, based on initialized model parameter, calculate corresponding to sample local feature x _iand y _ihidden variable z _i,kand posterior probability γ i, the estimation of k.First under each Local Gaussian Model k, calculate the posterior probability γ of sample local feature i _i,k, computing formula is as follows: described hidden variable z _i,kthe computing formula of estimation as follows: wherein, described ω _krepresent the prior probability of k mixed number, υ _i=[x _i, y _i] represent i local feature pair, it is the center of k mixed number.

And then renewal is corresponding to expectation, covariance and the correlation matrix of hidden variable in each partial model.Wherein, hidden variable expectation calculating formula is: ${\hat{z}}_{i,k}=E\left(z_{i,k}\right)=[{\left(W_x^k\right)}^T,{\left(W_y^k\right)}^T]{\mathrm{\&Sigma;}}_k^{-1}\left[\begin{array}{c}{x}_i-{\mathrm{\&mu;}}_x^k\\ y_i-{\mathrm{\&mu;}}_y^k\end{array}\right],$ Covariance calculating formula is: $E_z^{i,k}=\mathrm{Var}\left(z_{i,k}\right)=I-[{\left(W_x^k\right)}^T,{\left(W_y^k\right)}^T]{\mathrm{\&Sigma;}}_k^{-1}\left[\begin{array}{c}{W}_x^k\\ W_y^k\end{array}\right],$ The computing formula of correlation matrix is: $<z_{i,k}{z_{i,k}}^T>=E\left(z_{i,k}{z_{i,k}}^T\right)={\mathrm{\&Sigma;}}_z^{i,k}+{\hat{z}}_{i,k}{{\hat{z}}_{i,k}}^T.$

Step S203, according to calculated hidden variable and posterior probability, upgrades described parameter, and described parameter comprises center, the projection matrix of described covariance matrix, Local Gaussian Model, the weights of each local matrix.

Concrete optional, based on hidden variable z _i,kand posterior probability γ i, other parameter of k Renewal model, change be weight in each Local Gaussian Model, Local Gaussian Model center,, projection matrix, each local matrix, be specifically as follows:

According to formula calculate the weight of every Local Gaussian Model;

According to formula ${\mathrm{\&mu;}}_x^k=\frac{{\mathrm{\&Sigma;}}_i\widehat{\mathrm{\&gamma;}}i,k(x_i-W_x^k{\hat{z}}_{i,k})}{{\mathrm{\&Sigma;}}_i\widehat{\mathrm{\&gamma;}}i,k},{\mathrm{\&mu;}}_y^k=\frac{{\mathrm{\&Sigma;}}_i\widehat{\mathrm{\&gamma;}}i,k(y_i-W_y^k{\hat{z}}_{i,k})}{{\mathrm{\&Sigma;}}_i\widehat{\mathrm{\&gamma;}}i,k}$ Calculate the center of Local Gaussian Model;

According to formula:

$W_x^k=\left\{\underset{i}{\mathrm{\&Sigma;}}{\widehat{\mathrm{\&gamma;}}}_{i,k}(x_i-{\mathrm{\&mu;}}_x^k){\hat{z}}_{i,k}^T\right\}{\{\underset{i}{\mathrm{\&Sigma;}}{\widehat{\mathrm{\&gamma;}}}_{i,k}<{\hat{z}}_{i,k},{\hat{z}}_{i,k}^T>\}}^{-1},$

$W_y^k=\left\{\underset{i}{\mathrm{\&Sigma;}}{\widehat{\mathrm{\&gamma;}}}_{i,k}(y_i-{\mathrm{\&mu;}}_y^k){\hat{z}}_{i,k}^T\right\}{\{\underset{i}{\mathrm{\&Sigma;}}{\widehat{\mathrm{\&gamma;}}}_{i,k}<{\hat{z}}_{i,k},{\hat{z}}_{i,k}^T>\}}^{-1}$ Calculate covariance matrix;

According to formula

${\mathrm{\&Psi;}}_x^k=\frac{{\mathrm{\&Sigma;}}_i{\widehat{\mathrm{\&gamma;}}}_{i,k}(x_i-W_x^k{\hat{z}}_{i,k}-{\mathrm{\&mu;}}_x^k){(x_i-W_x^k{\hat{z}}_{i,k}-{\mathrm{\&mu;}}_x^k)}^T}{{\mathrm{\&Sigma;}}_i{\widehat{\mathrm{\&gamma;}}}_{i,k}}+W_x^k{\mathrm{\&Sigma;}}_z^k{W}_x^{\mathrm{kT}},$

${\mathrm{\&Psi;}}_y^k=\frac{{\mathrm{\&Sigma;}}_i{\widehat{\mathrm{\&gamma;}}}_{i,k}(y_i-W_y^k{\hat{z}}_{i,k}-{\mathrm{\&mu;}}_y^k){(y_i-W_y^k{\hat{z}}_{i,k}-{\mathrm{\&mu;}}_y^k)}^T}{{\mathrm{\&Sigma;}}_i{\widehat{\mathrm{\&gamma;}}}_{i,k}}+W_y^k{\mathrm{\&Sigma;}}_z^k{W}_y^{\mathrm{kT}}$ Calculate projection matrix;

Wherein, the sequence number that wherein k is mixed number, ω _k=P (k) represents the prior probability of k mixed number, z _krepresent the shared variable of x and y in k mixed number, conditional probability normal Distribution, z represents the shared information of x and two kinds of low-level image features of y, W _x, W _ybe respectively linear transformation, z _xand z _yrepresent respectively the distinctive independent information of x and y.

Step S204, judge whether described parameter convergence restrains, or whether the number of times that calculates described parameter iteration reaches the maximum times of iteration, if described parameter does not restrain or the number of times of described parameter iteration does not reach the maximum times of iteration, proceed to step S202, otherwise proceed to the parameter after the study that step S205 obtains described mixing independent information decomposition model.

In step S103, according to described mixing independent information decomposition model, shared information and the independent information of dissimilar local feature are encoded.Because image can be regarded the set of a dissimilar local feature to (sample) as, we can carry out summary by the coding of local feature on to the basis of local feature coding, and then the various visual angles super vector that obtains described image represents.

The various visual angles super vector that utilize to mix independent information decomposition model design of graphics picture in this step represents, the dissimilar local feature extracting from image can be regarded right the set { (x of local feature of image as _i, y _i).Then we utilize mixing independent information decomposition model to encode to the shared information of dissimilar local feature and various independent information,, and the coding of local feature is carried out to summary, thus the various visual angles super vector of acquisition image represents.Specifically obtaining step that the various visual angles super vector of image represents can be as shown in Figure 3:

In step S301, according to the parameter of described mixing independent information decomposition model, determine the estimation of the hidden variable of each sample, and be weighted integration by posterior probability, obtain the estimation of each Local Gaussian Model hidden variable, and by the estimation series connection of the hidden change of each Local Gaussian Model, obtain the super vector of the information of sharing.

Concrete, mix independent information decomposition model EM algorithm and be averaging as shown in step S202, from each sample, obtain corresponding hidden variable z _i,kestimation, then by these estimate by and posterior probability γ i, k is weighted fusion, just obtains each Local Gaussian Model hidden variable z _kcomputing formula as follows:

the super vector of the information of sharing represents that Z is exactly all local hidden variable z _kseries connection vector, be about to the hidden variable that each local Gaussian obtains and string together form and share information vector.

In step S302, obtain mix independent information decomposition model sample likelihood function respectively with respect to the gradient vector of the parameter of x and y, also referred to as Fisher vector, English full name is Fisher information metric.The independent information that this gradient vector comprises dissimilar local feature x and y.

Concrete, ask model respectively with respect to the parameter of x and y with gradient vector g _xand g _y.In concrete with according to formula

$\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&mu;}}_x^k}=2{\mathrm{\&omega;}}_k{\left({\mathrm{\&Psi;}}_x^k\right)}^{-1}\{{\mathrm{\&mu;}}_x^k-\frac{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}(x_i-W_x^k{\hat{z}}_{i,k})}{\underset{i}{\mathrm{\&Sigma;}}{\mathrm{\&gamma;}}_{i,k}}\}$

$\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&Psi;}}_x^k}={\mathrm{\&omega;}}_k{\left({\mathrm{\&Psi;}}_x^k\right)}^{-1}\{{\mathrm{\&Psi;}}_x^k-\frac{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}(x_{i,k}-W_x^k{\hat{z}}_{i,k}){(x_{i,k}-W_x^k{\hat{z}}_{i,k})}^T}{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}}\}{\left({\mathrm{\&Psi;}}_x^k\right)}^{-1}$ Calculate described mixing independent information decomposition model with respect to the gradient vector of the parameter of x;

According to formula

$\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&mu;}}_y^k}=2{\mathrm{\&omega;}}_k{\left({\mathrm{\&Psi;}}_x^k\right)}^{-1}\{{\mathrm{\&mu;}}_y^k-\frac{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}(y_i-W_y^k{\hat{z}}_{i,k})}{\underset{i}{\mathrm{\&Sigma;}}{\mathrm{\&gamma;}}_{i,k}}\}$

$\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&Psi;}}_y^k}={\mathrm{\&omega;}}_k{\left({\mathrm{\&Psi;}}_y^k\right)}^{-1}\{{\mathrm{\&Psi;}}_y^k-\frac{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}(y_{i,k}-W_y^k{\hat{z}}_{i,k}){(y_{i,k}-W_y^k{\hat{z}}_{i,k})}^T}{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}}\}{\left({\mathrm{\&Psi;}}_y^k\right)}^{-1}$ Calculate described mixing independent information decomposition model with respect to the gradient vector of the parameter of y;

Wherein, representative mixes the log-likelihood function of independent information decomposition model, the sequence number that k is mixed number, ω _k=P (k) represents the prior probability of k mixed number, z _krepresent the shared variable of x and y in k mixed number, W _x, W _ybe respectively linear transformation.

In step S303, by the super vector of described shared information and the described gradient vector that comprises independent information, be together in series, obtain various visual angles super vector and represent.

Be about to share information super vector and represent Z and gradient vector

$G_x=[\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&mu;}}_x^k},\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&Psi;}}_x^k}]$ $G_y=[\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&mu;}}_y^k},\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&Psi;}}_y^k}]$ Be together in series, just obtain last various visual angles super vector and represent.

As in the further embodiment of the embodiment of the present invention, the expression of described image and disposal route also can comprise step S104, according to the various visual angles super vector of obtained different classes of image, training classifier, make sorter to carry out classification step according to the various visual angles super vector of image, or the various visual angles super vector training detecting device according to image block, makes to detect in detected image piece, whether to have certain objects.

Or step S105, according to the phase similarity estimation function of obtained various visual angles super vector and described various visual angles super vector, retrieves the image similar to it.Thereby more convenient effectively completing the classification of image and retrieval.

The embodiment of the present invention is treated the local feature of at least two types in presentation video by extraction, combine the mixing independent information decomposition model of setting up the shared information and the independent information that comprise dissimilar local feature, and according to described mixing independent information decomposition model, shared information and the independent information of dissimilar local feature are encoded, the various visual angles super vector that obtains image represents.The various visual angles super vector obtaining due to the present invention is expressed as encodes to shared information and the independent information of dissimilar local feature, can be so that the image information after coding be complete time, also there will not be redundancy, thereby can significantly improve the encoding efficiency of image.

Expression and the treating apparatus of another embodiment of the invention image as shown in Figure 4, it comprises:

Extraction unit 401, for treating that presentation video extracts the local feature of at least two types;

Modeling unit 402, for according to treating different types of local feature that presentation video extracts, combines to set up and mixes independent information decomposition model, and described mixing independent information decomposition model comprises shared information and the independent information in dissimilar local feature;

Coding unit 403, for according to described mixing independent information decomposition model, encodes to shared information and the independent information of dissimilar local feature, and the coding of local feature is carried out to summary, and the various visual angles super vector that obtains described image represents.

Further, the present invention is afraid of that expression and the disposal route of stating image can also comprise:

Recognition unit 404, for according to the various visual angles super vector of obtained different classes of image, training classifier, can judge according to the various visual angles super vector of image sorter to its classification; Or the various visual angles super vector training detecting device according to image block, makes to detect in detected image piece, whether to have certain objects.

Or retrieval unit 405, for according to the phase similarity estimation function of obtained various visual angles super vector and described various visual angles super vector, retrieves the image similar to it.

Optionally, described mixing independent information decomposition model is:

$p(x,y)=\underset{k}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_{k}p(x,y|k)=\underset{k}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_k\&Integral;p\left(x\right|z_k,k)p\left(y\right|z_k,k)p\left(z_k\right){\mathrm{dz}}_k,$

The sequence number that wherein k is mixed number, ω _k=P (k) represents the prior probability of k mixed number, z _krepresent the shared variable of x and y in k mixed number, conditional probability z represents the shared information of x and two kinds of low-level image features of y, W _x, W _ybe respectively linear transformation, z _xand z _yrepresent respectively the distinctive independent information of x and y;

Described modeling unit comprises:

Initialization subelement, the parameter of mixing independent information decomposition model for initialization, the parameter of initialized mixing independent information decomposition model comprise the weights of covariance matrix, local Gaussian center of distribution, projection matrix and each local matrix;

Computation subunit, for according to the parameter of initialized mixing independent information decomposition model, calculate the estimation corresponding to hidden variable and the posterior probability of sample local feature;

Upgrade subelement, for according to calculated hidden variable and posterior probability, upgrade described parameter, described parameter comprises center, the projection matrix of described covariance matrix, Local Gaussian Model, the weights of each local matrix;

Judgment sub-unit, be used for judging whether described parameter convergence restrains, or whether the number of times that calculates described parameter iteration reaches the maximum times of iteration, if described parameter does not restrain or the number of times of described parameter iteration does not reach the maximum times of iteration, proceed to computing unit, otherwise obtain the parameter after the study of described mixing independent information decomposition model.

Optionally, described coding unit extracts dissimilar local feature from image, form polymorphic local feature sample set, the mixing independent information decomposition model that utilization trains, calculate shared information and the independent information vector of sample set, and then the various visual angles super vector that represents of composing images, specifically comprise:

Super vector obtains subelement, for determine the estimation of the hidden variable of each sample according to the parameter of described mixing independent information decomposition model, and be weighted integration by posterior probability, obtain the estimation of each Local Gaussian Model hidden variable, and by the estimation series connection of the hidden change of each Local Gaussian Model, obtain the super vector of the information of sharing;

Gradient vector is obtained subelement, for obtaining, mixes independent information decomposition model respectively with respect to the gradient vector of the parameter of x and y;

Sub-series unit, is together in series for the super vector by described shared information and described gradient vector, obtains various visual angles super vector and represents.

Described in embodiment of the present invention Fig. 4, the expression of image is corresponding with expression and the disposal route of the image shown in Fig. 1, Fig. 2 and Fig. 3 with treating apparatus, at this, do not repeat.

The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, all any modifications of doing within the spirit and principles in the present invention, be equal to and replace and improvement etc., within all should being included in protection scope of the present invention.

Claims (10)

1. the expression of image and a disposal route, is characterized in that, described method comprises:

In treating presentation video, extract the local feature of at least two types;

According to different types of local feature for the treatment of to extract in presentation video, to combine to set up and mix independent information decomposition model, described mixing independent information decomposition model comprises shared information and the independent information in dissimilar local feature;

According to described mixing independent information decomposition model, shared information and the independent information of dissimilar local feature to be encoded, and the coding of local feature is carried out to summary, the various visual angles super vector that obtains described image represents.

2. method according to claim 1, is characterized in that, described mixing independent information decomposition model is:

$p(x,y)=\underset{k}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_{k}p(x,y|k)=\underset{k}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_k\&Integral;p\left(x\right|z_k,k)p\left(y\right|z_k,k)p\left(z_k\right){\mathrm{dz}}_k,$

The sequence number that wherein k is mixed number, ω _k=P (k) represents the prior probability of k mixed number, z _krepresent the shared variable of x and y in k mixed number, conditional probability normal Distribution; Z represents the shared information of x and two kinds of low-level image features of y, W _x, W _ybe respectively linear transformation, z _xand z _yrepresent respectively the distinctive independent information of x and y;

Different types of local feature that described basis is extracted from treat presentation video, combine foundation mixing independent information decomposition model step and comprise:

A, initialization mixes the parameter of independent information decomposition model, the parameter of initialized mixing independent information decomposition model comprise the weights of covariance matrix, local Gaussian center of distribution, projection matrix and each local matrix;

B, according to the parameter of initialized mixing independent information decomposition model, calculate the estimation corresponding to hidden variable and the posterior probability of sample local feature;

C, according to calculated hidden variable and posterior probability, upgrades described parameter, and described parameter comprises center, the projection matrix of described covariance matrix, Local Gaussian Model, the weights of each local matrix;

D, judge whether described parameter convergence restrains, or whether the number of times that calculates described parameter iteration reaches the maximum times of iteration, if described parameter does not restrain or the number of times of described parameter iteration does not reach the maximum times of iteration, proceed to step B, otherwise obtain the parameter after the study of described mixing independent information decomposition model.

3. method according to claim 2, is characterized in that, described according to the parameter of initialized mixing independent information decomposition model, calculate in the hidden variable and posterior probability step corresponding to sample local feature,

Described posterior probability γ _i,kthe computing formula of estimation as follows: wherein, described ω _krepresent the prior probability of k mixed number, υ _i=[x _i, y _i] represent i local feature pair, it is the center of k mixed number; Described hidden variable z _i,kthe computing formula of estimation as follows: ${\hat{z}}_k=[W_x^{\mathrm{kT}},W_y^{\mathrm{kT}}]{\mathrm{\&Sigma;}}_k^{-1}\underset{i}{\mathrm{\&Sigma;}}{\mathrm{\&gamma;}}_{i,k}({\mathrm{\&upsi;}}_i-{\mathrm{\&mu;}}_k);$

Described according to calculated hidden variable and posterior probability, upgrade described parameter, described parameter comprises center, the projection matrix of described covariance matrix, Local Gaussian Model, the weights step of each local matrix is:

According to formula calculate the weight of every Local Gaussian Model;

According to formula ${\mathrm{\&mu;}}_x^k=\frac{{\mathrm{\&Sigma;}}_i\widehat{\mathrm{\&gamma;}}i,k(x_i-W_x^k{\hat{z}}_{i,k})}{{\mathrm{\&Sigma;}}_i\widehat{\mathrm{\&gamma;}}i,k},{\mathrm{\&mu;}}_y^k=\frac{{\mathrm{\&Sigma;}}_i\widehat{\mathrm{\&gamma;}}i,k(y_i-W_y^k{\hat{z}}_{i,k})}{{\mathrm{\&Sigma;}}_i\widehat{\mathrm{\&gamma;}}i,k}$ Calculate the center of Local Gaussian Model;

According to formula:

$W_x^k=\left\{\underset{i}{\mathrm{\&Sigma;}}{\widehat{\mathrm{\&gamma;}}}_{i,k}(x_i-{\mathrm{\&mu;}}_x^k){\hat{z}}_{i,k}^T\right\}{\{\underset{i}{\mathrm{\&Sigma;}}{\widehat{\mathrm{\&gamma;}}}_{i,k}<{\hat{z}}_{i,k},{\hat{z}}_{i,k}^T>\}}^{-1},$

$W_y^k=\left\{\underset{i}{\mathrm{\&Sigma;}}{\widehat{\mathrm{\&gamma;}}}_{i,k}(x_i-{\mathrm{\&mu;}}_y^k){\hat{z}}_{i,k}^T\right\}{\{\underset{i}{\mathrm{\&Sigma;}}{\widehat{\mathrm{\&gamma;}}}_{i,k}<{\hat{z}}_{i,k},{\hat{z}}_{i,k}^T>\}}^{-1}$ Calculate covariance matrix;

According to formula

${\mathrm{\&Psi;}}_x^k=\frac{{\mathrm{\&Sigma;}}_i{\widehat{\mathrm{\&gamma;}}}_{i,k}(x_i-W_x^k{\hat{z}}_{i,k}-{\mathrm{\&mu;}}_x^k){(x_i-W_x^k{\hat{z}}_{i,k}-{\mathrm{\&mu;}}_x^k)}^T}{{\mathrm{\&Sigma;}}_i{\widehat{\mathrm{\&gamma;}}}_{i,k}}+W_x^k{\mathrm{\&Sigma;}}_z^k{W}_x^{\mathrm{kT}},$

${\mathrm{\&Psi;}}_y^k=\frac{{\mathrm{\&Sigma;}}_i{\widehat{\mathrm{\&gamma;}}}_{i,k}(y_i-W_y^k{\hat{z}}_{i,k}-{\mathrm{\&mu;}}_y^k){(y_i-W_y^k{\hat{z}}_{i,k}-{\mathrm{\&mu;}}_y^k)}^T}{{\mathrm{\&Sigma;}}_i{\widehat{\mathrm{\&gamma;}}}_{i,k}}+W_y^k{\mathrm{\&Sigma;}}_z^k{W}_y^{\mathrm{kT}}$ Calculate projection matrix;

Wherein, the sequence number that wherein k is mixed number, ω _k=P (k) represents the prior probability of k mixed number, z _krepresent the shared variable of x and y in k mixed number, conditional probability normal Distribution, z represents the shared information of x and two kinds of low-level image features of y, W _x, W _ybe respectively linear transformation, z _xand z _yrepresent respectively the distinctive independent information of x and y.

4. method according to claim 1, it is characterized in that, described according to described mixing independent information decomposition model, shared information and independent information to dissimilar local feature are encoded, and the coding of local feature is carried out to summary, the various visual angles super vector that obtains described image represents, concrete steps comprise:

According to each sample of parameter estimation of described mixing independent information decomposition model, it is the right shared hidden variable of local feature, and be weighted integration by the posterior probability of each sample, obtain the estimation of each Local Gaussian Model hidden variable, and by the estimation series connection of the hidden change of each Local Gaussian Model, obtain the super vector of the information of sharing;

Obtain mix independent information decomposition model sample likelihood function respectively with respect to the parameter of x and y gradient vector, the independent information that this gradient vector comprises dissimilar local feature x and y;

By the super vector of described shared information and the described gradient vector that comprises independent information, be together in series, obtain various visual angles super vector and represent.

5. method according to claim 4, it is characterized in that, the described parameter estimation according to described mixing independent information decomposition model is determined each sample, it is the estimation of the right shared hidden variable of local feature, and be weighted integration by the posterior probability of each sample, obtain the estimation of each Local Gaussian Model hidden variable, and by the estimation series connection of the hidden change of each Local Gaussian Model, obtain in the super vector step of the information of sharing, according to formula calculate the estimation of each Local Gaussian Model hidden variable;

The hidden variable that each local Gaussian is obtained strings together form and share information vector;

Described obtaining mixed independent information decomposition model respectively with respect to the parameter of x and y gradient vector step in, according to formula

$\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&mu;}}_x^k}=2{\mathrm{\&omega;}}_k{\left({\mathrm{\&Psi;}}_x^k\right)}^{-1}\{{\mathrm{\&mu;}}_x^k-\frac{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}(x_i-W_x^k{\hat{z}}_{i,k})}{\underset{i}{\mathrm{\&Sigma;}}{\mathrm{\&gamma;}}_{i,k}}\}$

$\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&Psi;}}_x^k}={\mathrm{\&omega;}}_k{\left({\mathrm{\&Psi;}}_x^k\right)}^{-1}\{{\mathrm{\&Psi;}}_x^k-\frac{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}(x_{i,k}-W_x^k{\hat{z}}_{i,k}){(x_{i,k}-W_x^k{\hat{z}}_{i,k})}^T}{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}}\}{\left({\mathrm{\&Psi;}}_x^k\right)}^{-1}$ Calculate described mixing independent information decomposition model with respect to the gradient vector of the parameter of x;

According to formula

$\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&mu;}}_y^k}=2{\mathrm{\&omega;}}_k{\left({\mathrm{\&Psi;}}_x^k\right)}^{-1}\{{\mathrm{\&mu;}}_y^k-\frac{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}(y_i-W_y^k{\hat{z}}_{i,k})}{\underset{i}{\mathrm{\&Sigma;}}{\mathrm{\&gamma;}}_{i,k}}\}$

$\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&Psi;}}_y^k}={\mathrm{\&omega;}}_k{\left({\mathrm{\&Psi;}}_y^k\right)}^{-1}\{{\mathrm{\&Psi;}}_y^k-\frac{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}(y_{i,k}-W_y^k{\hat{z}}_{i,k}){(y_{i,k}-W_y^k{\hat{z}}_{i,k})}^T}{{\mathrm{\&Sigma;}}_i{\mathrm{\&gamma;}}_{i,k}}\}{\left({\mathrm{\&Psi;}}_y^k\right)}^{-1}$ Calculate described mixing independent information decomposition model with respect to the gradient vector of the parameter of y;

Wherein, representative mixes the log-likelihood function of independent information decomposition model, the sequence number that k is mixed number, ω _k=P (k) represents the prior probability of k mixed number, z _krepresent the shared variable of x and y in k mixed number, W _x, W _ybe respectively linear transformation;

Wherein, $G_x=[\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&mu;}}_x^k},\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&Psi;}}_x^k}]$ With $G_y=[\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&mu;}}_y^k},\frac{\&PartialD;E\left(L\right)}{\&PartialD;{\mathrm{\&Psi;}}_y^k}]$ Respectively in representative image about the independent information of x and y; By sharing information vector and independent information vector, string together the various visual angles super vector that gets final product composing images and represent [Z, Gx, Gy].

6. method according to claim 1, is characterized in that, described method also comprises:

According to the various visual angles super vector of obtained different classes of image, training classifier, can judge according to the various visual angles super vector of image sorter to its classification;

Or the various visual angles super vector training detecting device according to image block, makes to detect in detected image piece, whether to have certain objects.

7. method according to claim 1, is characterized in that, described method also comprises:

According to the phase similarity estimation function of obtained various visual angles super vector and described various visual angles super vector, retrieve the image similar to it.

8. the expression of image and a treating apparatus, is characterized in that, described device comprises:

Extraction unit, for treating that presentation video extracts the local feature of at least two types;

Modeling unit, for according to treating different types of local feature that presentation video extracts, combines to set up and mixes independent information decomposition model, and described mixing independent information decomposition model comprises shared information and the independent information in dissimilar local feature;

Coding unit, for according to described mixing independent information decomposition model, encodes to shared information and the independent information of dissimilar local feature, and the coding of local feature is carried out to summary, and the various visual angles super vector that obtains described image represents.

9. install according to claim 8, it is characterized in that, described mixing independent information decomposition model is:

$p(x,y)=\underset{k}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_{k}p(x,y|k)=\underset{k}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_k\&Integral;p\left(x\right|z_k,k)p\left(y\right|z_k,k)p\left(z_k\right){\mathrm{dz}}_k,$

The sequence number that wherein k is mixed number, ω _k=P (k) represents the prior probability of k mixed number, z _krepresent the shared variable of x and y in k mixed number, conditional probability z represents the shared information of x and two kinds of low-level image features of y, W _x, W _ybe respectively linear transformation, z _xand z _yrepresent respectively the distinctive independent information of x and y;

Described modeling unit comprises:

Initialization subelement, the parameter of mixing independent information decomposition model for initialization, the parameter of initialized mixing independent information decomposition model comprise the weights of covariance matrix, local Gaussian center of distribution, projection matrix and each local matrix;

Computation subunit, for according to the parameter of initialized mixing independent information decomposition model, calculate the estimation corresponding to hidden variable and the posterior probability of sample local feature;

Upgrade subelement, for according to calculated hidden variable and posterior probability, upgrade described parameter, described parameter comprises center, the projection matrix of described covariance matrix, Local Gaussian Model, the weights of each local matrix;

Judgment sub-unit, be used for judging whether described parameter convergence restrains, or whether the number of times that calculates described parameter iteration reaches the maximum times of iteration, if described parameter does not restrain or the number of times of described parameter iteration does not reach the maximum times of iteration, proceed to computing unit, otherwise obtain the parameter after the study of described mixing independent information decomposition model.

10. install according to claim 8, it is characterized in that, described coding unit extracts dissimilar local feature from image, form polymorphic local feature sample set, the mixing independent information decomposition model that utilization trains, calculate shared information and the independent information vector of sample set, and then the various visual angles super vector that represents of composing images, specifically comprise:

Super vector obtains subelement, for determine the estimation of the hidden variable of each sample according to the parameter of described mixing independent information decomposition model, and be weighted integration by posterior probability, obtain the estimation of each Local Gaussian Model hidden variable, and by the estimation series connection of the hidden change of each Local Gaussian Model, obtain the super vector of the information of sharing;

Gradient vector is obtained subelement, for obtaining, mixes independent information decomposition model respectively with respect to the gradient vector of the parameter of x and y, represents the independent information of all kinds of local features;

Sub-series unit, is together in series for the super vector by described shared information and described gradient vector, obtains various visual angles super vector and represents.