CN102495865A - Image annotation method combined with image internal space relation and visual symbiosis relation - Google Patents

Abstract

The invention discloses an image annotation method combined with an image internal space relation and a visual symbiosis relation. The method consists of image segmentation, characteristic extraction and an annotation algorithm, and comprises the following steps of: firstly, segmenting an image into a plurality of regions by using an over-segmentation method; secondly, extracting the visual characteristics of each region; and lastly, establishing an image annotation classifying model by using context information such as a space position relation, the visual symbiosis relation and the like among regions in an image. The image annotation method has the advantages that: the image annotation accuracy is high, and the accuracy of image annotation can be increased by fully and effectively using two kinds of different context information of the space position relation and the visual symbiosis relation in the image.

Description

The image labeling method of combining image inner space relation and vision symbiosis

Technical field

The present invention relates to a kind of image labeling algorithm, relate in particular to the image labeling method of a kind of combining image inner space relation and vision symbiosis, belong to technical field of image processing based on contextual information.

Background technology

Along with the development of Internet and Digital image technology, view data magnanimity increases, and all is a great challenge to tissue, analysis, retrieval and the management etc. of image.People have reached unprecedented scale more for the interest of the semantic concept that image comprised, thereby press for a kind of image management method that meets human perception and cognitive mechanism, understands based on semantic concept.Image labeling can solve " semantic wide gap " problem that exists in the image retrieval to a certain extent through setting up the mapping relations between low layer visual signature and the high-level semantic.Image labeling can be divided into manual mark and mark two types automatically.Using artificial mode to carry out image labeling is the most also to be effective and efficient manner; But this also is a job that takes time and effort very much, and therefore having at present much has a lot of websites and tissue to encourage the users on the web to mark for the image that it provides at present.Yet along with the sharp increase of amount of images, depending artificial mark alone can not meet the demands, and this has also promoted the research of the automatic marking method of image.

Hanbury etc. are divided into three kinds according to the form of mark with the mark of image, based on the mark of keyword, based on the mark of body with based on the mark of natural language.In the research work of image labeling, study maximum marks that is based on keyword at present.And the area marking of image is based on a kind of form common in the image labeling of keyword, and contextual information comprises that spatial relationship, symbiosis etc. are used widely, and effectively raises the accuracy of image labeling in the image-region mark.Its main flow process comprises three parts: image segmentation, feature extraction and dimensioning algorithm.But the conventional images mask method can not utilize various different contextual informations in the image simultaneously.

Summary of the invention

Goal of the invention: to the problems and shortcomings that exist in the prior art, the present invention provides a kind of can utilize the combining image inner space relation of various different contextual informations in the image and the image labeling method of vision symbiosis simultaneously.

Technical scheme: the image labeling method of a kind of combining image inner space relation and vision symbiosis comprises the steps:

(1) utilize the over-segmentation method that every width of cloth image segmentation is several regions, this target of cutting apart is that the different objects in the image is divided into different zones, makes only to comprise other object of unitary class in each cut zone;

(2) to each zone in the image, extract characteristic informations such as color, texture, shape, locus, constitute the proper vector of one group of successive value;

(3) utilize Kmeans that the proper vector of all successive values is carried out cluster, obtain K cluster centre, constitute vocabulary V;

(4) utilize vocabulary V that each regional proper vector is quantized, thereby obtain the vision keyword W of each pixel;

(5) to the spatial relation of all cut zone in the image, consider the spatial relationship between the adjacent area, set up single order markov network model;

(6) according to the vision keyword of All Ranges in the image, add up its vision keyword histogram, utilize implicit Di Li Cray to divide the symbiosis between the pairing vision keyword to carry out modeling;

(7) spatial relationship between each zone and symbiosis in the combining image are set up a kind of probability graph model that combines single order markov network and implicit Di Li Cray to distribute;

(8) characteristic, quantification, structure and training pattern are cut apart, extracted to the image data set of the artificial mark of utilization to (7), according to said step (1), obtains the parameter of a group model;

(9) to the image that does not mark, the parameter initialization model that utilizes training to obtain, and according to characteristic of being extracted and vision keyword, each cut zone is marked.

Method of the present invention comprises structure, the training of model parameter and four parts of classification that do not mark image that the cutting apart of image, feature extraction and quantification, single order markov network model and implicit Di Li Cray distribute; Wherein step (1) to (4) has been described cutting apart and the extraction of visual signature and the construction process of vision keyword of image; Step (5) to (7) has been described the structure of single order markov network model and implicit Di Li Cray distribution; And the combination of two kinds of probability graph models, step (8) to (9) has been described the training and the classification problem of how carrying out image labeling based on above-mentioned probability graph model.

Beneficial effect: method of the present invention compared with prior art; Its remarkable advantage is: can effectively separate the accuracy that the spatial positional information that combines in the image and two kinds of different contextual informations of vision symbiosis improve image labeling; Through using single order markov network model the spatial relation of adjacent area in the image is described; And through utilizing implicit Di Li Cray to divide the vision symbiosis in the pairing image to carry out modeling; And combine two kinds of different probability graph models in view of the above, carry out Parameter Optimization according to training dataset, thereby reached better pictures mark accuracy.

Description of drawings

Fig. 1 is the latent dirichlet allocation model in the embodiment of the invention;

Fig. 2 is the single order markov network model in the embodiment of the invention;

Fig. 3 is the structure that single order markov network and the implicit Di Li Cray in the embodiment of the invention distributes;

Fig. 4 is the disaggregated model that combines single order markov network model and latent dirichlet allocation model in the embodiment of the invention: implicit Di Li Cray markov network model.

Embodiment

Below in conjunction with accompanying drawing and specific embodiment; Further illustrate the present invention; Should understand these embodiment only be used to the present invention is described and be not used in the restriction scope of the present invention; After having read the present invention, those skilled in the art all fall within the application's accompanying claims institute restricted portion to the modification of the various equivalent form of values of the present invention.

When utilizing latent dirichlet allocation model that the target in the image is detected, discerns and locating; Ignore the spatial positional information between each zone in the image, need quantize each regional visual signature simultaneously, can only handle the vision keyword of discretize; And single order markov network model can be handled the visual signature (or proper vector) of successive value; Therefore spatial relationship between simultaneously can processing region, the present invention proposes a kind of image labeling model that combines latent dirichlet allocation model and single order markov network model; The subject information that utilizes latent dirichlet allocation model to generate is described the vision symbiosis in the image; Thereby improve single order markov network model classification effect, when utilizing latent dirichlet allocation model to generate each regional subject information simultaneously, not only consider the keyword that each node is corresponding; Also add corresponding class information, thereby further improved the accuracy of image labeling.

The detailed process of the inventive method is following:

This method comprises the following steps:

Step (1) utilizes the over-segmentation method that every width of cloth image segmentation is several regions, and this target of cutting apart is that the different objects in the image is divided into different zones, makes only to comprise other object of unitary class in each cut zone;

Step (2) is extracted characteristic informations such as color, texture, shape, locus to each zone in the image, constitutes the proper vector of one group of successive value;

Step (3): utilize Kmeans that all successive value proper vectors are carried out cluster, obtain K cluster centre, constitute vocabulary V.To the successive value proper vector that 27 of all pixels in the present image are tieed up, utilize Kmeans to carry out cluster, obtain K cluster centre (c ₁, c ₂..., c _K), constitute visual vocabulary Table V={ c ₁, c ₂..., c _K.

Step (4): utilize vocabulary V to each successive value proper vector H _iQuantize, to the successive value proper vector H of each pixel _iIn vocabulary, select the minimum vision keyword of Euclidean distance

Thereby original image is converted into the image of forming by the vision keyword of each pixel.

Step (5) is considered the spatial relationship between the adjacent area to the spatial relation of all cut zone in the image, sets up single order markov network model, and is as shown in Figure 3.Single order markov network model model is the mark that is used for natural language the earliest equally; Image data processing need be used the single order markov network model of two dimension; Its structure is as shown in Figure 2; Wherein distinguish each regional visual signature and classification in the representative image, one group of corresponding probability of mark sequence of image is:

$P\left(c\right|x)=\frac{\exp (\underset{i\∈N}{\mathrm{\Σ}}\mathrm{uF}(x_i,c_i)+\underset{\mathrm{ij}\∈E}{\mathrm{\Σ}}\mathrm{vF}(x_i,x_j,c_i,c_j))}{Z\left(x\right)}.---\left(1\right)$

$Z\left(x\right)=\underset{c}{\mathrm{\Σ}}\exp (\underset{i\∈N}{\mathrm{\Σ}}\mathrm{uF}(x_i,c_i)+\underset{\mathrm{ij}\∈E}{\mathrm{\Σ}}\mathrm{vF}(x_i,x_j,c_i,c_j)).---\left(2\right)$

Wherein N and E represent node (Node) and the set of limit (Edge) in the single order markov network model respectively, and u is the corresponding weight of visual signature of each node, and v is the corresponding weight of boundary characteristic, F (x _i, c _i) characteristic of corresponding present node, F (x _i, x _j, c _i, c _j) characteristic on corresponding current border.

Step (6) is added up its vision keyword histogram according to the vision keyword of All Ranges in the image, utilizes implicit Di Li Cray to divide the symbiosis between the pairing vision keyword to carry out modeling.It is the topic model a kind of commonly used in the natural language processing that implicit Di Li Cray distributes, and obtains the subject information P (w of different keywords in different document through the information such as the keyword frequency of occurrences in the statistics document _n| z _k, d).When utilizing implicit Di Li Cray to distribute to come image data processing, at first need image be carried out piecemeal, then each segmented areas is extracted characteristic, and be quantified as plurality of keywords.Here suppose total N keyword w in the vocabulary _n, n=1 ..., N representes k theme, the quantity that K is the theme, d=1 ..., D represents D document.The probability graph structure that implicit Di Li Cray distributes is as shown in Figure 1, and wherein α is the K dimensional vector, and P (θ | α) satisfy Dirichlet and distribute, P (z| θ) satisfies polynomial expression and distributes β _Kn=P (w=n|z=k).P (w _n| z _k, d) be illustrated among the document d corresponding theme z _kKeyword be w _nProbability.Quantity according to the artificial definite cut zone of priori is K; According to collected works C; Training a theme quantity is that the implicit Di Li Cray of K distributes, thereby obtains the probability

that each pixel in each zone belongs to different themes

Spatial relationship and symbiosis in step (7) combining image between each zone are set up a kind of probability graph model that combines single order markov network and implicit Di Li Cray to distribute.Here we use single order markov network model to handle spatial relationship and successive value characteristic (or proper vector), and the classification of each node also can receive the influence of subject information simultaneously, and the condition probability formula of implicit Di Li Cray markov network is:

$p\left(c\right|z,x,u,v)=\frac{\exp (\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\underset{i}{\mathrm{\Σ}}\mathrm{\δ}(z_i=k)u_{k}F(x_i,c_i)+\underset{\mathrm{ij}}{\mathrm{\Σ}}\mathrm{vF}(x_i,x_j,c_i,c_j))}{Z(u,v,x,z)}.---\left(3\right)$

$\mathrm{\δ}(x-a)=\left\{\begin{array}{cc}1& \mathrm{if\; x}=a\\ 0& o.w.\end{array}\right..---\left(4\right)$

As can be seen from Figure 4; Compare with latent dirichlet allocation model; When generating each regional subject information, not only consider the vision keyword that each zone is corresponding, also can utilize each regional class label; And the class label between the zone exists spatial relationship, distributes the deficiency that can't utilize spatial positional information thereby remedied implicit Di Li Cray; Compare with single order markov network model model, the branch time-like is being carried out in each zone, not only consider the visual signature of present node and neighborhood node, also can utilize the subject information of current region to assist and classify.Thereby improve the effect of image labeling through the advantage that combines two kinds of models.

Step (8) is utilized the image data set of artificial mark, according to above-mentioned steps, cuts apart, extracts characteristic, quantification, structure and training pattern, obtains the parameter of a group model.The target of the implicit Di Li Cray markov network of training is in order to seek parameter { α, β, s}=argmax _{{ α, β, s}}(s), for this reason, we at first set up expansion single order markov network model model to logP, utilize variational method (Variational Methods) and expansion single order markov network model to find the solution parameter then for D| α, β.The conditional probability distribution of expansion single order markov network model is:

$P\left(c\right|x,\mathrm{\φ},s)=\frac{\exp (\underset{i}{\mathrm{\Σ}}{\left[\begin{array}{c}{\mathrm{\φ}}_{i1}\\ M\\ {\mathrm{\φ}}_{\mathrm{ik}}\end{array}\right]}^T\left[\begin{array}{c}{u}_1\\ M\\ u_k\end{array}\right]F(x_i,c_i)+\underset{\mathrm{ij}}{\mathrm{\Σ}}\mathrm{vF}(x_i,x_j,c_i,c_j))}{Z(x,\mathrm{\φ},s)}.---\left(5\right)$

φ wherein _Ik=P (z _i=k) be the probability of corresponding k the theme of node i.Can find out that expansion single order markov network model utilizes the probability of theme that the node diagnostic of single order markov network model is expanded.

The training step of implicit Di Li Cray markov network is as shown in the table:

Claims (2)

1. the image labeling method of combining image inner space relation and vision symbiosis is characterized in that comprising the following steps:

(1) utilizes the over-segmentation method that every width of cloth image segmentation is several regions, the different objects in the image is divided into different zones, make only to comprise other object of unitary class in each cut zone;

(2) to each zone in the image, extract color, texture, shape and locus characteristic information, constitute the proper vector of one group of successive value;

(3) utilize Kmeans that all successive value proper vectors are carried out cluster, obtain K cluster centre, constitute vocabulary V;

(4) utilize vocabulary V that each regional proper vector is quantized, thereby obtain the vision keyword W of each pixel;

(5) to the spatial relation of all cut zone in the image, consider the spatial relationship between the adjacent area, set up single order markov network model;

(6) according to the vision keyword of All Ranges in the image, add up its vision keyword histogram, utilize implicit Di Li Cray to divide the symbiosis between the pairing vision keyword to carry out modeling;

(7) spatial relationship between each zone and symbiosis in the combining image are set up a kind of probability graph model that combines single order markov network and implicit Di Li Cray to distribute;

(8) characteristic, quantification, structure and training pattern are cut apart, extracted to the image data set of the artificial mark of utilization to (7), according to said step (1), obtains the parameter of a group model;

(9) to the image that does not mark, the parameter initialization model that utilizes training to obtain, and according to characteristic of being extracted and vision keyword, each cut zone is marked.

2. the image labeling method of combining image as claimed in claim 1 inner space relation and vision symbiosis; It is characterized in that: in the said step (6) when utilizing implicit Di Li Cray to distribute to come image data processing; At first need image be carried out piecemeal; Then each segmented areas is extracted characteristic; And be quantified as plurality of keywords, and utilize implicit Di Li Cray to distribute the vision symbiosis contextual information of handling in the image, utilize single order markov network model that the spatial positional information between the zone in the image is carried out modeling simultaneously.

Images