JP2013061732A - Image identification information provision program and image identification information provision device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image identification information provision program and an image identification information provision device, which provide input image information with identification information by using a learning result obtained by learning using a probability model.SOLUTION: An image identification information provision device 1 includes: image obtainment means 100 for obtaining image information, for each piece of identification information, from image information pre-correlated with the identification information; feature vector generation means 102 for generating feature information from the obtained image information; and model learning-means 104 for generating a learning result by learning relation between the generated feature information and the identification information of image information corresponding the feature information by using a probability model by mixing probability distribution. Further, the model learning-means 104 determines variables in the process of generating mixing elements of the probability model by using a set of feature information obtained from all pieces of image information obtained by the obtainment means regardless of contents of the identification information and determines prior distribution for the variables of the probability distribution.

Description

  The present invention relates to an image identification information providing program and an image identification information providing apparatus.

  As a conventional image identification information providing device, when identification information (hereinafter referred to as “label”) attached based on feature information obtained from image information or the like is prepared in advance, the relationship between the feature information and the label is previously determined. There is one that learns and identifies which label the input image information belongs to based on the result of the learning (see, for example, Patent Document 1).

  The pattern recognition device described in Patent Literature 1 generates multidimensional feature information from image information, learns by associating a label with feature information in which feature information is prepared in advance as learning data, By calculating the feature information of the input image information and referring to the learned feature information, a value indicating the degree to which the feature information of the input image information belongs to each label is calculated, and the degree to which the information belongs An image identification information assigning device that recognizes that image information input to a label having the maximum value belongs is disclosed.

JP-A-9-231366

  An object of the present invention is to provide an image identification information adding program and an image identification information adding device for adding identification information to input image information using a learning result learned by a probability model.

  In order to achieve the above object, one aspect of the present invention provides the following image identification information providing program and image identification information providing apparatus.

[1]
Acquisition means for acquiring image information for each identification information from image information associated with the identification information in advance;
Generating means for generating feature information from the image information acquired by the acquiring means;
A learning result is generated by learning the relationship between the feature information generated by the generating means and the identification information of the image information corresponding to the feature information using a probability model based on a mixture of probability distributions, and the mixture of the probability models Learning to determine a variable of an element generation process using a set of feature information obtained from all image information acquired by the acquisition means regardless of the content of identification information, and to determine a prior distribution for the variable of the probability distribution Means,
Based on the learning result generated by the learning unit, a calculation unit that calculates a posterior distribution of feature information generated by the generation unit from image information to which identification information is added using the probability model for each piece of identification information When,
An image identification information addition program that functions as an estimation unit that estimates identification information associated with image information to which the identification information is to be added based on the posterior distribution.

[2] The image identification information according to [1], wherein the learning unit determines an initial value of the prior distribution variable using a set of feature information obtained from all image information regardless of the content of the identification information. Grant program.

[3] Acquisition means for acquiring image information for each identification information from image information associated with the identification information in advance;
Generating means for generating feature information from the image information acquired by the acquiring means;
A learning result is generated by learning the relationship between the feature information generated by the generating means and the identification information of the image information corresponding to the feature information using a probability model based on a mixture of probability distributions, and the mixture of the probability models Learning to determine a variable of an element generation process using a set of feature information obtained from all image information acquired by the acquisition means regardless of the content of identification information, and to determine a prior distribution for the variable of the probability distribution Means,
Based on the learning result generated by the learning unit, a calculation unit that calculates a posterior distribution of feature information generated by the generation unit from image information to which identification information is added using the probability model for each piece of identification information When,
An image identification information providing apparatus comprising: estimation means for estimating identification information associated with image information to which the identification information is to be applied based on the posterior distribution.

  According to the invention which concerns on Claim 1 or 3, compared with the case where it does not have the structure of this invention, the over-learning with respect to learning data with which identification information is known is suppressed, and the identification performance with respect to the input image information is improved. The identification information can be given to the input image information.

  According to the invention of claim 2, the initial value of the prior distribution variable can be determined using a set of feature information obtained from all image information regardless of the content of the identification information.

FIG. 1 is a diagram showing an example of the configuration of an image identification information providing apparatus according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing an example of a probability generation model used by the model learning means for the learning operation. FIG. 3 is a flowchart illustrating an operation example of the image identification information adding apparatus. FIG. 4 is a flowchart showing the contents of the learning algorithm. FIG. 5 is a flowchart illustrating an operation example of the image identification information adding apparatus.

(Configuration of image identification information adding device)
FIG. 1 is a diagram showing an example of the configuration of an image identification information providing apparatus according to an embodiment of the present invention.

  The image identification information assigning device 1 is composed of a CPU (Central Processing Unit) and the like, and is configured by a control unit 10 that controls each unit and executes various programs, and a storage medium such as an HDD (Hard Disk Drive) and a flash memory. A storage unit 11 that stores information and a communication unit 12 that communicates with the outside via a network are provided.

  When the image input through the communication unit 12 or the like includes an object such as “mountain”, “river”, “child”, etc., the image identification information assigning device 1 (Hereinafter referred to as “annotation word”) is assigned as identification information for the image information (hereinafter referred to as “label”). Moreover, the image identification information provision apparatus 1 learns using the image information for learning by which the label was previously provided stored in the memory | storage part 11 grade | etc.,.

  The control unit 10 executes an image identification information adding program 110 to be described later, whereby an image acquisition unit 100, an image division unit 101, a feature vector generation unit 102, a learning data set acquisition unit 103, a model learning unit 104, a likelihood calculation. Functions as means 105, annotation word estimation means 106, output means 107, and the like.

  The image acquisition unit 100 acquires image information stored in the storage unit 11 during learning, and acquires image information input from an external terminal device or the like via the communication unit 12 when estimating a label. .

  The image dividing unit 101 divides the image information acquired by the image acquiring unit 100 and the learning image information 111 stored in the storage unit 11 into a plurality of regions, and generates partial regions. The image dividing unit 101 uses, for example, a method of dividing a rectangle into a mesh shape or a method of determining similar adjacent pixels as the same partial region using a clustering method such as a k-nearest neighbor method.

  The feature vector generating unit 102 generates a feature vector from each of the partial regions generated by the image dividing unit 101 by, for example, a method using a Gabor filter or a method of extracting feature quantities such as RGB, normalized RG, CIELAB. Here, the feature vector is an example of feature information.

  The learning data set acquisition unit 103 acquires image information provided with the same label from the image information 111, and acquires a set of feature vectors included in the image information as a learning data set. Also, a set of feature vectors (hereinafter referred to as “universal model”) obtained from all image information 111 regardless of the contents of the label is acquired as a learning data set.

  The model learning unit 104 performs learning using the learning data set acquired by the learning data set acquisition unit 103.

  The likelihood calculating unit 105 calculates the likelihood for an arbitrary label of the feature vector of the image information acquired by the image acquiring unit 100.

  The annotation word estimation unit 106 estimates an annotation word corresponding to a label with high likelihood as identification information of the input image information.

  The output unit 107 outputs, for example, the top several words with the highest likelihood among the annotation words estimated by the annotation word estimation unit 106 to the display device, the printing device, the storage unit 11, and the like.

  The storage unit 11 includes an image identification information adding program 110 that causes the control unit 10 to operate as each unit described above, image information 111 used during learning, label information 112 that associates image information and labels included in the image information 111, and The learning information 113 stored as a learning result by the model learning unit 104 is stored.

(Operation of the image identification information adding device)
Hereinafter, the operation example of the image identification information assigning apparatus will be described by dividing into (1) basic learning operation, (2) detailed learning operation, and (3) annotation estimation operation with reference to each drawing.

  FIG. 3 is a flowchart illustrating an operation example of the image identification information adding apparatus.

(1) Basic Learning Operation First, the image acquisition unit 100 acquires the image information 111 as learning data from the storage unit 11 (S1). The image information 111 includes a plurality of pieces of image information.

  Next, the image dividing unit 101 divides the image into n regions and generates a partial region (S2). As an example, a rectangular mesh is divided. This operation is performed for each of a plurality of pieces of image information included in the image information 111.

Next, the feature vector generation unit 102 extracts a plurality of feature amounts from the partial region using a Gabor filter or the like as an example, and feature vectors x ₁ , x ₂ ,... Using these feature amounts as components for each partial region. , _Xn are generated (S3). This operation is also performed for each of a plurality of pieces of image information included in the image information 111.

Next, the learning data set obtaining unit 103 refers to the label information 112, from the image information 111, first, acquires the image information associated with the label C _1, the feature vector generated from the acquired image information A set is acquired as a learning data set (S4, S5).

Next, the model learning unit 104 performs learning on the learning data of the label C ₁ acquired by the learning data set acquisition unit 103 (S6), and stores the learning result in the learning information 113 of the storage unit 11 (S7).

  Steps S5 to S7 are executed for all labels (upper limit M) (S18, S19).

  In the following “(2) Detailed learning operation”, details of the learning operation executed by the model learning unit 104 in step S6 will be described.

ここで、入力される学習データ数をＮ、混合要素数をＫとしている。π_kは混合比を表わし、Ｎ(ｘ_i｜μ_k,Λ_k ^-1)は、平均μ_k、分散Λ_k ^-1であるＤ次元ガウス分布である。 (2) Detailed Learning Operation The model learning unit 104 uses a Gaussian Mixture Model (GMM) as a probability generation model. If the input learning data set is X = {x ₁ ,... X _N } and the dimension of the feature vector is D, the mixed Gaussian model p is defined by the following equation.

Here, the number of input learning data is N, and the number of mixed elements is K. π _k represents a mixing ratio, and N (x _i | μ _k , Λ _k ⁻¹ ) is a D-dimensional Gaussian distribution having an average μ _k and a variance Λ _k ⁻¹ .

ここで、混合要素数Ｋは、事前に定めず、無限個の混合比率の和としている。 The mixing ratio satisfies the following formula.

Here, the number K of mixing elements is not determined in advance, and is the sum of an infinite number of mixing ratios.

ここで、は、パラメータαをもつベータ分布Beta(1,α)に従い、さらにαは、Ｃ＝{c₁,c₂}をハイパーパラメータに持つガンマ分布Gamma(c₁,c₂)に従うものとする。 Further, in order to appropriately select the mixing element in accordance with the data, the mixing ratio π is assumed to be generated from a Dirichlet process (Stick breaking expression).

Here, follows a beta distribution Beta (1, α) having a parameter α, and α follows a gamma distribution Gamma (c ₁ , c ₂ ) having C = {c ₁ , c ₂ } as hyperparameters. To do.

Further, in order to enable more flexible probability estimation, a prior distribution is determined for the parameters (mean and variance) of the Gaussian distribution. Here, the Gaussian distribution and the Wishart distribution, which are conjugate prior distributions of the average value and the variance, are determined as the respective prior distributions. That is,

ここで、

Here, the Wishart distribution W (Λ _k | W ₀ , υ ₀ ) is defined by the following equation.

here,

  The above is the description of the probability generation model used in the embodiment of the present invention. This is illustrated by a Bayesian network expression as shown in FIG.

  FIG. 2 is a schematic diagram illustrating an example of a probability generation model used by the model learning unit 104 for a learning operation.

C ₁ , c ₂ , W ₀ , ν ₀ , β ₀ , m ₀ indicated by black circles represent constants, and α _π , π, v, z _n , x _n , Λ, indicated by hollow circles μ represents a random variable. _Xn represents an observation node. Arrows represent probabilistic dependencies. A square including z _n and x _n indicates an observation value of N independent simultaneous distributions.

Finding the maximum likelihood solution of model parameters {τ _{1, k} , τ _{2, k} , λ ₁ , λ ₂ , β _k , υ _k , W _k , m _k } for the training data set This is called learning. Here, a simple EM algorithm cannot be applied, and a maximum likelihood solution is obtained using a variational Bayes method (Varial Bayes method: VB method).

  A learning algorithm derived using the variational Bayes method is shown below.

  FIG. 4 is a flowchart showing the contents of the learning algorithm.

First, the model learning unit 104 initializes the parameters {τ _{1, k} , τ _{2, k} , λ ₁ , λ ₂ , β _k , υ _k , W _k , m _k } (S11). The initialization method includes a method of initializing at random, using a result of k-means clustering, and the like. Here, a parameter of a learning result of the universal model is set as an initial value of the parameter.

Next, the variation lower limit defined by the following equation is calculated.

ただし、

ここで、

Next, the model learning unit 104 calculates a burden rate r _jk as the VB-E step.

However,

here,

  φ (•) is a digamma function, and I (•) is a parameter that returns 1 if the argument is true and returns 0 if the argument is false.

ここで、

である。 Next, the model learning unit 104 updates the parameters as the VB-M step.

here,

It is.

  Next, the model learning unit 104 determines whether or not the convergence condition is satisfied (S14), and if the change of the variation lower limit (formula (7)) is equal to or less than a predetermined value (S14; Yes), The calculation is terminated, and if it is not less than the predetermined value (S14; No), the process returns to step S12.

In the embodiment of the present invention, more flexible learning is realized by determining the prior distribution for the parameters. However, the hyperparameters β ₀ , υ ₀ , m ₀ , and W ₀ for determining the parameter distribution are still appropriate. A value must be defined.

  The model learning unit 104 uses a universal model and performs probable hyperparameter setting. The hyper parameters of the universal model are set to default values, and learning is performed in advance according to the above steps S11 to S14.

とする。ユニバーサルモデルの学習結果を用いることによって、一律のハイパーパラメータをそれぞれのガウス分布に与えるのではなく、その学習データの全体の傾向を反映した、混合要素毎に異なるハイパーパラメータの設定を行うことができる。 Each of the default hyperparameters

And By using the learning results of the universal model, it is possible to set different hyper parameters for each mixing element, reflecting the overall tendency of the learning data, instead of giving uniform hyper parameters to each Gaussian distribution. .

ここで、変数の右肩にあるｕは、ユニバーサルモデルの学習結果であることを示している。 The hyperparameters of the k-th mixing element are {m ₀ ^k , W ₀ ^k , υ ₀ ^k , β ₀ ^k }
And At this time, the hyper parameter is set as follows using the learning result of the universal model.

Here, u on the right shoulder of the variable indicates a learning result of the universal model.

Furthermore, since the influence of the universal model becomes too large when the number of samples of the universal model is large, a new parameter κ (0 ≦ κ ≦ 1) for controlling the influence of the universal model (prior distribution) may be introduced.

としてもよい。いずれの場合も、κ＝１の場合は、（２７）〜（２８）となり、κ＝０の場合には、デフォルトのハイパーパラメータが設定される。 Furthermore, as a simple method,

It is good. In any case, when κ = 1, (27) to (28), and when κ = 0, default hyperparameters are set.

を用いてもよい。また、さらに単純に

としてもよい。ここで、Ｆ(・)は、原点を通り、１に漸近する単調増加関数となる。 One method of determining κ is to compare the number of samples of the universal model with the number of samples of the model that is currently being learned. If the number of samples of the universal model is large, κ is set to a small value. Value. For example, if N ^u and N are the number of samples of the universal model and the model currently being learned, respectively, as an example of a function that determines κ,

May be used. And even more simply

It is good. Here, F (•) is a monotonically increasing function passing through the origin and asymptotic to 1.

(3) Annotation Estimation Operation FIG. 5 is a flowchart showing an operation example of the image identification information adding device.

  First, the image acquisition unit 100 acquires image information input from the outside via the communication unit 12 as a target of label estimation (S21).

  Next, the image dividing unit 101 divides the image into n regions and generates a partial region (S22).

Next, the feature vector generation unit 102 extracts a plurality of feature amounts from the partial region, and generates feature vectors x ₁ , x ₂ ,..., X _n using these feature amounts as components for each partial region (S23). .

Next, the likelihood calculating means 105 reads the model of each label learned in step S6 from the learning information 113 (S24). Specifically, the model parameters {τ _{1, k} , τ _{2, k} , λ ₁ , λ ₂ , β _k , υ _k , W _k , m _k } are read from the storage unit 11 and developed into a memory (not shown). .

ここで、ｐ(c)は、ラベルｃの事前確率である。これには、学習データ集合における相対頻度を用いる。p(x₁…x_n)は、特徴ベクトル集合の事前分布であるが、ラベルに対しては一定値をとる。したがって、画像Ｉに対するラベルｃの対数尤度は、定数部分を除いて、次式のように表わせる。

式（40）が大きいほど画像Ｉのラベルとして適していると考えられる。式の大きいものから数個を画像Ｉのラベル（アノテーション単語）とする。 Next, the likelihood calculating means 105 calculates a posteriori probability for the feature vector of each partial region (S25). Given a set of feature vectors X = {x ₁ ,... X _n } extracted from the input image I to be predicted, the posterior probability p (c | X) of the label c is _expressed as follows using Bayes' theorem: Is calculated as follows.

Here, p (c) is the prior probability of the label c. For this, the relative frequency in the learning data set is used. p (x ₁ ... x _n ) is a prior distribution of the feature vector set, but takes a constant value for the label. Therefore, the logarithmic likelihood of the label c for the image I can be expressed as the following equation except for the constant part.

It can be considered that the larger the formula (40), the more suitable the label for the image I. Several of the large expressions are used as labels (annotation words) of the image I.

ただし、

ここで、Γ(・)は、ガンマ関数を表わす。 Next, the likelihood calculating means 105 calculates the likelihood of the feature vector x _i of the partial image for a certain label c by the following equation (S26).

However,

Here, Γ (·) represents a gamma function.

  When the likelihood is calculated by the above calculation, the annotation word estimation unit 106 acquires, for example, the top five labels in descending order of the likelihood, and the annotations associated with the labels as the identification information of the image information. A word is assigned (S27).

  Next, the output means 107 outputs the annotation word estimation result to a predetermined output device (display, printer, hard disk, etc.) (not shown) (S28).

[Other embodiments]
The present invention is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the present invention.

  Further, the image identification information providing program 110 used in the above embodiment may be read from a storage medium such as a CD-ROM into a storage unit in the apparatus, or from a server apparatus or the like connected to a network such as the Internet. You may download to the memory | storage part. Moreover, you may implement | achieve part or all of the means 100-107 used by the said embodiment with hardware, such as ASIC.

DESCRIPTION OF SYMBOLS 1 Image identification information provision apparatus 10 Control part 11 Storage part 12 Communication part 100 Image acquisition means 101 Image division means 102 Feature vector generation means 103 Learning data set acquisition means 104 Model learning means 105 Likelihood calculation means 106 Annotation word estimation means 107 Output Means 110 Image identification information adding program 111 Image information 112 Label information 113 Learning information

Claims (3)

Computer
Acquisition means for acquiring image information for each identification information from image information associated with the identification information in advance;
Generating means for generating feature information from the image information acquired by the acquiring means;
A learning result is generated by learning the relationship between the feature information generated by the generating means and the identification information of the image information corresponding to the feature information using a probability model based on a mixture of probability distributions, and the mixture of the probability models Learning to determine a variable of an element generation process using a set of feature information obtained from all image information acquired by the acquisition means regardless of the content of identification information, and to determine a prior distribution for the variable of the probability distribution Means,
Based on the learning result generated by the learning unit, a calculation unit that calculates a posterior distribution of feature information generated by the generation unit from image information to which identification information is added using the probability model for each piece of identification information When,
An image identification information addition program that functions as an estimation unit that estimates identification information associated with image information to which the identification information is to be added based on the posterior distribution.   2. The image identification according to claim 1, wherein the learning unit determines an initial value of the variable of the prior distribution using a set of feature information obtained from all image information regardless of the content of the identification information by the acquisition unit. Information grant program. Acquisition means for acquiring image information for each identification information from image information associated with the identification information in advance;
Generating means for generating feature information from the image information acquired by the acquiring means;
A learning result is generated by learning the relationship between the feature information generated by the generating means and the identification information of the image information corresponding to the feature information using a probability model based on a mixture of probability distributions, and the mixture of the probability models Learning to determine a variable of an element generation process using a set of feature information obtained from all image information acquired by the acquisition means regardless of the content of identification information, and to determine a prior distribution for the variable of the probability distribution Means,
Based on the learning result generated by the learning unit, a calculation unit that calculates a posterior distribution of feature information generated by the generation unit from image information to which identification information is added using the probability model for each piece of identification information When,
An image identification information providing apparatus comprising: estimation means for estimating identification information associated with image information to which the identification information is to be applied based on the posterior distribution.

Images