JP6601965B2 - Program, apparatus and method for quantizing using search tree - Google Patents

Description

  The present invention relates to a technique for specifying quantization target data for input data. In particular, the present invention can be applied to a technique for searching for reference content similar to query content (content serving as a search key) from a set of reference content (content to be searched).

  In recent years, image recognition and search techniques based on local feature points have attracted attention (for example, see Patent Document 1). As a feature vector extraction algorithm used for object recognition, for example, SIFT (Scale-Invariant Feature Transform) (see, for example, Non-Patent Document 1) or SURF (Speeded Up Robust Features), which is robust to changes in rotation and scale, is used. For example, in the case of SIFT, a set of 128-dimensional feature vectors is extracted from one image. SIFT is a technique for analyzing a characteristic local region using a scale space and describing a feature vector that is invariant to scale change and rotation. On the other hand, in the case of SURF, higher-speed processing is possible than SIFT, and a set of 64-dimensional feature vectors is extracted from one image. While SIFT has a high processing cost and is difficult to perform real-time matching, SURF uses an integral image to speed up the processing.

  In addition, as mobile terminals such as smartphones and tablets have become widespread, further memory saving and faster matching have been required for content search processing. In particular, according to the technical field of image recognition in the use of augmented reality (AR), it is required to search for content at a higher speed than SIFT or SURF in order to perform real-time processing. For this purpose, attention is paid to ORB (for example, see Non-Patent Document 2) and FREEAK (for example, Non-Patent Document 3) which are binary feature vector extraction algorithms. This makes it possible to extract feature vectors at a higher speed than SIFT and SURF, and to make the extracted feature vectors more compact.

  In image recognition based on local feature points, since the processing cost of matching between feature points is high, a framework called BoVW (Bag-of-Visual Words) is widely used (for example, Non-Patent Document 1, 4). In BoVW, a representative vector set called VW (Visual Words) is generated in advance from a large set of training vectors using a clustering algorithm such as k-means. Each feature vector is then quantized to the most similar VW. It is determined that feature vectors quantized to the same VW match, and feature vectors quantized to different VWs are determined not to match. This enables matching that is not based on the number of feature vectors in the database.

  In addition, there is a technique in which content is represented by one feature vector and searched by modeling a training vector with a mixed Bernoulli distribution and deriving a Fisher vector (see, for example, Patent Document 2). According to this technique, the similarity between images can be calculated by the Fisher vector, but the matching result of the feature points cannot be acquired.

  FIG. 1 is an explanatory diagram illustrating a quantization process.

  FIG. 1 shows matching between a binary feature vector derived from query content and reference content and a code book (quantization target data, representative vector) derived from training content. The code book is created in advance from training content, and consists of a set of a plurality of VWs (representative vectors) as quantization target data. VW is also expressed by a binary feature vector. The binary feature vector of the query content and the binary feature vector of the reference content are each linked to the VW having the shortest Hamming distance in the code book. The binary feature vectors associated with the same VW are recognized as matched.

JP-A-2015-201123 JP 2014-146207 A

J. Sivic et al., "Video Google: A Text Retrieval Approach to Object Matching in Videos," in Proc. ICCV, 2003. E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, "ORB: An efficient alternative to SIFT or SURF," in Proc. ICCV, 2011. A. Alahi, R. Ortiz, and P. Vandergheynst, "FREAK: Fast Retina Keypoint," in Proc. CVPR, 2012. D. Galvez-Lopez and J. D. Tardos, "Real-time loop detection with bags of binary words," in Proc. Of IROS, 2011, pp. 51-58. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object retrieval with large vocabularies and fast spatial matching," in Proc. Of CVPR, 2007, pp. 1-8.

According to the technique described in Non-Patent Document 4, the generation of the BoVW codebook uses the k-means algorithm as in the conventional non-binary feature. This k-means algorithm implicitly assumes a mixed Gaussian distribution as a feature vector distribution.
However, binary feature vectors only take values of 0/1. Codebooks generated without considering the distribution of binary feature vectors have a problem that the quantization error is large, and the matching accuracy between feature points ultimately decreases.

Further, according to the technique described in Patent Document 2, a binary feature vector of training content is modeled, but the content is expressed by one feature vector. Therefore, there is a problem that a matching result between local feature points cannot be obtained.
In order to obtain a high-precision search result, the geometric consistency between the query content and the reference content is generally verified using the result of matching between local feature points (for example, non-patented). Reference 5). However, according to the technique described in Patent Document 2, such geometric consistency cannot be verified.

  Here, the inventors of the present application select a plurality of modeled data to be quantized (for example, BoVW in a code book), and specify the most suitable data to be quantized for input data as much as possible. Can't you? I thought. In particular, it is possible to match the optimal data to be quantized as much as possible according to the mixture distribution that is the data to be quantized, without simply matching the data having high similarity between the input data and the data to be quantized. Can't you? I thought.

  Therefore, an object of the present invention is to provide a program, an apparatus, and a method that can match optimal quantization target data to input data according to a mixture distribution that is quantization target data.

According to the present invention, in a program that causes a computer to function to identify one or more nodes i from among a plurality of nodes i (= 1 to N) to be quantized with respect to input data x,
From the root node sequentially, and the search tree storing means mixing elements P _i in the mixture distribution to each node i and stored by the search tree tied,
For the input data x, in order from the root node of the search tree, the node having the larger likelihood P _i (x) based on the mixture distribution P (x) of each node i is searched, and the likelihood P _i (x) is obtained. The computer is caused to function as a quantizing means for specifying nodes i having a posterior probability γ _i (x) based on a predetermined threshold value or higher or a predetermined upper number.

According to another embodiment of the program of the present invention,
The mixture distribution P (x) is P (x) = Σ _{i = 1} ^N w _i P _i (x) based on the mixture ratio w _i of the node i,
It is also preferable to make the computer function such that the posterior probability γ _i (x) is γ _i (x) = w _i P _i (x) / (Σ _{i = 1} ^N w _i P _i (x)).

According to another embodiment of the program of the present invention,
The quantization means searches the search tree with depth priority,
The likelihood P _i (x) in the unsearched node i is actually stored in the priority queue,
It is also preferable to make the computer function so as to search recursively in order from the unsearched node having the highest likelihood P _i (x) stored in the priority queue.

According to another embodiment of the program of the present invention,
The search tree connects _K mixed elements P _{1 to} P _K selected at random (K ≧ 1) as child nodes to the root node, and the unselected mixed elements P _i are connected to P _{1 to} P It is also preferable to further cause the computer to function as search tree creation means for creating a search tree by recursively repeating the association with the node corresponding to _K.

According to another embodiment of the program of the present invention,
Search tree generating means, a mixing element P _i that have not been selected, when attaching the cord to the node corresponding to P ₁ to P _K, P _i and P _j ∈P _1, ···, non between P _K It is also preferable to make the computer function so as to use KL (Kullback Leibler) divergence or JS (Jensen Shannon) divergence as the similarity measure.

According to another embodiment of the program of the present invention,
It is also preferable to cause the computer to function so that the mixed distribution P (x) is a mixed Bernoulli distribution, a mixed Gaussian distribution, a mixed multinomial distribution, a mixed Dirichlet distribution, or a mixed Laplace distribution.

According to another embodiment of the program of the present invention,
Binary feature vector extraction means for extracting a set of binary feature vectors for each of the input training content, reference content and query content;
Model estimation means for calculating a model parameter λ including a mixture ratio w _i for the mixture distribution P (x) for a set of binary feature vectors of training content;
Causing the computer to function as model parameter storage means for storing the model parameter λ,
The quantization means specifies the node i using the model parameter λ for the binary feature vector of the reference content or the query content,
For each binary feature vector of the reference content, transposed index storage means for storing information of the binary feature vector in a list of transposed indexes corresponding to the identifier i of one or more nodes specified by the quantizing means;
For each binary feature vector of the query content, a list of transposed indexes corresponding to the identifier i of one or more nodes specified by the quantization means is searched, and the reference content corresponding to the binary feature vector stored in the list is searched. It is also preferable to search the reference content similar to the query content by accumulating the score, and further causing the computer to function as search means for searching the reference content using the accumulated score as the similarity.

According to another embodiment of the program of the present invention,
When the mixed distribution P (x) is a mixed Bernoulli distribution,
The model parameter λ is the mixture ratio w _i for the mixed Bernoulli distribution at node i and the d (1 ≦ d ≦ D) th parameter μ _id of the mixed Bernoulli distribution at node i,
It is also preferable that the quantization means causes the computer to function so as to specify the mixture distribution P _i corresponding to each node i using the mixture ratio w _i and the parameter μ _id .

According to another embodiment of the program of the present invention,
The model estimation means uses a set of binary feature vectors x _{1 to} x _{T of} training content,
For the E (Expectation) step, estimate the posterior probability γ _t (i) = w _i P _i (x) / (Σ _{i = 1} ^N w _i P _i (x)) of the latent variable i for each binary feature vector x _i And
For the M (Maximization) step, update the mixture ratio w _i and parameter μ _i using the posterior probability γ _t (i),
The parameter group λ of the mixture ratio w _i and the parameter μ _id is calculated by repeating these E step and M step until convergence or a certain number of times. Λ = (w ₁ ,..., W _N and μ ₁₁ ,. .., μ _ND )
It is also preferable to make the computer function.

According to the present invention, in the quantizing device for identifying one or more nodes i among a plurality of nodes i (= 1 to N) to be quantized with respect to input data x,
From the root node sequentially, and the search tree storing means mixing elements P _i in the mixture distribution to each node i and stored by the search tree tied,
For the input data x, in order from the root node of the search tree, the node having the larger likelihood P _i (x) based on the mixture distribution P (x) of each node i is searched, and the likelihood P _i (x) is obtained. And a quantizing unit that identifies nodes i having a posterior probability γ _i (x) based on a predetermined threshold or higher or a predetermined upper number.

According to the present invention, in the quantization method of an apparatus for identifying one or more nodes i from among a plurality of nodes i (= 1 to N) to be quantized with respect to input data x,
The apparatus includes a search tree storage unit that stores a search tree in which mixed elements P _i of a mixed distribution are linked to each node i in order from the root node,
For the input data x, in order from the root node of the search tree, the apparatus searches the node side with the higher likelihood P _i (x) based on the mixture distribution P (x) of each node i, and the likelihood P _i ( The a posteriori probability γ _i (x) based on x) is specified to be a predetermined threshold value or higher or a predetermined number of nodes i.

  According to the program, apparatus, and method of the present invention, optimal quantization target data can be matched to input data according to the mixture distribution that is quantization target data.

It is explanatory drawing showing a quantization process. It is a functional block diagram of the content search device in the present invention. It is a block diagram of the search tree referred by the quantization part of this invention. It is the 1st explanatory view showing the creation step of the search tree of the present invention. It is the 2nd explanatory view showing the creation step of the search tree of the present invention. It is the 1st explanatory view showing the 1st search step in a search tree. It is the 2nd explanatory view showing the 2nd search step in a search tree. It is the 3rd explanatory view showing the 3rd search step in a search tree. It is the 4th explanatory view showing the 4th search step in a search tree. It is the 5th explanatory view showing the 5th search step in a search tree. It is explanatory drawing showing the process of the transposed index referred by the search part of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

An essential part of the present invention is the “quantization function” for the input data. The quantization function is to specify one or more quantization target data from among a plurality of quantization target data for input data x. Here, according to the present invention, the quantization target data of the code book is stored as a search tree in which the mixed elements P _i of the mixed distribution are linked. The search tree is a codebook quantization target data (for example, VW) configured in a tree structure based on a mixed distribution (for example, mixed Bernoulli distribution) that models the distribution of binary feature vectors of training content. By quantizing using this tree structure, it is not necessary to compare the input data with all the data to be quantized, so that it can be quantized at high speed.

According to the quantization function of the present invention, for input data x, the likelihood P _i (x) based on the mixture distribution P (x) of each quantization target data (node) i is large in order from the root node of the search tree. The other node side is searched, and the quantization target data i whose posterior probability γ _i (x) based on the likelihood P _i (x) is equal to or higher than a predetermined threshold or the upper predetermined number is specified. The input data x can be linked to the specified quantization target data i.

  FIG. 2 is a functional configuration diagram of the content search apparatus according to the present invention.

  According to FIG. 2, the search device 1 includes a binary feature vector extraction unit 11, a model estimation unit 12, a model parameter storage unit 13, a search tree creation unit 14, a search tree storage unit 15, and a quantization unit 16. And an inverted index storage unit 17 and a search unit 18. These functional components are realized by executing a program that causes a computer installed in the apparatus to function. The processing flow of these functional components can also be understood as a device quantization method.

  According to FIG. 2, training content, reference content, and query content are input to the search device 1. Here, the content may be, for example, an image. In that case, the search device 1 can create model parameters and a search tree from a large number of training images, and search for a reference image similar to the query image.

According to FIG. 2, the above-described functional configuration unit functions in accordance with each phase. There are three phases.
<Training phase>
A binary feature vector extraction unit 11, a model estimation unit 12,
Model parameter storage unit 13, search tree creation unit 14, search tree storage unit 15
<Index construction phase>
Binary feature vector extraction unit 11, quantization unit 16, transposed index storage unit 17
<Search phase>
Binary feature vector extraction unit 11, quantization unit 16, search unit 18

<Training phase>
According to the training phase, the search device 1 inputs a large number of training contents and extracts a set of binary feature vectors for each training content. Then, a model parameter of the mixed distribution is estimated using a set of binary feature vectors, and a search tree referred to in the quantization process is generated using the model parameter.

<Index construction phase>
According to the index construction phase, the search device 1 inputs a large number of reference contents (contents to be searched) and extracts a set of binary feature vectors for each reference content. Then, the quantization process is executed for each binary feature vector using the model parameter and the search tree. As a result, the specified quantization target data is stored as a transposed index for search.

<Search phase>
According to the search phase, the search device 1 inputs query content (content of search key) and extracts a set of binary feature vectors from the query content. Next, quantization processing is executed for each binary feature vector using the model parameter and the search tree. As a result, the most similar reference content is searched from the identified quantization target data with reference to the transposed index.
The essential functional configuration of the present invention is in the “search phase”, and for the input data x, one or more nodes among a plurality of nodes i (= 1 to N) to be quantized. To identify i.

  Below, it explains in full detail for every function structure part of the search device 1. FIG.

[Binary Feature Vector Extraction Unit 11] <Training / Index Construction / Search Phase>
The binary feature vector extraction unit 11 extracts a set of binary feature vectors for each of the input training content, reference content, and query content.
In the training phase, the set of extracted binary feature vectors is output to the model estimation unit 12.
In the index construction phase and the search phase, the set of extracted binary feature vectors is output to the quantization unit 16.

The binary feature vector extraction unit 11 extracts a set of binary local features by an arbitrary binary feature vector extraction algorithm. According to the present invention, it is preferable to use ORB (Oriented FAST and Rotated BRIEF) as the binary feature vector extraction algorithm. In the case of ORB, a set of 256-bit binary feature vectors is extracted from one content.
ORB introduces rotation invariance into BRIEF (Binary Robust Independent Elementary Features) and performs feature description by binary code in order to execute matching at high speed. According to the ORB, it is possible to maintain an accuracy equal to or higher than that of SIFT or SURF and realize a speed increase of several hundred times.
Of course, it is not limited to the ORB, and any binary feature vector called FRAK (Fast Retina Keypoint), BRISK, or AKAZE can also be used.

<About ORB>
The ORB is composed of two steps of “feature point detection processing” and “feature vector description processing”.

(Feature point detection processing)
According to the feature point detection process in the ORB, FAST (Features from Accelerated Segment Test) is used to detect key points at high speed. In addition, since FAST is not robust to scale changes, an image is converted into a plurality of sizes, and feature points are extracted from images of each size.

  Further, the existing FAST does not have an algorithm for calculating the key point orientation for obtaining rotation invariance. Therefore, ORB adopts Oriented FAST in order to obtain rotational invariance. By describing the features based on the orientation, even if the input image is rotated, the same key point can be detected as the same feature amount. For this purpose, the direction vector of the center of the key point and the center of gravity of the luminance of the patch is used.

(Feature vector description processing)
Next, according to the feature vector description processing in the ORB, a binary feature vector is extracted for each detected feature point using the BRIEF feature vector descriptor. These are obtained from the magnitude relationship of the luminance of two pixels around the feature point.

  BRIEF can execute keypoint feature description by binary code. According to SIFT and SURF, high-dimensional real numbers are used for feature description. However, when a high-dimensional real number is used, there is an increase in memory capacity and similarity calculation. Therefore, by using BREF based on ORB, it is possible to save memory by describing features by binary code, and it is possible to reduce processing costs by using a Hamming distance for similarity calculation.

  According to BRIEF, a binary code is generated from the sign of the luminance difference between two points randomly selected in the patch. The pixels to be selected are randomly selected according to a Gaussian distribution centered on the key point position. Here, the ORB selects pixels using learning in order to perform matching with higher accuracy. The selected pixel position is used for feature description as a binary code having high feature description capability when the bit variance of the pair is large and the correlation of N pairs is low. N pairs are narrowed down using the Greedy algorithm.

[Model estimation unit 12] <Training phase>
The model estimation unit 12 calculates a model parameter λ including the mixture ratio w _i regarding the mixture distribution P (x) for the set of binary feature vectors of the training content.
The mixed distribution P (x) is preferably a mixed Bernoulli distribution, but may be a mixed Gaussian distribution, a mixed multinomial distribution, a mixed Dirichlet distribution, or a mixed Laplace distribution.

  In the following description, it is assumed that the mixed distribution P (x) is a mixed (multivariate) Bernoulli distribution.

When the mixed distribution P (x) is a mixed Bernoulli distribution, the model parameter λ includes the mixing ratio w _i for the mixed Bernoulli distribution at the node i (i-th) and d (1 ≦ d ≦ 1) of the mixed Bernoulli distribution at the node i. D) The second parameter μid. The model estimation unit 12 calculates these model parameters λ.
λ (w ₁ ,..., w _N and μ ₁₁ ,..., μ _ND )

N：混合数 <Calculation of parameters w _i and μ _id based on mixed Bernoulli distribution>
According to the present invention, a model parameter λ obtained by modeling a set of binary feature vectors with a “mixed Bernoulli distribution” is estimated. The Bernoulli distribution is a discrete probability distribution with a probability p of 1 and a probability q = 1−p of 0. If X is a random variable according to Bernoulli distribution, the mean of the random variable X is p, and the variance is pq = p (1-p). The “mixed Bernoulli distribution” expresses a probability p (x _t | λ) that a binary feature vector x _t is generated.

N: Number of mixtures

μ_id：i番目の混合ベルヌーイ分布のd番目のパラメータ
x_t,d：バイナリ特徴ベクトルx_tのd番目のビット
D：バイナリ特徴ベクトルのビット長
p_i（x_t|λ）：バイナリ特徴ベクトルx_tがi番目の混合ベルヌーイ分布から生成
された際に、d番目のビットが1となる確率 Since they are mixed distributions, different mixed Bernoulli distributions from p ₁ to p _N are selected with the respective mixing ratios w _i to generate x _t . The probability that the binary feature vector x _t is generated from the i-th mixed Bernoulli distribution is expressed by the following equation.

μ _id : d-th parameter of i-th mixed Bernoulli distribution
x _{t, d} : d-th bit of binary feature vector x _t
D: Bit length of binary feature vector
p _i (x _t | λ): Binary feature vector x _t is generated from the i-th mixed Bernoulli distribution
The probability that the dth bit will be 1

Specifically, these parameters are estimated from a set of binary feature vectors x _{1 to} x _T of the training content by an iterative process of an EM (Expectation-Maximization) algorithm. The EM algorithm is an estimation method for statistical parameters based on the maximum likelihood method, and is used when the probability model depends on a latent variable that cannot be observed.

γt(i)：t番目の訓練ベクトルがi番目の混合ベルヌーイ分布から生成された
事後確率 In the E (Expectation) step, the likelihood posterior probability γ _t (i) of the model likelihood is estimated for each binary feature vector x _i based on the distribution of the latent variable z _ti .

γt (i): t-th training vector generated from i-th mixed Bernoulli distribution
Posterior probability

In the M (Maximization) step, the mixture ratio w _i and the parameter μ _i are updated in order to maximize the posterior probability γ _t (i) of the likelihood calculated in the E step. The parameter calculated in the M step is used to determine the distribution of latent variables in the next E step.

By repeating these E step and M step until convergence, the parameter group λ of the mixture ratio w _i and the parameter μ _i that maximizes the log likelihood is calculated λ (w ₁ ,..., W _N and μ ₁₁・ ・ ・ ・ ・ ・ μ _ND ）

[Model parameter storage unit 13] <Training phase>
The model parameter storage unit 13 stores the model parameter λ of the mixture distribution P (x). The model parameter storage unit 13 is referred to by the quantization unit 16.

[Search tree creation unit 14] <Training phase>
The search tree creation unit 14 creates a search tree in which a plurality of nodes (quantization target data, representative vectors of VW) are linked to a tree structure in order from the root node.

  FIG. 3 is a configuration diagram of a search tree referred to by the quantization unit of the present invention.

The search tree of the present invention is such that a mixed element P _i of a mixed distribution is linked to each node i in order from the root node. The node number represents the i-th of the mixed element P _i .

A set B of mixed Bernoulli distributions estimated by the model estimation unit 12 is input to the search tree creation unit 14. The element Bi of B is composed of parameters wi and μ _{i1 to} μ _iD . In addition, the following parameters are set as parameters for creating a tree.
Total number of nodes B
Maximum number of leaf nodes SL
Number of branches K

The search tree creation unit 14 connects _K mixed elements P _{1 to} P _K selected at random (K ≧ 1) as child nodes to the root node, and selects the mixed elements P _i that have not been selected as P _A search tree is created by recursively repeating further association with the nodes corresponding to _{1 to} P _K.

The search tree creation step is expressed by the following algorithm, for example.
number of elements in if B ≤ SL then
Create leaf node holding B
else
Select K elements at random from B and make them P ₁ , ~, P _K
Assign the remaining elements of B to the nearest Pi and create clusters C ₁ , ~, C _K
for each cluster C _i do
Create intermediate node using P _i and execute recursively on C _i
end for
end if

FIG. 4 is a first explanatory diagram showing the search tree creation step of the present invention.
FIG. 5 is a second explanatory diagram showing the search tree creation step of the present invention.

The search trees in FIGS. 3 to 5 are represented as the total node number B = 12, the leaf node maximum number SL = 4, and the branch number K = 2.
(S1) According to FIG. 4A, since the total node number B is 12> SL, the root node does not become a leaf node but has a child node.
(S2) According to FIG. 4B, k = 2 children nodes are randomly selected from all nodes B. Here, the nodes i = 3 and i = 7 are selected, and the nodes P ₁ and P ₂ are created.
(S3) according to FIG. 4 (c), the then remaining 10 nodes of B, allocated to nearer one of the mixing elements P ₁ or P _2. This assignment creates a new set of C ₁ and C ₂ . The algorithm is recursively applied to this set.
(S4) According to FIG. 5A, since the number of nodes of C ₁ is 7> SL, C ₁ is not a leaf node. This set C ₁ for, applying a recursively algorithm. On the other hand, since the number of nodes of C ₂ is 3 ≦ SL, C ₂ is a leaf node.
(S5) According to FIG. 5 (b), finally, a search tree with the number of leaf nodes ≦ SL is created.

Further, the search tree creation unit 14 needs to calculate the similarity between the mixed Bernoulli distributions when assigning the element B to Pi. Search tree creation unit 14, the mixing element P _i that have not been selected, when attaching the cord to the node corresponding to P ₁ to P _K, P _i and P _j ∈P _1, ···, between P _K It is also preferable to use KL (Kullback Leibler) divergence or JS (Jensen Shannon) divergence as the dissimilarity measure.
KL divergence is also referred to as relative entropy, and derives the magnitude of the difference between distributions.
The JS divergence is a KL divergence with a target.

The KL divergence of the mixed Bernoulli distribution p _i for p _j is defined as follows:

[Search tree storage unit 15] <Training phase>
The search tree storage unit 15 stores, in order from the root node, a search tree in which a mixed element P _i of a mixed distribution is associated with each node i. The search tree storage unit 15 is referred to by the quantization unit 16.

[Quantization unit 16] <Index construction / search phase>
The quantization unit 16 specifies the node i to be quantized with reference to the model parameter λ and the search tree for the binary feature vector of the reference content or the query content.
The essential function of the present invention is the quantization unit 16 in the search phase, and for the input data x, one or more nodes i among a plurality of nodes i (= 1 to N) to be quantized. Is specified.

The quantization unit 16 searches the input data x in order from the root node of the search tree for the node having the larger likelihood P _i (x) based on the mixture distribution P (x) of each node i, and the likelihood. The number of nodes i whose posterior probability γ _i (x) based on P _i (x) is greater than or equal to a predetermined threshold value or higher is specified.
The mixture distribution P (x) is P (x) = Σ _{i = 1} ^N w _i P _i (x) based on the mixture ratio w _i of the node i.
The posterior probability γ _i (x) is γ _i (x) = w _i P _i (x) / (Σ _{i = 1} ^N w _i P _i (x)).
In the case of the mixed Bernoulli distribution, the mixed distribution P _i corresponding to each node _i is specified using the model parameter λ (mixing ratio w _i , parameter μ _id ).

The quantization unit 16 searches the search tree with depth priority. Here, the likelihood P _i (x) at the unsearched node i is actually stored in the priority queue. And it searches recursively in an order from the unsearched node with the highest likelihood P _i (x) stored in the priority queue.

The posterior probability P _i (x) in which the input data x is generated from the i-th mixed Bernoulli distribution is defined as follows.

  According to the prior art, in order to calculate a mixed element of a mixed Bernoulli distribution with a high posterior probability, it is necessary to calculate a posterior distribution for all the mixed elements. On the other hand, according to the present invention, search processing is not performed more than necessary using a search tree, and high real-time performance can be realized in quantization processing.

これによって、事後確率γ_i(x)が所定閾値以上又は上位所定件数のノード（混合分布の識別子）iを特定することができる。
。 Here, p _i (x) is defined as follows.

Thus, it is possible to specify nodes (mixed distribution identifiers) i whose posterior probabilities γ _i (x) are equal to or higher than a predetermined threshold value or a predetermined upper number.
.

FIG. 6 is a first explanatory diagram showing a first search step in the search tree.
First, for input data x, for nodes i = 3 and i = 7 connected to the root node, i = 3 mixed element likelihood P ₃ (x) and i = 7 mixed element likelihood P ₇ Compare with (x).
P ₃ (x) = 0.050, P ₇ (x) = 0.42
Here, the node i = 3 having the highest likelihood is traced. At this time, it is recorded in the priority queue as follows.
Prioritized queue of unsearched nodes
-> [P ₇ of i = 7 (x) = 0.042 ]
Priority queue for search results
-> [P ₃ (x) = 0.050 for i = 3] [P ₇ (x) = 0.42 for i = 7]

In the “queue with priorities of unsearched nodes”, the likelihood of a node that has not been traced (a node other than the node with the highest likelihood) is recorded. When the leaf node is reached by the depth search, the depth search is recursively performed in order from the node with the highest likelihood recorded in the priority queue of the unsearched node.
In the “queue with priority of search results”, the likelihoods are recorded in descending order for all the traced nodes.

FIG. 7 is a second explanatory diagram illustrating the second search step in the search tree.
Next, for input data x, for nodes i = 5 and i = 11 connected to node i = 3, i = 5 mixed element likelihood P ₅ (x) and i = 11 mixed element likelihood. Compare the degree P ₁₁ (x).
P ₅ (x) = 0.037, P ₇ (x) = 0.061
Here, the node i = 7 having the highest likelihood is traced. At this time, it is recorded in the priority queue as follows in descending order of likelihood.
Prioritized queue of unsearched nodes
-> [Likelihood P ₇ (x) = 0.042 for i = 7] [likelihood P ₅ (x) = 0.037 for i = 5]
Priority queue for search results
-> [P ₁₁ (x) = 0.61 for i = 11] [P ₃ (x) = 0.050 for i = 3]
[Likelihood P ₇ (x) = 0.042 for i = 7] [likelihood P ₅ (x) = 0.037 for i = 5]

FIG. 8 is a third explanatory diagram illustrating a third search step in the search tree.
Next, for input data x, for nodes i = 9 and i = 10 connected to node i = 11, the likelihood P ₉ (x) of the mixed element of i = 9 and the likelihood of the mixed element of i = 10 Compare the degree P ₁₀ (x).
P ₉ (x) = 0.070, P ₁₀ (x) = 0.051
Here, the node i = 9 having the highest likelihood is traced. At this time, it is recorded in the priority queue as follows in descending order of likelihood.
Prioritized queue of unsearched nodes
-> [Likelihood P ₇ (x) = 0.042 for i = 7] [likelihood P ₅ (x) = 0.037 for i = 5]
Priority queue for search results
-> [P ₉ (x) = 0.070 for i = 9] [P ₁₁ (x) = 0.61 for i = 11]
[P ₁₀ (x) = 0.51 for i = 10] [P ₃ (x) = 0.050 for i = 3]
[Likelihood P ₇ (x) = 0.042 for i = 7] [likelihood P ₅ (x) = 0.037 for i = 5]

FIG. 9 is a fourth explanatory diagram illustrating a fourth search step in the search tree.
The search is completed to the end of nodes i = 9 and i = 10 by the depth search. In this case, the depth search is recursively performed on the node with the highest likelihood in the queue with priorities of the unsearched nodes.
Next, the likelihood P ₇ (x) of the mixed element of the node i = 7 is searched for the input data x. At this time, it is recorded in the priority queue as follows in descending order of likelihood.
Prioritized queue of unsearched nodes
-> [Likelihood P ₅ (x) = 0.037 for i = 5]
Priority queue for search results
-> [P ₉ (x) = 0.070 for i = 9] [P ₁₁ (x) = 0.61 for i = 11]
[P ₁₀ (x) = 0.51 for i = 10] [P ₃ (x) = 0.050 for i = 3]
[Likelihood P ₇ (x) = 0.042 for i = 7] [likelihood P ₅ (x) = 0.037 for i = 5]

FIG. 10 is a fifth explanatory diagram illustrating the fifth search step in the search tree.
Next, for input data x, for nodes i = 2, i = 6, and i = 8 connected to node i = 7, i = 2 mixed element likelihood P ₂ (x) and i = 6 The likelihood P ₆ (x) of the mixed element is compared with the likelihood P ₈ (x) of the mixed element of i = 8.
P ₂ (x) = 0.085, P ₆ (x) = 0.048, P ₈ (x) = 0.028
At this time, it is recorded in the priority queue as follows in descending order of likelihood.
Prioritized queue of unsearched nodes
-> [Likelihood P ₅ (x) = 0.037 for i = 5]
Priority queue for search results
-> [P ₂ (x) = 0.85 for i = 2] [P ₉ (x) = 0.070 for i = 9] [P ₁₁ (x) = 0.61 for i = 11]
[P ₁₀ (x) = 0.51 for i = 10] [P ₃ (x) = 0.050 for i = 3] [P ₆ (x) = 0.48 for i = 6]
[Likelihood P ₇ (x) = 0.042 for i = 7] [P ₅ (x) = 0.037 for i = 5]
[P ₈ (x) = 0.028 for i = 8]

When a predetermined number of likelihoods are recorded in the “search result priority queue” (when a predetermined number of nodes are searched), the search is terminated. Then, the posterior probability p _i (x) = w _i P _i (x) is calculated based on the likelihood P _i (x) recorded in the “priority queue of search results”, and the one with the higher posterior probability is calculated. The above node i (data to be quantized) is output as a result of quantization.

[Transposed index storage unit 17] <Index construction phase>
The transposed index storage unit 17 stores binary feature vector information in a transposed index list corresponding to the identifier i of one or more nodes specified by the quantizing unit 16 for each binary feature vector of the reference content. is there.

  FIG. 11 is an explanatory diagram showing the processing of the transposed index referred to by the search unit of the present invention.

  The transposed index holds a list for each identifier i of the mixed distribution. Each list stores information about the binary feature vector quantized to the identifier i. As information to be stored, there is a content ID (identifier). In addition, binary feature coordinates, binary feature scales, binary feature angles, binary feature vectors, and the like may be stored. For quantization, if multiple mixed Bernoulli distribution identifiers are available, information about binary features is registered in all the lists.

[Search unit 18] <Search phase>
The search unit 18 searches the list of transposed indexes corresponding to the identifier i of one or more nodes specified by the quantization unit 16 for each binary feature vector of the query content. Then, the score is accumulated for the reference content corresponding to the binary feature vector stored in the list, and the reference content is searched using the accumulated score as the similarity. Specifically, the score may be voted for each reference content (ID).

  In this case, the distance between the binary feature vector of the query content and the binary feature of the reference content can be calculated, and the score can be calculated according to the distance. For example, if the square distance between the binary feature vector of the query content and the binary feature of the reference content is d, exp (-d / a) can be used as the score. a means an adjustable parameter. Further, the score may be corrected based on the appearance frequency of each quantized identifier based on IDF (Inverse Document Frequency).

  As described above in detail, according to the program, apparatus, and method of the present invention, optimal quantization target data can be matched to input data according to the mixture distribution that is quantization target data. .

According to the present invention, a query book and a reference content are created by creating a codebook including data to be quantized in which binary feature vectors of training content are modeled by a mixed distribution and configuring the data to be quantized in a search tree. Can be performed at high speed and with high accuracy. This takes into account the independence between the bits of the binary feature vector. In particular, according to the present invention, similarity calculation and feature point matching can be executed within the framework of BoVW in the prior art.
Further, when the content is an image, it is possible to suppress the score for the incorrect image pair while maintaining the score for the correct image pair for geometric verification between images. This is useful for recognizing an object with high accuracy.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Search apparatus 11 Binary feature vector extraction part 12 Model estimation part 13 Model parameter memory | storage part 14 Search tree creation part 15 Search tree memory | storage part 16 Quantization part 17 Transposition index memory | storage part 18 Search part

Claims (11)

In a program for causing a computer to function to identify one or more nodes i from among a plurality of nodes i (= 1 to N) to be quantized with respect to input data x,
From the root node sequentially, and the search tree storing means mixing elements P _i in the mixture distribution to each node i and stored by the search tree tied,
For the input data x, in order from the root node of the search tree, the node having the larger likelihood P _i (x) based on the mixture distribution P (x) of each node i is searched, and the likelihood P _i (x) A program that causes a computer to function as a quantizing unit that specifies a number of nodes i having a posterior probability γ _i (x) that is greater than or equal to a predetermined threshold value or a higher predetermined number. The mixture distribution P (x) is P (x) = Σ _{i = 1} ^N w _i P _i (x) based on the mixture ratio w _i of the node i,
The computer is made to function so that the posterior probability γ _i (x) is γ _i (x) = w _i P _i (x) / (Σ _{i = 1} ^N w _i P _i (x)). The program according to claim 1. The quantization means searches the search tree with depth priority,
The likelihood P _i (x) in the unsearched node i is actually stored in the priority queue,
3. The computer according to claim 1, wherein the computer is functioned to search recursively in order from an unsearched node having the highest likelihood P _i (x) stored in the priority queue. program. The search tree connects _K mixed elements P _{1 to} P _K selected at random (K ≧ 1) as child nodes to the root node, and the unselected mixed elements P _i are connected to P ₁ to P ₁ . 4. The computer further functions as search tree creation means for creating the search tree by recursively repeating further association with a node corresponding to P _K. The program described in. The search tree generating means, a mixing element P _i to the not selected, when attaching the cord to the node corresponding to P ₁ to P _K, between P _i and _{P j ∈P 1, ···, P} K 5. The program according to claim 4, wherein the computer is caused to function so as to use KL (Kullback Leibler) divergence or JS (Jensen Shannon) divergence as a measure of dissimilarity. 6. The computer according to claim 1, wherein the mixed distribution P (x) is a mixed Bernoulli distribution, a mixed Gaussian distribution, a mixed multinomial distribution, a mixed Dirichlet distribution, or a mixed Laplace distribution. The program according to item 1. Binary feature vector extraction means for extracting a set of binary feature vectors for each of the input training content, reference content and query content;
Model estimation means for calculating a model parameter λ including a mixture ratio w _i for the mixture distribution P (x) for a set of binary feature vectors of training content;
Causing a computer to function as model parameter storage means for storing the model parameter λ,
The quantization means specifies a node i using the model parameter λ for a binary feature vector of reference content or query content,
For each binary feature vector of the reference content, transposed index storage means for storing information of the binary feature vector in a list of transposed indexes corresponding to an identifier i of one or more nodes specified by the quantizing means;
For each binary feature vector of the query content, a list of transposed indexes corresponding to the identifier i of one or more nodes specified by the quantization means is searched, and reference content corresponding to the binary feature vector stored in the list 7. The computer according to claim 1, further comprising a computer functioning as search means for searching for reference content using the accumulated score as a similarity, and searching for reference content similar to query content. The program according to any one of the above items. When the mixed distribution P (x) is a mixed Bernoulli distribution,
The model parameter λ is a mixture ratio w _i related to the mixed Bernoulli distribution of the node i and a d (1 ≦ d ≦ D) th parameter μ _id of the mixed Bernoulli distribution of the node i,
8. The program according to claim 7, wherein the quantization means causes the computer to specify a mixture distribution P _i corresponding to each node i using the mixture ratio w _i and the parameter μ _id. . The model estimation means includes a set of binary feature vectors x _{1 to} x _{T of} training content,
For the E (Expectation) step, estimate the posterior probability γ _t (i) = w _i P _i (x) / (Σ _{i = 1} ^N w _i P _i (x)) of the latent variable i for each binary feature vector x _i And
For the M (Maximization) step, the mixture ratio w _i and the parameter μ _i are updated using the posterior probability γ _t (i),
By repeating these E and M steps until convergence or a certain number of times, the parameter group λ of the mixture ratio w _i and the parameter μ _id is calculated. Λ = (w ₁ ,..., W _N and μ ₁₁ ,. .., μ _ND )
The program according to claim 8, wherein the computer functions as described above. In a quantizing device that identifies one or more nodes i among a plurality of nodes i (= 1 to N) to be quantized with respect to input data x,
From the root node sequentially, and the search tree storing means mixing elements P _i in the mixture distribution to each node i and stored by the search tree tied,
For the input data x, in order from the root node of the search tree, a node having a larger likelihood P _i (x) based on the mixture distribution P (x) of each node i is searched, and the likelihood P _i (x Quantization means for identifying the number of nodes i whose posterior probability γ _i (x) based on (1) is equal to or higher than a predetermined threshold value or the upper predetermined number. In the quantization method of an apparatus for identifying one or more nodes i from among a plurality of nodes i (= 1 to N) to be quantized with respect to input data x,
The apparatus includes, in order from the root node, a search tree storage unit that stores a search tree in which mixed elements P _i of a mixed distribution are linked to each node i,
The apparatus searches the input data x from the root node of the search tree in order from the node side with the higher likelihood P _i (x) based on the mixture distribution P (x) of each node i, and the likelihood A method for quantizing an apparatus, characterized by identifying a number of nodes i whose posterior probability γ _i (x) based on P _i (x) is greater than or equal to a predetermined threshold value or an upper predetermined number.

Images