EP1182588A2

EP1182588A2 - Signal indexing

Info

Publication number: EP1182588A2
Application number: EP01300750A
Authority: EP
Inventors: Yang-Lim Choi; S.Bangalore Manjunath; Peng Wu
Original assignee: Samsung Electronics Co Ltd; University of California; University of California Berkeley; University of California San Diego UCSD
Current assignee: Samsung Electronics Co Ltd; University of California; University of California Berkeley; University of California San Diego UCSD
Priority date: 2000-08-21
Filing date: 2001-01-29
Publication date: 2002-02-27
Also published as: EP1182588A3; CN1339730A; CN1193308C; JP2002074360A

Abstract

Signals are indexed by a modified form of vector approximation files. Before being approximated, the dimensions of the vectors are quantized according a law depending on the distribution of values in the input data set. It is these quantized values that are then approximated.

Description

The present invention relates to a method of processing a plurality of time and/or spatially varying signals, the method comprising determining a feature vector for each signal, generating a signature for each feature vector, said signatures approximating respective feature vectors and being coarser than said vectors, storing said signatures so as to provide an index to the signals to which they relate.
The high dimensionality of typical multimedia data descriptors poses challenging problems in designing effective indexing schemes. Therefore, recently several new indexing structures have been proposed. One of the common assumption made is that the feature vectors are uniformly distributed within a vector space. However, many media descriptors such as image texture descriptors are not uniformly distributed. For example, in the case of a method for using well-known vector approximation (VA) files, which are coarser grained than the feature vectors themselves, the performance of the method depends on the uniform distribution of the feature vectors, and generally the method has the problem that its performance abruptly drops when indexing the feature vector data within vector space having high-dimensionality, in which the feature vectors are not uniformly distributed.
A method according to the present invention is characterised in that said feature vectors are quantized according to a quantization law depending on the distribution of their values for producing said signatures. Each dimension of the feature vectors may be quantized independently in dependence on the distribution of values for that dimension.
Preferably, the method is adaptive and includes generating a new quantization law in dependence on said feature vectors and an additional feature vector for a further time and/or spatially varying signal.
Preferably, each quantization step represents a range of values that is substantially as probable for a feature vector as are the ranges of the others.
Preferably, said quantization law is derived from the feature vector distribution for a first set of signals and signatures are produced using said quantization law for a second set of signals, the second set being a super set of said first set.
Preferably, said distribution is a sum of a plurality of Gaussian distributions approximating the actual distribution of said feature vectors.
An embodiment of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which:-
Figure 1 is a flowchart showing the main steps of an indexing method according to the present invention;
Figure 2 illustrates the case where data joint distribution is still not uniform but agglomerated even though the marginal distribution of the data is uniform in each dimension;
Figure 3A is a histogram showing the distribution of feature vector data within the feature vector data space;
Figure 3B is a graph showing the probability distribution function estimate on the histogram;
Figure 4A is a graph showing feature vector values of the data sets;
Figure 4B is a graph showing the results of calculating a histogram of the data sets of Figure 4A;
Figures 4C, 4D, and 4E are graphs showing the estimated probability distribution functions when the number of elements used for the estimation is 1700, 3400, and 5000, respectively; and
Figures 5A and 5B are graphs showing the comparison of the number of feature vectors visited in first and second filtering steps, using a conventional indexing method and an indexing method of the present invention.
According to the present invention, vector approximation (VA) files are formed adaptively on the basis of statistical distributions of feature vector data within the feature vector data space. That is, since densely distributed cells can deteriorate the performance of indexing, the approximation of the feature vectors is formed adaptively according to statistical features of the data in the present invention.
Referring to Figure 1, in an indexing method according to the present invention, statistical distributions of the feature vector data are measured within the feature vector data space (step 102). Then, marginal distribution is estimated using the statistical distribution (step 104). Next, estimated marginal distributions are divided into a plurality of grids in which a probability of disposing the data in each grid becomes uniform (step 106). The number of grids is determined by the number of bits assigned to the dimension. Then, the feature vector data space is indexed using the divided grids (step 108). The step 108 can be performed on the basis of the indexing method using well known vector approximation (VA) files.
The approximation formed by the above method reduces the possibility of having densely distributed cells. Therefore, the performance of indexing is enhanced.
The fact that the marginal distributions of the data can only capture partial information of high dimensional distributions should be noted. Figure 2 illustrates the case that the data joint distribution is still not uniform but agglomerated even though the marginal distributions of the data are uniform in each dimension. With reference to Figure 2, the marginal distributions of the data are uniform in each dimension within the entire feature vector data space 20. However, if it is considered that a correlation of data is reduced in different dimensions as the properties of the dimensionality, and image/video database increase, it can still be an effective method to estimate the marginal distributions of the data by capturing the statistical properties of the high dimensional data.
Hereinafter, methods of realizing a method of the present invention will be described in greater detail. First, a probability distribution function of data in i-dimension is denoted by p_i(x). By assuming that data of each dimension are independent of each other, the algorithm described hereinafter can be independently applied to each dimension. Also, as described, the uniformity of the data distribution actually has an irregular probability distribution function of data or is not modeled by a well-defined function such as the Gaussian function. In the present invention, a probability distribution function of one-dimensional data is modeled using the Gaussian mixture function in order to endure a change of the data distribution.
First, it is assumed that a probability distribution function of a one-dimensional signal, p(x) is defined as follows:
Here, p(x|j) is defined as follows:
The coefficients P(j) are called the mixing parameter and satisfy the criterion 0 ≤ P(j) ≤ 1 and the following formula.
Thus, in this embodiment, the probability distribution function is defined using a weighted sum of the Gaussain function. Then, the task of estimating probability distribution functions is then converted to be a problem of parameter estimation. Here, when the parameters which must be estimated are j=1,...,M, and l=1,...,N, and v[l] is an assigned data set, Φ_j, which maximizes the following formula, is obtained.
Parameters are obtained using an expectation-maximization (EM) algorithm. According to the algorithm, N predetermined data are used as an input for the estimation, and parameters are estimated iteratively using all of the N assigned data in each iteration.
Let t denote the iteration number, the following equations are used to update the estimated parameters.
However, if the data's distribution has a singular value which can not be grouped with other values as a part of a Gaussian function, crash of estimation can be caused.
When this is the case, to capture this value accurately, we have to let a certain_ approach that value and let the corresponding σ² be converged to 0. To avoid this singularity problem, a very small value is set as a lower bound for an estimated variance.
In order to explain the efficiency of the use of the EM algorithm for estimating parameters of the Gaussian mixture function, a histogram illustrates a distribution of feature vector data within the feature vector data space in Figure 3A, and a graph of a probability distribution function estimated based on the histogram is illustrated in Figure 3B. As shown in Figure 3A, even though the data's distribution is irregular and cannot be modelled by some simple form function, using the Gaussian mixtures as the modelling tool and EM algorithm to estimate the parameters, the data's probability distribution function can be modelled well as shown in Figure 3B.
Parameters can be estimated for N predetermined data using the formulas 5, 6, and 7. In the case of a large database, N is generally only a small portion, compared with the total number of elements. In realistic database applications, an estimation of a prescribed point is required to be updated. For example, there can be a case where a larger portion of data is required to be used for a better estimation. Alternatively, when the database is non-static, statistical characteristics of data change, so that the probability distribution function must be re-estimated. In either cases, memories of the previous estimation are not required to be totally erased. In view of the parameter estimation, when a data set changes, a strategy should concentrate on tracking the change of the estimated probability distribution function. For this objective, an algorithm which can sequentially update the estimation is used.
When an estimated parameter set {P(j) ^N ,µ N / j,(σ 2 / j) ^N using N data v[l] is given, the updated parameter set, when new data v[N+1] coming in, can be calculated as follows.
In the formulas 8 and 9, relations are realized as follows. ( N+1 j )-1 = P(j|ν[N] P(j|ν[N+1]) ( N j )-1+1
In order to evaluate the performance of tracking using an on-line estimation, experimentation is performed on synthetic data set. Feature vector values of data sets are illustrated in Figure 4A. Referring to Figure 4A, the data sets include 5000 elements. In Figure 4B, the results of calculating histograms on the data sets of Figure 4A are illustrated. Each individual element is sequentially added up for the estimation. Then, parameters are calculated using formulas 8, 9, and 10. Next, when a certain number of elements are used for the estimation, a probability distribution function is formed from the estimated parameters.
In Figures 4C, 4D, and 4E, the probability distribution functions are illustrated when the number of elements used for the estimation is 1700, 3400, and 5000, respectively. Referring to Figures 4C, 4D, and 4E, when the distribution of input data changes, it can be shown that the on-line estimation tracks very well. Here, it is necessary to pay attention to the fact that efficiency of the on-line estimation partially depends on the method of choosing the data used as the input.
For example, when the probability distribution function of the data shown in Figure 4A is estimated, the data is chosen in the same order of indexing the data, and then, one estimated probability distribution function is obtained as shown in Figure 4E. That is, ideally, the data must be chosen so as not to place inappropriate emphasis on the data.
The estimated probability distribution function is called
(x). An objective of a nonlinear quantization is to divide the probability distribution function using a plurality of grids to make areas covered by each grid the same. If boundary points are indicated as c[l], the boundary points must satisfy the following criterion.
It is possible to determine boundary points from one pass scan of a probability distribution function estimated using the criterion. For example, all N points are agglomerated to 2^b clusters, so that boundary points of each dimension are determined in the formula 4. Also, not only boundary points can be found very effectively by calculation using the formula 12, but also dependency of distance measure can be avoided.
According to the above method, a probability distribution function is able to be updated. This feature is very important for keeping satisfied indexing for the irregular database. That is, every time when previous estimation doesn't fit with the updated estimation, the approximation needs to be updated. For this reason, a measure is required to decide when to update the approximation based on the change of probability distribution function estimation. Also, thanks to the parallel scheme of utilizing the probability distribution function to construct the approximation, the measure for updating the approximation can be defined in each dimension. When the previous probability distribution function is
(x), and the updated probability distribution function is
(x), a measurement of changes in the probability distribution functions can be defined as follows.
Here, when ρ is bigger than a predetermined threshold value, an approximation of its dimension is updated.
The process of evaluating an image database including 34,698 aerial photograph images, according to the present invention and, by way of contrast, according to a prior art method, will now be described. First, 48-dimensional feature vectors describing texture characteristics of each of the images were extracted using a conventional method for extracting texture feature vectors. A probability distribution function was estimated from an entire data set on the basis of the extracted feature vectors. Figures 5A and 5B are graphs illustrating comparison of the number of feature vectors which are visited in the first step filtering and the second step filtering using the conventional indexing method and the indexing method of the present invention. In Figure 5A, a plot 502 shows the number of feature vectors visited in the first filtering step using an indexing method of the present invention which adaptively forms VA files, and a plot 504 shows the number of feature vectors visited in the first step filtering using the conventional indexing method which uses fixed VA files. Also, the number of feature vectors visited in the first step filtering is indicated by the vertical axis N1.
Also, in Figure 5B, a plot 512 shows the number of feature vectors visited in the second filtering step using an indexing method of the present invention which adaptively forms VA files, and a plot 514 shows the number of feature vectors visited in the second step filtering using the conventional indexing method which uses fixed VA files. Also, the number of feature vectors visited in the second step filtering is indicated by the vertical axis N2. Comparing the plots 502 and 504 with the plots 512 and 514, the number of feature vectors, visited in the first step filtering and the second step filtering using the indexing method of the present invention which adaptively forms VA files, is much bigger than the number of feature vectors, visited in the first step filtering and the second step filtering using the conventional indexing method which uses fixed VA files.
The present invention may be embodied in a database system for signals, e.g. image or audio representing signals such as may be obtained by sensing the physical world. In such a system, a dataset comprising a plurality of signals is processed according to the present invention to produce an index. The database can then be queried by inputting a sample signal, which is processed to derive indexing values using the feature vector quantization used for generating the database index. These indexing values are then compared with the values in the index to find signals having similar characteristics.
The indexing method of the present invention can be written as a program which is performed in a personal computer or a server computer. Program codes and code segments which form the program can be easily derived by computer programmers in the art. Also, the program can be stored on computer readable recording media. Such recording media includes magnetic recording media, optical recording media. Additionally, the program codes may be conveyed in the form of electric or electromagnetic signals.

Claims

A method of processing a plurality of time and/or spatially varying signals, the method comprising determining a feature vector for each signal, generating a signature for each feature vector, said signatures approximating respective feature vectors and being coarser than said vectors, storing said signatures so as to provide an index to the signals to which they relate, characterised in that said feature vectors are quantized according to a quantization law depending on the distribution of their values for producing said signatures.
A method according to claim 1, including generating a new quantization law in dependence on said feature vectors and an additional feature vector for a further time and/or spatially varying signal.
A method according to claim 1 or 2, wherein each quantization step represents a range of values that is substantially as probable for a feature vector as are the ranges of the others.
A method according to claim 1, 2 or 3, wherein said quantization law is derived from the feature vector distribution for a first set of signals and signatures are produced using said quantization law for a second set of signals, the second set being a super set of said first set.
A method according to any preceding claim, wherein said distribution is a sum of a plurality of Gaussian distributions approximating the actual distribution of said feature vectors.
A method for indexing feature vector data space comprising the step of:

(a) indexing feature vector data space by adaptively approximating feature vectors on the basis of statistical distribution of feature vector data in the feature vector data space.
The method of claim 6, wherein the step (a) comprises:

(a-1) measuring the statistical distribution of the feature vector data in the feature vector data space;

(a-2) estimating marginal distribution of the data using the statistical distribution;

(a-3) dividing the estimated marginal distribution into a plurality of grids in which a distribution of disposing the data in each grid becomes uniform; and

(a-4) indexing the feature vector data space using the divided grids.
The method of claim 7, before the step (a-4), further comprising the step of updating the grids on the basis of the previous probability distribution function and the updated probability distribution function when new data enters.
The method of claim 7, wherein the step (a-4) comprises indexing using vector approximation (VA) files.
The method of claim 7, wherein the number of the plurality of grids is determined by the number of bits assigned to the dimension.
The method of claim 7, wherein the step (a-2) comprises:

(a-2-1) defining a probability distribution function using a weighted sum of the predetermined distribution function; and

(a-2-2) obtaining an estimated probability distribution function by estimating predetermined parameters using the probability distribution function defined in the step (a-2-1).
The method of claim 11, wherein the step (a-2-2) comprises obtaining an estimated probability distribution function by estimating predetermined parameters using all N predetermined data every time, wherein N is a positive integer, on the basis of expectation-maximization algorithm using the probability distribution function defined in the step (a-2-1).
The method of claim 11, wherein the predetermined distribution function is the Gaussian function.
The method of claim 11, wherein the step (a-2-1) comprises: assuming that a probability distribution function of the one-dimensional signal, p(x), is
wherein p(x|j) is defined as
and mixing parameter P(j) satisfies the criterions that 0 ≤ P(j) ≤ 1 and
P(j)= 1, a probability distribution function of a one-dimensional signal.
The method of claim 11, wherein the step (a-2-2) comprises: obtaining an estimated probability distribution function by finding Φ_j which maximizes Φ(Φ₁,...,Φ _M )=
p(ν[l]|(Φ₁,...,Φ _M )), wherein parameters to be estimated are j=1,...,M, and l=1,...,N, and v[l] is a given data set.
The method of claim 15, wherein the step (a-2-2) further comprises a step of obtaining estimated parameters according to

and
wherein t is a positive integer representing the number of iteration.
The method of claim 10, wherein the step (a-2-2) further comprises: if new data, v[N+ 1] enters finding a updated parameter set by calculating

and
when a parameter set {P(j) ^N ,µ N / j,(σ 2 / j) ^N , estimated using the N data, v[l], is given.
The method of claim 16, wherein the step (a-2-2) further comprises:

measuring a change of a probability distribution function which is defined as
for each dimension, wherein previous probability distribution function is
(x), and updated probability distribution function is
(x); and

updating approximation for the dimension if ρ is larger than a predetermined threshold value.
The method of claim 7, wherein the step (a-3) comprises: dividing the probability distribution function in order to make areas covered by each grid the same using a plurality of grids determined by boundary points, c[l], which satisfies a criterion,
wherein the estimated probability distribution function is
(x).