JP2015049574A - Index generation device and retrieval device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an index technology capable of calculating an index value indicating a distance between original data with index data.SOLUTION: An index generation device 100 includes: a data acquisition unit 101 acquiring high dimension data; a coefficient acquisition unit 102 acquiring transformation coefficients of the number of dimensions of the high dimension data; a transformation unit 103 mapping the high dimension data only into a one-dimensional space with a plurality of transformation coefficients acquired by the coefficient acquisition unit 102; and an index generation unit 104 generating an index contained as index data in a state where one-dimensional data obtained by the transformation unit 103 is aligned in an ascending or descending order and having a hierarchical structure.

Description

  The present invention relates to a data indexing technique.

  Currently, various similar search methods have been proposed. Such similarity search is often performed on multidimensional data such as image feature data or high-dimensional data.

  For example, Patent Documents 1, 3, and 6 describe techniques related to a similar image search method. Patent Document 2 describes a method of searching for similar data using a database in which a link for tracing from one data to the other data is set between the data. Patent Document 5 describes a method of hierarchically classifying an arbitrary image set. Patent Document 7 describes a technique for searching a feature vector similar to a query feature vector from a set of high-dimensional feature vectors. In Patent Literature 9, each learning pattern is classified into a bucket corresponding to a hash value using a hash function, and a learning pattern that is most similar to the input pattern among the learning patterns belonging to the bucket corresponding to the hash value of the input pattern. A technique for searching is described. Patent Document 10 describes a data matching method that specifies desired conditions and extracts desired data for multidimensional data that can represent a plurality of feature quantities as vectors. Hereinafter, “high dimension” and “multidimensional” are used without any particular distinction.

  In such a similarity search, the similarity between target data is usually calculated using a similarity function or the like. It can be determined that the higher the similarity is, the more similar the target data is. Alternatively, the distance between the target data is calculated using a distance function or the like. It can be determined that the smaller the distance, the more similar the target data. For example, image feature data is represented by a multidimensional numerical vector, and the similarity between the feature data to be compared is calculated by a similarity function. Patent Document 4 calculates similarity with other feature values for all feature values in the database, and stores ID information for the upper f (x) items in descending order of similarity. A technique for searching for similar feature amounts by searching the stored contents is described.

  In addition, an index (index) is constructed with respect to the target data, and a similar search is performed using this index, thereby speeding up the search. An R-Tree is known as an index generation method for multidimensional data (see Non-Patent Document 1). Patent Document 8 describes a method of dividing a feature vector space into a plurality of approximate regions and generating a hierarchical index tree according to the density of each approximate region.

  There is also a technique for mapping multidimensional data from an original data space to a one-dimensional space and constructing an index using the mapped one-dimensional data. According to this method, it is possible to speed up the search by performing a similar search using one-dimensional data. A technique for mapping multidimensional data to a one-dimensional space is called a spatial filling curve. As the space filling curve, a Z curve (Z-curve or Z-order) (see Non-Patent Document 2), a Hilbert curve (see Non-Patent Document 3), and the like are known.

Japanese Patent No. 4545641 JP 2011-090352 A JP 2012-079186 A JP 2000-035965 A Japanese Patent Laid-Open No. 2001-160057 Japanese Patent No. 4906900 JP 2011-257970 A JP 2002-163272 A JP 2009-020769A JP 2004-046612 A

Antonin Guttman, “R-Trees: A Dynamic Index Structure for Spatial Searching”, SIGMOD Conference, 1984, pp. 47-57 G. M. Morton, "A Computer Oriented Geodetic Data Base and a New Technique in File Sequencing", Technical report, IBM, Ottawa, Canada, 1966 David Hilbert, `` Ueber die stetige Abbildung einer Line auf ein Flachenstuck '', Mathematische Annalen Volume 38, Issue 3, pp 459-460, 1891

  The similarity search method as described above uses the index data only for accessing the original data when calculating the distance (similarity) in the search process, leaving room for further speed-up of the search process. . In other words, if the index data can be used instead of the original data and an index value indicating the distance between the data can be calculated, the search process can be further speeded up.

  A method using a spatial index such as an R-tree uses a minimum bounding rectangle (MBR) to hierarchically surround the original data and construct a tree structure index. Then, pruning is performed at the time of search based on the index, thereby reducing the number of accesses to the original data and, as a result, speeding up the similar search. Such a spatial index technique does not assume that index data is used to calculate the distance between data. Therefore, in a similar search process using the spatial index, the distance between data is calculated using the index data. I can't.

  On the other hand, according to the method using the space filling curve, since the generated index data indicates the order in the one-dimensional space, the order of the order is determined using the index data, and this determination result is determined between the data. It can be used as a distance. However, according to this method, since the distances between adjacent index data are all equal, it is difficult to distinguish the magnitudes of the distances between the data, and the similarity search efficiency is lowered.

  The present invention has been made in view of the above-described circumstances, and provides an index technique that makes it possible to calculate an index value indicating a distance between original data using index data.

  Each aspect of the present invention employs the following configurations in order to solve the above-described problems.

  The first aspect relates to an index generation device. An index generation device according to a first aspect includes a data acquisition unit that acquires high-dimensional data, a coefficient acquisition unit that acquires conversion coefficients corresponding to the number of dimensions of the high-dimensional data that cannot be mutually reducible, and a coefficient acquisition unit As the index data in a state where the conversion unit uniquely mapping the high-dimensional data to the one-dimensional space using the plurality of conversion coefficients acquired in step 1 and the one-dimensional data obtained by the conversion unit are arranged in ascending or descending order And an index generation unit that generates an index having a hierarchical structure.

  The second aspect relates to an index generation method executed by at least one computer. The index generation method according to the second aspect acquires high-dimensional data, acquires conversion coefficients for the number of dimensions of the high-dimensional data that cannot be mutually reducible, and uses the acquired plurality of conversion coefficients, The high-dimensional data is uniquely mapped to a one-dimensional space, and the one-dimensional data obtained by the mapping is included as index data in an ascending or descending order, and an index having a hierarchical structure is generated.

  The third aspect relates to a search apparatus that uses an index generated by the index generation apparatus according to the first aspect. The search device according to the third aspect uses a query acquisition unit that acquires search target data having the same number of dimensions as the high-dimensional data, and a plurality of conversion coefficients that are the same as the plurality of conversion coefficients acquired by the coefficient acquisition unit. When the similarity between the search target conversion unit that uniquely maps the search target data to the one-dimensional space and the high-dimensional data and the search target data is evaluated by the same method as the conversion unit, the search target A distance calculation unit that calculates a distance between the search target one-dimensional data obtained by the conversion unit and the one-dimensional data included in the index as index data.

  The fourth aspect relates to a search method using an index generated by the index generation method according to the second aspect. The search method according to the fourth aspect is the same as the mapping included in the index generation method according to the second aspect using the plurality of transform coefficients, acquiring search target data having the same number of dimensions as the high-dimensional data. When the search target data is uniquely mapped to the one-dimensional space by the technique and the similarity between the high-dimensional data and the search target data is evaluated, the search target one-dimensional data obtained by mapping the search target data Calculating a distance between the one-dimensional data included in the index.

  Another aspect of the present invention may be a program that causes at least one computer to execute the method according to the second aspect or the fourth aspect, or a computer-readable recording that records such a program. It may be a medium. This recording medium includes a non-transitory tangible medium.

  According to each said aspect, the index technique which makes it possible to calculate the index value which shows the distance between the original data using index data can be provided.

It is a figure which shows notionally the process structural example of the index production | generation apparatus in 1st Embodiment. It is a flowchart which shows the operation example of the index production | generation apparatus in 1st Embodiment. It is a figure which shows notionally the process structural example of the search device in 1st Embodiment. It is a flowchart which shows the operation example of the search device in 1st Embodiment. It is a figure which shows notionally the hardware structural example of the high-dimensional data search apparatus in 2nd Embodiment. It is a figure which shows notionally the process structural example of the high-dimensional data search device in 2nd Embodiment. It is a flowchart which shows the operation example regarding index generation of the high-dimensional data search device in 2nd Embodiment. It is a flowchart which shows the operation example regarding the search method regarding the range query (Range Query) of the high-dimensional data search device in 2nd Embodiment. It is a figure which shows notionally the process structural example of the high-dimensional data search device in 3rd Embodiment. It is a flowchart which shows the operation example of k nearest neighbor search of the high-dimensional data search device in 3rd Embodiment.

  Embodiments of the present invention will be described below. In addition, each embodiment given below is an illustration, respectively, and this invention is not limited to the structure of each following embodiment.

[First Embodiment]
First, as a first embodiment, an index generation device, an index generation method, a search device using an index generated by the index generation device, and a search method using an index generated by the index generation method will be described.

  FIG. 1 is a diagram conceptually illustrating a processing configuration example of the index generation device 100 according to the first embodiment. The index generation device 100 according to the first embodiment includes a data acquisition unit 101 that acquires high-dimensional data, a coefficient acquisition unit 102 that acquires conversion coefficients corresponding to the number of dimensions of the high-dimensional data that cannot be mutually reduced, and a coefficient Using a plurality of conversion coefficients acquired by the acquisition unit 102, the conversion unit 103 that uniquely maps the high-dimensional data to a one-dimensional space, and the one-dimensional data obtained by the conversion unit 103 are arranged in ascending or descending order. An index generation unit 104 that generates an index having a hierarchical structure that is included as index data in a state.

  The index generation device 100 has, for example, the same hardware configuration as the high-dimensional data search device 1 in a detailed embodiment (second embodiment and later) described later. Similar to the high-dimensional data search device 1, each processing unit described above is realized by processing a program. The hardware configuration of the index generation device 100 is not limited.

  Next, the index generation method in the first embodiment will be described with reference to FIG. FIG. 2 is a flowchart illustrating an operation example of the index generation device 100 according to the first embodiment. In the following description, the index generation device 100 is the execution subject of the index generation method, but each of the processing units included in the index generation device 100 may be the execution subject.

  The index generation method in the first embodiment is executed by at least one computer such as the index generation apparatus 100. The index generation method acquires high-dimensional data (S21), acquires conversion coefficients corresponding to the number of dimensions of the high-dimensional data that cannot be mutually reduced (S22), and a plurality of conversions acquired in (S22). Using the coefficient, the high-dimensional data is uniquely mapped to the one-dimensional space (S23), and the one-dimensional data obtained by the mapping of (S23) is included as index data in a state of being arranged in ascending or descending order. An index having a structure is generated (S24).

  In the present embodiment, high-dimensional data to be indexed is acquired, and conversion coefficients corresponding to the number of dimensions of the high-dimensional data are acquired. The data type of the acquired high-dimensional data is not limited. Furthermore, as described above, high-dimensional data means multi-dimensional data and is not distinguished from multi-dimensional data or multi-dimensional data. Further, the obtained plurality of conversion coefficients have a characteristic that they cannot be mutually reduced. Here, mutually inducible means that no further simplifications can be made. Specifically, when all transform coefficients are integers, it means that all pairs of the transform coefficients do not have a common divisor other than 1, and when the transform coefficients include decimal transform coefficients, Means that all pairs of are not divisible by natural numbers.

  The high-dimensional data is uniquely mapped to a one-dimensional space using a plurality of transform coefficients that are not mutually reducible. The only mapping means that a plurality of high-dimensional data are not mapped to the same point (one one-dimensional data) in the one-dimensional space. This only mapping can be realized by using a plurality of transform coefficients that cannot be mutually contracted. The present embodiment does not limit the mapping method itself as long as the mapping uses a plurality of transform coefficients that cannot be mutually reduced.

  In the present embodiment, an index including one-dimensional data converted from high-dimensional data by the mapping as index data is generated. In the generated index, the index data is hierarchically managed in a state of being arranged in ascending order or descending order. For the generation of this index, a known hierarchical index called a B-tree (B + tree, B * tree, etc.) can be used. This index data may also be used as a pointer to the original high-dimensional data.

  As described above, according to the present embodiment, each index data corresponding to each high-dimensional data becomes a unique (unique) real number, and therefore, the distance (similarity) between the high-dimensional data using each index data. Index value indicating (degree) can be calculated. That is, the distance between index data can be an index value indicating the distance between corresponding high-dimensional data. Here, the index value means a value that can be an index of the distance even if it does not completely match the value of the distance between the high-dimensional data. In addition, according to the present embodiment, by using the one-dimensional data obtained by uniquely mapping the original high-dimensional data on the one-dimensional space as index data, the information that the original high-dimensional data has is not lost. The complexity of the index space can be greatly reduced.

  Here, the technical idea embodied in the index generation apparatus 100 and the index generation method described above will be described, and a specific example related to mapping processing that cannot be mutually reduced and high-dimensional data mapping to a one-dimensional space. Illustrate.

  The present inventor has focused on the basic principle of prime factorization in number theory, and has come up with the idea of adopting the reverse of this basic principle, that is, the reverse of prime factorization, for generating index data for similarity search. Prime factorization is the representation of any positive integer (natural number) in the form of a product of prime numbers. The “basic principle of prime factorization” means a special property in the conversion process of prime factorization. Prime factorization has the following properties.

  For any positive integer, the prime factorization is uniquely determined. This property is also expressed as uniqueness of prime factorization.

For example, when a positive integer 288 is factored, the result is as follows.
288 = 2 × 2 × 2 × 2 × 2 × 3 × 3 = 2 ⁵ × 3 ²
That is, the positive integer 288 is uniquely factored into “2 × 2 × 2 × 2 × 2 × 3 × 3”. Such a continuous product can also be expressed as a product of a power of a prime number and a power of another prime number. That is, the positive integer 288 can also be expressed as a product of the prime number 2 5 and the prime number 3 square.

Due to the nature of prime factorization, the inverse operation of prime factorization is also unique. Therefore, a unique positive integer is always obtained by selecting a prime number in advance and giving a corresponding power to each selected prime number.
For example, if prime number 2 and prime number 3 are selected, 5 is given to the power of prime 2 and 2 is given to the power of prime 3, and the inverse operation of prime factorization is performed, positive integer 288 is uniquely calculated.
2 ⁵ × 3 ² = 2 × 2 × 2 × 2 × 2 × 3 × 3 = 288

  The present inventor changed the viewpoint of prime factorization and considered the following while taking into account the uniqueness of the inverse operation of prime factorization.

  First, the number of prime numbers selected as described above is defined as the number of dimensions in the data space of the similarity search process. As a result, prime numbers corresponding to the number of dimensions of the data space are selected in advance so that the number of dimensions of the data space becomes the number of prime numbers to be selected. In the above example, since two prime numbers 2 and 3 are selected, the number of dimensions in the data space is 2.

  Further, the power given to each selected prime number is defined as an element value of each dimension forming high-dimensional data on the data space or a numerical value obtained by normalizing the element value to a natural number. Thereby, when the element value of each dimension of the target high-dimensional data is a natural number, the element value is given as a power to each selected prime number. When the element value of each dimension of the target high-dimensional data is not a natural number, a numerical value obtained by normalizing the element value to the natural number is given as a power to each selected prime number. In the above example, the target high-dimensional data is normalized as two-dimensional vector data having the powers 5 and 2 given to the prime numbers 2 and 3 as element values in each dimension, or the element values in each dimension. It can be considered that the numerical value obtained in this way is two-dimensional vector data having a power of 5 and 2.

  Here, if the inverse operation of prime factorization as described above is regarded as a specific conversion function, the inverse operation of prime factorization can be regarded as a kind of transformation (or mapping). Then, when this idea is combined with the above definition, the above example corresponds to converting the point (5, 2) in the two-dimensional space into a one-dimensional point (288). Because of the uniqueness of prime factorization and its inverse, this transformation is also unique.

  From such considerations, the present inventor stated that “the inverse operation of prime factorization can be used to uniquely convert vector data in an arbitrary d-dimensional space into a single positive integer in a one-dimensional space. I found what I can do "(hereinafter referred to as the first lemma). Hereinafter, a specific example using the first lemma will be described as a first embodiment.

  In the index generation device 100 according to the first embodiment, the coefficient acquisition unit 102 acquires prime numbers corresponding to the number of dimensions of the high-dimensional data as a plurality of transform coefficients. The conversion unit 103 normalizes the high-dimensional data acquired by the data acquisition unit 101 to a natural number, and uses the element data of each dimension forming the normalized high-dimensional data as a power, and the coefficient acquisition unit 102 Calculate the product of the values obtained by raising each acquired conversion coefficient to the power. The index generation unit 104 generates an index in which the one-dimensional data calculated in this way by the conversion unit 103 is included as index data.

  Hereinafter, the above-described processing of the index generation device 100 according to the first embodiment will be described in more detail with reference to FIG.

The index generation device 100 acquires d-dimensional data v to be indexed (S21). d is an integer of 2 or more. The data v is an arbitrary point on the d-dimensional space and is expressed as follows.
v (x ₁ , x ₂ ,..., x _d )

Here, each dimension element data (x ₁ , x ₂ ,..., X _d ) is a natural number. However, the element data of each dimension of the acquired d-dimensional data v may not be a natural number. In this case, the index generation device 100 may normalize the d-dimensional data v to a natural number. Specifically, the index generation device 100 converts the element data of each dimension of the d-dimensional data v into a natural number so that it can be restored to the original data.

The index generating apparatus 100 appropriately selects d prime numbers (S22). The index generation device 100 may hold a prime table in advance. Selected _{prime, p 1, p 2, ···} , is denoted as _{p d.}

The index generating apparatus 100 calculates the product of the values obtained by raising each of the selected prime numbers to the power using the element data of each dimension as the power (S23). This process is expressed by the following (formula 1).

The above (Equation 1) indicates that the data N ^d in the d-dimensional space that also has a natural number as element data is converted into a one-dimensional natural number N using the conversion function f. Here, the conversion function f can be expressed by the following (Formula 2). In (Formula 2), p and v are as described above. u represents a value after conversion into one dimension (after mapping).

  However, in the method of the first embodiment, the value (u) of the converted one-dimensional data increases rapidly as the dimension number d increases. Therefore, according to the first embodiment, the required performance of the computer (index generation device 100) to be executed is increased. Therefore, the present inventor made the following consideration on the problem of the first embodiment and further developed the first lemma.

In the above example of the inverse operation of prime factorization, the logarithm with the product of prime numbers 2 and 3 as the base is taken for both sides. This operation is expressed as follows.

Here, when α ₁ and α ₂ are defined as follows and α ₁ and α ₂ are used, the above (Equation 3) is expressed as the following (Equation 4). The right side of the following (Formula 4) is a real number. Further, α ₁ and α ₂ are constants and depend only on the selected prime number, and can be calculated in advance.

Thus, the conversion of the point (5, 2) in the two-dimensional space into the one-dimensional value (288) using the prime numbers 2 and 3 shown in the above example is shown in (Equation 4). Replaced with (Equation 4) indicates a linear transformation using constants α ₁ and α ₂ as coefficients, and the constants α ₁ and α ₂ are linear transformation coefficients obtained by logarithmic calculation with the product of prime numbers as the base. The underlying essence of this linear transformation is that the transformation coefficients α ₁ and α ₂ are not mutually reducible.

  From such considerations, the present inventor has made it possible to convert vector data on an arbitrary d-dimensional space into a single positive real number on a one-dimensional space by using a linear transformation that takes transformation coefficients that cannot be mutually reduced. It can be uniquely converted to "(hereinafter referred to as the second lemma). Hereinafter, a specific example using the second lemma will be described as a second embodiment.

  In the index generation device 100 according to the second embodiment, the conversion unit 103 calculates the sum of products of element data of each dimension forming high-dimensional data and each conversion coefficient acquired by the coefficient acquisition unit 102. In other words, the conversion unit 103 linearly converts high-dimensional data using each conversion coefficient. For example, the coefficient acquisition unit 102 acquires conversion coefficients such as the constants α1 and α2 defined in (Equation 4) above. Specifically, the coefficient acquisition unit 102 selects prime numbers corresponding to the number of dimensions of the high-dimensional data, and acquires the logarithm of each prime number with the product of the selected prime numbers as the conversion coefficient.

  Hereinafter, the above-described processing of the index generation device 100 according to the second embodiment will be described in more detail with reference to FIG.

The index generation device 100 acquires d-dimensional data v to be indexed (S21). d is an integer of 2 or more. The data v is an arbitrary point on the d-dimensional space and is expressed as follows. In the second embodiment, the element data (x ₁ , x ₂ ,..., X _d ) for each dimension may not be a natural number.
v (x ₁ , x ₂ ,..., x _d )

The index generation device 100 appropriately selects d prime numbers. The index generation device 100 may hold a prime table in advance. Selected _{prime, p 1, p 2, ···} , is denoted as _{p d.} Furthermore, the index generating apparatus 100 acquires the logarithm of each prime number with the product of the selected prime numbers as a base (S22). The calculation of the conversion coefficient can be expressed by the following (Formula 5).

上記（式５）において、α_ｉは、取得される複数の変換係数を示す。ｄ及びｐは上述のとおりである。

In the above (Formula 5), α _i represents a plurality of acquired conversion coefficients. d and p are as described above.

The index generation device 100 calculates the sum of products of the element data of each dimension forming the high-dimensional data and the conversion coefficient α _i (S23). This process is expressed by the following (formula 6).

The above (Equation 6) indicates that the data N ^d in the d-dimensional space is converted into a one-dimensional positive real number R using the conversion function g. Here, the conversion function g can be expressed by the following (formula 7). In (Formula 7), d, v, and α are as described above. u represents a value after conversion into one dimension (after mapping).

  As shown in the second lemma, the only mapping of high-dimensional data to a one-dimensional space can be realized by a linear transformation using transformation coefficients that cannot be mutually reduced. Therefore, the transform coefficient itself is not limited to the above contents, that is, the logarithm of each prime number with the product of prime numbers as the base. For example, the index generation device 100 (coefficient acquisition unit 102) can also acquire the square root of each selected prime as a conversion coefficient.

  As described above, according to the first and second embodiments, by using the uniqueness of the inverse operation of prime factorization or the uniqueness of linear transformation using coefficients that are not mutually reducible, Vector data can be uniquely converted to a single positive real number (including a natural number) in a one-dimensional space. Then, since the obtained one-dimensional data is used as index data, and the numerical values in the index data are real numbers having different uniqueness, an index value indicating a distance between the original data is calculated using the index data. be able to.

  FIG. 3 is a diagram conceptually illustrating a processing configuration example of the search device 200 in the first embodiment. The search device 200 in the first embodiment uses the index generated by the index generation device 100 described above. The search device 200 may acquire the index data (one-dimensional data) included in the index from the index generation device 100 as necessary, or acquire and hold the entire index from the index generation device 100 in advance. Also good.

  The search device 200 uses the query acquisition unit 201 that acquires the search target data having the same number of dimensions as the high-dimensional data, and the plurality of conversion coefficients that are the same as the plurality of conversion coefficients acquired by the coefficient acquisition unit 102. When evaluating the similarity between the search target conversion unit 202 that uniquely maps the search target data to the one-dimensional space and the high-dimensional data and the search target data in the same manner as the conversion unit 103 A distance calculation unit 203 that calculates a distance between the search target one-dimensional data obtained by the target conversion unit 202 and the one-dimensional data included in the index.

  The search device 200 has, for example, the same hardware configuration as the high-dimensional data search device 1 in a detailed embodiment (second embodiment and later) described later. Similar to the high-dimensional data search device 1, each processing unit described above is realized by processing a program. The hardware configuration of the search device 200 is not limited.

  Hereinafter, the search method in the first embodiment will be described with reference to FIG. FIG. 4 is a flowchart illustrating an operation example of the search device 200 according to the first embodiment. In the following description, the search device 200 is an execution subject of the search method, but each of the above-described processing units included in the search device 200 may be the execution subject.

  The search method in the first embodiment is a method that is executed by at least one computer such as the search device 200 and that uses an index generated by the above-described index generation method. The search method in the first embodiment acquires search target data having the same number of dimensions as the high-dimensional data (S41), and uses the same plurality of transform coefficients acquired by the index generation method described above, and In the same method as the mapping included in the index generation method, the search target data is uniquely mapped to a one-dimensional space (S42), and when evaluating the similarity between the high-dimensional data and the search target data, Calculating the distance between the search target one-dimensional data obtained by mapping (S42) of the search target data and the one-dimensional data included in the index as index data (S43).

  According to the index generation device 100 and the index generation method described above, as described above, the distance between index data can be an index value indicating the distance between corresponding high-dimensional data. Therefore, in this embodiment, similarly to the index data generation method, the search target data is uniquely mapped to the one-dimensional space, and the search target one-dimensional data is acquired. Then, when evaluating the similarity between the high-dimensional data that is the source of the index data and the search target data, the distance between the search target one-dimensional data and the one-dimensional data as the index data is Calculated. Since the calculated distance can be an index value of the distance between the corresponding high-speed data, the search process can be performed using the distance.

  As described above, according to the present embodiment, since the index data can be used to calculate the index value of the distance between corresponding high-dimensional data, the high-dimension that is the correct answer of the search result using the index value. The number of data can be reduced to some extent. Therefore, according to the present embodiment, the number of high-dimensional data for which the actual distance is to be calculated can be reduced, and further efficiency and speedup of the search process can be realized.

  Hereinafter, the details of the first embodiment will be described. Below, 2nd Embodiment and 3rd Embodiment are illustrated as detailed embodiment. Each of the following embodiments is an example when the index generation device 100, the search device 200, the index generation method, and the search method in the first embodiment are applied to a high-dimensional data search device. The first embodiment described above is not limited to application to a search apparatus that handles high-dimensional data, and can be applied to various data search apparatuses that can calculate similarity.

[Second Embodiment]
〔Device configuration〕
FIG. 5 is a diagram conceptually illustrating a hardware configuration example of a high-dimensional data search device (hereinafter simply referred to as a search device) 1 in the second embodiment. As shown in FIG. 5, the search device 1 according to the second embodiment has a CPU (Central Processing Unit) 10, a memory 11, and an input / output interface (I / F) that are connected to each other via a bus as a hardware configuration. 12, communication device 13 and the like.

  The memory 11 is a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk, or the like. The input / output I / F 12 includes an input device (not shown) that receives input of a user operation such as a keyboard and a mouse, an output device (not shown) that provides information to the user such as a display device and a printer, It can be connected to a device that exchanges data with a portable recording medium or the like. The communication device 13 communicates with other nodes. The search device 1 may not have an input device or an output device, and the hardware configuration of the search device 1 is not limited.

[Processing configuration]
FIG. 6 is a diagram conceptually illustrating a processing configuration example of the search device 1 in the second embodiment. As illustrated in FIG. 6, the search device 1 in the second embodiment includes a data acquisition unit 20, an index generation unit 21, a database (DB) 27, a query acquisition unit 30, a search unit 31, and the like. Each of these processing units is realized, for example, by executing a program stored in the memory 11 by the CPU 10. The program is installed from a portable recording medium such as a CD (Compact Disc) or a memory card via the input / output I / F 12 or from another computer on the network via the communication device 13. May be stored in the memory 11.

  Further, the processing unit described above may be realized by a plurality of computers. For example, the data acquisition unit 20 and the index generation unit 21 for index generation may be realized by one computer, and the query acquisition unit 30 and the search unit 31 may be realized by another computer. The DB 27 may be realized by another different computer.

  The data acquisition unit 20 corresponds to the data acquisition unit 101 described above. The data acquisition unit 20 acquires high-dimensional feature data such as video as high-dimensional data. The feature amount data may be information input by a user operating an input screen or the like using an input device, or the input / output I / F 12 or the communication device 13 from a portable recording medium, another computer, or the like. It may be information acquired via. The content of the high-dimensional data acquired by the data acquisition unit 20 is not limited.

  The index generation unit 21 assigns an index to the feature amount data (high-dimensional data) acquired by the data acquisition unit 20, and stores the feature amount data and index information in the DB 27. The index generation unit 21 includes a coefficient acquisition unit 23, a conversion unit 24, a rearrangement processing unit 25, and the like. In FIG. 6, for convenience of explanation, the coefficient acquisition unit 23, the conversion unit 24, and the rearrangement processing unit 25 are illustrated as being included in the index generation unit 21. The relationship between these processing units is not limited to the relationship shown in FIG.

  The DB 27 stores a large number of feature amount data and index information thereof. However, the DB 27 may be realized on a computer other than the search device 1. In this case, the index generation unit 21 accesses the DB 27 by communicating with another computer.

  The coefficient acquisition unit 23 and the conversion unit 24 correspond to the coefficient acquisition unit 102 and the conversion unit 103 described above. The coefficient acquisition unit 23 and the conversion unit 24 may execute processing based on the lemma 1 or may execute processing based on the lemma 2. The rearrangement processing unit 25 corresponds to the index generation unit 104 described above. The rearrangement processing unit 25 rearranges the one-dimensional data converted from the feature amount data by the conversion unit 24 into ascending or descending order as index data, and generates an index using the B + tree.

  The query acquisition unit 30 corresponds to the query acquisition unit 201 described above. The query acquisition unit 30 acquires feature amount data to be searched. The feature quantity data to be searched is data having the same number of dimensions as the feature quantity data acquired by the data acquisition unit 20, and is hereinafter referred to as search target data. In addition to the search target data, the query acquisition unit 30 further acquires a distance condition from the search target data. This distance condition is distance information in the high-dimensional space of the search target data. For the distance condition, for example, a radius r centering on the search target data in the high-dimensional space of the search target data is used. The search target data and the distance condition acquired by the query acquisition unit 30 are information specified by the similar search range query, and indicate the query range of the similar search.

  Using the index stored in the DB 27, the search unit 31 searches the feature amount data stored in the DB 27 for feature amount data that matches the range query based on the search target data and the distance condition. The range inquiry means a similar search request for extracting feature data whose distance from the search target data matches the distance condition from the DB 27. The search result data may be output to a display device or a printing device via the input / output I / F 12, stored in a portable recording medium via the input / output I / F 12, or the communication device 13. It may be transmitted to other computers via.

  As shown in FIG. 6, the search unit 31 includes a search target conversion unit 32, a distance calculation unit 33, a range search unit 34, and the like.

  The search target conversion unit 32 corresponds to the search target conversion unit 202 described above. Similar to the search target conversion unit 202, the search target conversion unit 32 converts the search target data acquired by the query acquisition unit 30 into search target primary data. Further, the search target conversion unit 32 uniquely maps the upper bound data and the lower bound data acquired by the range acquisition unit 35, which will be described later, to the one-dimensional space in the same manner as the conversion of the search target data. This is equivalent to mapping the similar search range in the high-dimensional space indicated by the search target data and the distance condition to the one-dimensional space. By this processing, the upper bound data and the lower bound data in the high dimensional space to which the feature amount data belongs are converted into the upper bound one dimensional data and the lower bound one dimensional data.

  The search target conversion unit 32 may acquire a plurality of conversion coefficients acquired by the coefficient acquisition unit 23 from the index generation unit 21, or may acquire a plurality of conversion coefficients acquired by the coefficient acquisition unit 23 in the same manner as the coefficient acquisition unit 23. A plurality of the same conversion coefficients may be acquired. Further, the search target conversion unit 32 has the same conversion rule (mapping rule, conversion function) as the conversion unit 24.

  The distance calculation unit 33 corresponds to the distance calculation unit 203 described above. The distance calculation unit 33 calculates the distance between the index data specified by the first target specifying unit 36 described later and the search target one-dimensional data obtained by the search target conversion unit 32. Since the index data and the search target one-dimensional data are both one-dimensional values, the distance calculation unit 33 calculates a difference between the values as the distance.

  The range search unit 34 refers to the index data included in the index stored in the DB 27 to extract feature amount data that is a solution to the range query based on the search target data and the distance condition.

  As illustrated in FIG. 6, the range search unit 34 includes a range acquisition unit 35, a first target specifying unit 36, a candidate extraction unit 37, a first similarity calculation unit 38, and the like. The range search unit 34 extracts feature quantity data as a solution to the range query by comparing the actual distance calculated by the first similarity calculation unit 38, which will be described later, with the query range information. However, the range search unit 34 can also extract feature data corresponding to the index data extracted as a solution candidate by the candidate extraction unit 37 described later as a solution for the range query. In this case, the range search unit 34 may not have the first similarity calculation unit 38. Furthermore, the range search unit 34 can also extract feature amount data corresponding to index data specified by a first target specifying unit 36 described later as a solution to the range query. In this case, the range search unit 34 may not have the candidate extraction unit 37 and the first similarity calculation unit 38.

  The range acquisition unit 35 acquires upper bound data and lower bound data related to the query range in the high-dimensional space of the search target data indicated by the search target data and the distance condition acquired by the query acquisition unit 30. When the distance condition indicates the radius r, the upper bound data and the lower bound data indicate the upper bound and the lower bound in the feature amount data group included within the radius r from the point corresponding to the search target data in the high-dimensional space. Show. Therefore, the upper bound data and the lower bound data acquired by the range acquisition unit 35 have the same number of dimensions as the search target data.

  The first target specifying unit 36 is an index within the range between the upper bound one-dimensional data and the lower bound one-dimensional data obtained from the upper bound data and the lower bound data by the search subject converting unit 32 among the indexes stored in the DB 27. Identify the data. Specifically, the first target specifying unit 36 specifies index data that is larger than the lower bound one-dimensional data and smaller than the upper bound one-dimensional data from the index.

  The candidate extracting unit 37 filters the index data specified by the first target specifying unit 36 by comparing the one-dimensional distance condition in the one-dimensional space obtained from the distance condition with the distance calculated by the distance calculating unit 33. Then, index data obtained by this filtering is extracted as a solution candidate. As described above, the distance condition acquired by the query acquisition unit 30 indicates the distance in the high-dimensional space of the feature amount data, and the one-dimensional distance condition indicates the distance in the one-dimensional space corresponding to the distance in the high-dimensional space. Show. This one-dimensional distance condition is calculated using, for example, Helder's inequality. Therefore, the candidate extraction unit 37 selects each index data and the search target conversion unit 32 from the index data within the range between the upper bound one-dimensional data and the lower bound one-dimensional data specified by the first target specifying unit 36. The index data whose distance to the search target one-dimensional data obtained by the above does not match the one-dimensional distance condition is excluded, and the remaining index data is set as a solution candidate.

  The first similarity calculation unit 38 calculates the actual distance between the feature amount data corresponding to the solution candidate index data extracted by the candidate extraction unit 37 and the search target data. The calculated actual distance is a distance in a high-dimensional space to which the feature amount data and the search target data belong.

[Operation example]
Hereinafter, the index generation method and the search method in the second embodiment will be described based on the operation of the search device 1 in the second embodiment. In the following description, the search device 1 is an execution subject of each method, but each of the above-described processing units included in the search device 1 may be an execution subject. Further, the execution subject may be a plurality of devices (computers).

  First, an index generation method according to the second embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating an operation example related to index generation of the search device 1 according to the second embodiment. However, FIG. 7 illustrates an index generation method based on the first lemma.

  The search device 1 acquires the dimension number d of the feature amount data to be indexed (S71). The dimension number d may be input by a user operating an input screen or the like using an input device, or may be acquired from a portable recording medium, another computer, or the like.

  The search device 1 selects d prime numbers (S72). The search device 1 may select the prime number from the prime number table. In this case, the search device 1 may hold in advance a prime number table including a sufficient number of prime numbers, or may obtain it from another computer or the like.

  Subsequently, the search device 1 determines a transform coefficient based on the d prime numbers selected in (S72) (S73). In the example of FIG. 7, the selected d prime numbers are directly determined as d transform coefficients. In the case of the index generation method based on the second lemma, the search device 1 calculates the logarithm of each of the d prime numbers with the product of the selected d prime numbers as d transform coefficients. In the case of another method, the search device 1 may calculate the square root of each of the selected d prime numbers as d conversion coefficients. If a plurality of transform coefficients are determined so that all pairs of transform coefficients cannot be mutually reduced, a specific determination method is not limited.

  The search device 1 acquires feature quantity data to be indexed (S74). The feature amount data may be input by a user operating an input screen or the like using an input device, or may be acquired from a portable recording medium, another computer, or the like.

  The search device 1 normalizes the feature amount data acquired in (S74) (S75). Specifically, the search device 1 normalizes element data of each dimension of the feature amount data to natural numbers. Normalization here means converting the element data of each dimension into a natural number so that it can be restored to the original data. Therefore, converting a decimal number to a natural number by simply deleting all decimal places does not correspond to this normalization.

  Using the conversion coefficient determined in (S73), the search device 1 uniquely converts the feature data normalized in (S75) into one dimension (S76). In the example of FIG. 7, as illustrated in the first embodiment, the search device 1 uses the element data of each dimension of the normalized feature amount data as a power, and uses each conversion coefficient acquired in (S73). Calculate the product of the values obtained by raising each to the power.

  The search device 1 arranges the one-dimensional data converted in (S76) in ascending or descending order (S77).

  The search device 1 determines whether there is any other feature amount data to be indexed (S78). If there is other feature amount data (S78; YES), the search device 1 acquires the feature amount data (S74), and executes (S75) and subsequent steps on the acquired feature amount data.

  If there is no other feature amount data (S78; NO), the search device 1 generates a hierarchical index including the one-dimensional data arranged in (S77) as index data of each feature amount data. And feature quantity data are stored in the DB 27 (S79). However, the search device 1 may store the index and feature amount data in a portable recording medium or may transmit them to another computer.

  The index generation method in the second embodiment is not limited to the example of FIG. When the number of dimensions of the high-dimensional data to be processed is known in advance, the conversion coefficient determined in (S73) may be held in advance in the search device 1, and in this case, (S71) , (S72) and (S73) may not be included in the index generation method. In the example of FIG. 7, the first lemma is used. However, the index generation method in the second embodiment may be based on the second lemma. In this case, the search device 1 does not execute (S75). In (S76), the element data of each dimension of the feature amount data is added to the linear conversion function including each conversion coefficient determined in (S73). By substituting, the feature data is converted into one-dimensional data.

  Next, a search method in the second embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing an operation example related to a search method related to a range query (Range Query) of the search device 1 according to the second embodiment. In the following description, the search target data is expressed as query data q.

  The search device 1 acquires the query data q and the distance condition as query parameters (S81). The query data q is search target data and is high-dimensional feature data. In the example of FIG. 8, the distance radius r is specified as the distance condition.

  The search device 1 obtains upper and lower data points that can be a solution to the range query in the original high-dimensional space from the query data q and the distance radius r (S82). The upper bound data point is denoted as upper bound data, and the lower bound data point is denoted as lower bound data.

  The search device 1 uniquely converts the upper bound data and the lower bound data acquired in (S82) into one dimension by the same method as (S76) in FIG. 7 (S83). At this time, the search device 1 uses the same conversion coefficient as that determined in (S73) of FIG. Thereby, the upper bound data and the lower bound data are converted into the upper bound one-dimensional data and the lower bound one-dimensional data by the same conversion rule as the conversion from the feature quantity data to be indexed to the index data.

  The search device 1 specifies index data within the range between the upper bound one-dimensional data and the lower bound one-dimensional data obtained in (S83) from the index stored in the DB 27 (S84). When there is no index data within the range, the search device 1 determines that there is no solution.

  The search device 1 uniquely converts the query data q into one dimension by the same method as (S83) (S85). Thus, the query data q is converted into query one-dimensional data according to the same conversion rule as the conversion from the feature quantity data to be indexed to the index data.

  The search device 1 calculates the distance between the one-dimensional query data obtained in (S85) and each index data specified in (S84) (S86). Since the index data and the query one-dimensional data are both one-dimensional values, the search device 1 calculates the difference between the values as the distance.

  The search device 1 filters the index data specified in (S84) using the distance calculated in (S86), and extracts the remaining index data as solution candidates (S87). The search device 1 performs the filtering specifically as follows. The search device 1 calculates a one-dimensional distance condition in the one-dimensional space corresponding to the radius r obtained in (S81) using a Helder inequality, and the distance calculated in (S86) is the one-dimensional distance. Index data that does not match the distance condition is excluded from the solution candidates.

  The search device 1 acquires feature amount data corresponding to the index data of the solution candidate extracted in (S87) from the DB 27, and calculates an actual distance between the feature amount data and the query data q (S88).

  The search device 1 extracts feature amount data whose actual distance calculated in (S88) is smaller than the radius r obtained in (S82) as a solution to the range query (S89).

  The search method in the second embodiment is not limited to the example of FIG. For example, the search device 1 may set the index data specified in (S84) as a solution candidate. In this case, the search method may not include (S85), (S86), and (S87). Further, the search device 1 may set the feature amount data corresponding to the index data specified in (S84) as a solution to the range query. In this case, the search method may not include (S85) and subsequent steps. Further, the search device 1 may set feature amount data corresponding to the index data extracted as a solution candidate in (S87) as a solution for the range query. In this case, the search method may not include (S88) and subsequent steps. Further, the search method is not limited to the execution order of the steps shown in FIG. For example, (S85) may be executed at any time as long as it is after (S82) and before (S86).

[Operations and effects in the second embodiment]
As described above, in the second embodiment, high-dimensional feature quantity data to be indexed is uniquely converted into one-dimensional data by a conversion rule using conversion coefficients that cannot be mutually reduced. A hierarchical index included as index data is generated in a state where the original data is arranged in ascending or descending order. Then, a range query search process using this index is executed. In this search process, the index data corresponding to the feature data serving as a solution to the range query can be narrowed down to some extent only by calculation in the one-dimensional space to which the index data belongs. That is, according to the second embodiment, a low load such as a distance calculation (subtraction) in a one-dimensional space is performed without performing a high load process such as a distance calculation (similarity calculation) in a high-dimensional space. Since the range query solution candidates can be narrowed down in the process, the range query search process can be speeded up.

  Specifically, in the second embodiment, the upper and lower bounds of the high-dimensional space obtained from the distance condition of the range query are the same conversion rules as those for generating the index data, and the one-dimensional space of the index data Mapped to By this mapping, the upper bound point (upper bound one-dimensional data) and the lower bound point (lower bound one-dimensional data) on the one-dimensional space are obtained, and the index data within the range between the upper bound point and the lower bound point is specified. Is done. As described above, in the second embodiment, by converting the upper and lower bounds of the range query into a one-dimensional space, it is possible to narrow down index data that can correspond to the solution of the range query from all index data.

  Further, the query data (search target data) of the range query is also converted into one-dimensional data (query one-dimensional data) by the same conversion rule as that for generating index data, and the query one-dimensional data and index data are converted. Index data as solution candidates is further narrowed down from the distance (difference). Such further narrowing down of solution candidates can also be realized only by calculation in a one-dimensional space. With respect to the solution candidates narrowed down in this way, an actual distance in a high-dimensional space is calculated, and a final solution for a range inquiry is obtained according to the actual distance.

  As described above, according to the search processing of the range query in the second embodiment, the target of the real distance calculation in the high-dimensional space with a high processing load is reduced by narrowing down the solution candidates stepwise in the one-dimensional space. As a result, the search processing of the range query is speeded up.

[Third Embodiment]
In the second embodiment, only the range query search function has been described. The search device 1 according to the third embodiment includes a search function for a k-Nearest Neighbors Query in addition to a search function for a range query. Hereinafter, the search device 1 in the third embodiment will be described focusing on the content different from the second embodiment. In the following description, the same contents as those of the second embodiment are omitted as appropriate.

[Processing configuration]
FIG. 9 is a diagram conceptually illustrating a processing configuration example of the search device 1 in the third embodiment. In the search device 1 according to the third embodiment, the search unit 31 further includes a nearest neighbor search unit 40 in addition to the configuration of the second embodiment. The nearest neighbor search unit 40 is also realized by executing a program stored in the memory 11 by the CPU 10 as in the case of other processing units.

  The query acquisition unit 30 acquires data number information indicating search target data and the number of data k (k is a natural number) in the k nearest neighbor search.

  The nearest neighbor search unit 40 refers to the index data included in the index stored in the DB 27 to extract feature amount data that is a solution to the k nearest neighbor search indicated by the search target data and the data count information. The k nearest neighbor search is a similar search process for extracting the top k feature amount data from the DB 27 having the smallest distance to the search target data. The nearest neighbor search unit 40 operates the range search unit 34 using the kth smallest real distance of the real distances calculated by the second similarity calculation unit 42 described later as a distance condition together with the search target data. Then, from the feature amount data extracted thereby, the top k feature amount data is extracted as a solution of the k nearest neighbor search in ascending order of the actual distance.

The nearest neighbor searching unit 40 includes a second target specifying unit 41, a second similarity calculating unit 42, and the like.
The second target specifying unit 41 uses the search target one-dimensional data based on the position of the search target one-dimensional data obtained by the search target conversion unit 32 in the arrangement order of the index data included in the index stored in the DB 27. The number of index data that is a predetermined multiple of the number k indicated by the data number information is specified immediately before and immediately after. For example, the second target specifying unit 41 specifies k index data immediately before the search target one-dimensional data, specifies k index data immediately after the search target one-dimensional data, and 2k index data in total. Is identified. The specific identification method of index data that is a predetermined multiple of the number of data k is not limited. Different numbers of index data may be specified immediately before and after. For example, a number of index data that is a predetermined multiple of the number of data k may be specified in the order of closer to the search target one-dimensional data.

  The second similarity calculating unit 42 calculates an actual distance between each feature amount data corresponding to each index data specified by the second target specifying unit 41 and the search target data.

[Operation example]
Hereinafter, a search method in the third embodiment will be described with reference to FIG. FIG. 10 is a flowchart illustrating an operation example of the k nearest neighbor search of the search device 1 according to the third embodiment. In the following description, the search device 1 is an execution subject of each method, but each of the above-described processing units included in the search device 1 may be an execution subject. Further, the execution subject may be a plurality of devices (computers). In the following description, the search target data is expressed as query data q.

  The search device 1 acquires the query data q and the data number information (S101). The query data q is search target data and is high-dimensional feature data. In the example of FIG. 10, the data number information indicates the data number k.

  The search device 1 uniquely converts the query data q into one dimension by the same method as (S85) in FIG. 8 (S102). Thus, the query data q is converted into query one-dimensional data according to the same conversion rule as the conversion from the feature quantity data to be indexed to the index data.

  The search device 1 acquires the position of the query one-dimensional data obtained in (S102) in the arrangement order of the index data included in the index stored in the DB 27 (S103). For example, the search device 1 recognizes that the query one-dimensional data is located between the m-th index data from the front and the n-th index data from the front.

  The search device 1 specifies a number of index data that is a predetermined multiple of the number k indicated by the data number information immediately before and immediately after the search target one-dimensional data (S104). For example, the search device 1 specifies k index data immediately before the query one-dimensional data, specifies k index data immediately after the query one-dimensional data, and specifies 2k index data in total.

  The search device 1 calculates the actual distance between each feature amount data corresponding to the index data specified in (S104) and the query data q (S105). According to the above example, the search device 1 calculates the actual distance between each of the 2k feature amount data corresponding to the 2k index data and the query data q, and obtains 2k actual distances.

The search device 1 selects the kth smallest actual distance s from the actual distances calculated in (S105) (S106).
The search device 1 sets the actual distance s selected in (S106) as a distance condition, and performs the operation shown in FIG. 8 (S107). In this operation, since the query one-dimensional data has already been acquired in (S102), (S85) may not be executed.

  The search device 1 extracts the top k feature value data from the feature value data obtained as a solution in FIG. 8 (S89) in ascending order of the actual distance as a solution for the k nearest neighbor search (S108). .

[Operations and effects in the third embodiment]
In the third embodiment, the k nearest neighbor search process using the above-described index is executed. Specifically, based on the position of the query one-dimensional data in the one-dimensional space to which the index data belongs, the number of index data that is a predetermined multiple of the number k of data around the query one-dimensional data is specified. The actual distance on the high-dimensional space between the index data and the query data is calculated. Then, this actual distance is set as the radius of the distance condition, and the range inquiry search process of the second embodiment is executed. From the feature quantity data extracted as a solution in the range query search process, k feature quantity data are extracted as a solution of k nearest neighbor search in ascending order of the actual distance.

  As described above, according to the third embodiment, in the k nearest neighbor search process, the number of objects to be calculated for the actual distance in a high-dimensional space with a high processing load is reduced. Can do.

  In the plurality of flowcharts used in the above description, a plurality of steps (processes) are described in order, but the execution order of the steps executed in each embodiment is not limited to the description order. In each embodiment, the order of the illustrated steps can be changed within a range that does not hinder the contents. Moreover, each above-mentioned embodiment and each Example can be combined in the range in which the content does not conflict.

  A part or all of the above embodiments and examples can be specified as in the following supplementary notes. However, each embodiment and each example are not limited to the following description.

(Supplementary note 1) a data acquisition unit for acquiring high-dimensional data;
A coefficient acquisition unit that acquires conversion coefficients corresponding to the number of dimensions of the high-dimensional data that cannot be mutually reduced;
Using a plurality of conversion coefficients acquired by the coefficient acquisition unit, a conversion unit that uniquely maps the high-dimensional data to a one-dimensional space;
An index generation unit that includes the one-dimensional data obtained by the conversion unit as index data in an ascending or descending order, and generates an index having a hierarchical structure;
An index generation device comprising:

(Additional remark 2) The said conversion part calculates the sum of the product of the element data of each dimension which forms the said high-dimensional data, and each conversion coefficient acquired by the said coefficient acquisition part,
The index generation device according to attachment 1.

(Supplementary Note 3) The coefficient acquisition unit selects a prime number corresponding to the number of dimensions of the high-dimensional data, and acquires a logarithm of each prime number based on the product of the selected prime numbers as the plurality of transform coefficients.
The index generation device according to attachment 2.

(Additional remark 4) The said coefficient acquisition part acquires the prime number for the number of dimensions of the said high-dimensional data as said some conversion coefficient,
The conversion unit normalizes the high-dimensional data acquired by the data acquisition unit to a natural number, uses the element data of each dimension forming the normalized high-dimensional data as a power, and the coefficient acquisition unit Calculate the product of the values obtained by raising each obtained conversion coefficient to the power,
The index generation device according to attachment 1.

(Supplementary note 5) In the search device using the index generated by the index generation device according to any one of Supplementary notes 1 to 4,
A query acquisition unit that acquires search target data having the same number of dimensions as the high-dimensional data;
A search target conversion unit that uniquely maps the search target data to the one-dimensional space in the same manner as the conversion unit, using the same plurality of conversion coefficients as the plurality of conversion coefficients acquired by the coefficient acquisition unit; ,
When evaluating the similarity between the high-dimensional data and the search target data, the search target one-dimensional data obtained by the search target conversion unit and the one-dimensional data included in the index as the index data A distance calculation unit for calculating a distance between;
A search device comprising:

(Appendix 6) a first condition acquisition unit that acquires a distance condition from the search target data;
A range search unit that extracts high-dimensional data that is a solution to a range query based on the search target data and the distance condition by referring to the index data included in the index;
Further comprising
The range search unit
A range acquisition unit that acquires upper bound data and lower bound data related to a query range in a high-dimensional space of the search target data indicated by the search target data and the distance condition;
Including
The search target conversion unit uses the plurality of conversion coefficients to uniquely map the upper bound data and the lower bound data to the one-dimensional space,
The range search unit
Among the index data included in the index, the search target conversion unit specifies index data within a range between the upper bound one-dimensional data and the lower bound one-dimensional data obtained from the upper bound data and the lower bound data. 1st object specific part,
Further including
The distance calculation unit calculates a distance between the index data specified by the first target specifying unit and the search target one-dimensional data obtained by the search target conversion unit,
The range search unit
The index data specified by the first target specifying unit is filtered by comparing the one-dimensional distance condition in the one-dimensional space obtained from the distance condition with the distance calculated by the distance calculating unit, and the filtering A candidate extraction unit that extracts the index data obtained in step 1 as solution candidates,
Further including
The search device according to appendix 5.

(Appendix 7) The range search unit
A first similarity calculation unit that calculates an actual distance between high-dimensional data corresponding to the solution candidate index data extracted by the candidate extraction unit and the search target data;
Further including
Extracting high-dimensional data as a solution to the range query by comparing the actual distance calculated by the first similarity calculation unit and the distance condition;
The search device according to appendix 6.

(Additional remark 8) The 2nd condition acquisition part which acquires the data number information which shows the data number k (k is a natural number),
By referring to the index data included in the index, a nearest neighbor search unit that extracts high-dimensional data serving as a solution of the k nearest neighbor search indicated by the search target data and the data number information;
Further comprising
The nearest neighbor search unit includes:
Based on the position of the search target one-dimensional data obtained by the search target conversion unit in the order in which the index data included in the index is arranged, the data count information is obtained immediately before and immediately after the search target one-dimensional data. A second target specifying unit that specifies a predetermined number of index data of the number shown,
A second similarity calculating unit that calculates an actual distance between each high-dimensional data corresponding to each index data specified by the second target specifying unit and the search target data;
Including
The k-th smallest actual distance calculated by the second similarity calculation unit is used as the distance condition together with the search target data, and is extracted by operating the range search unit. Extracting the top k high-dimensional data from the dimensional data in ascending order of the real distance as a solution of the k nearest neighbor search;
The search device according to appendix 7.

(Supplementary note 9) In an index generation method executed by at least one computer,
Acquire high-dimensional data,
Obtain conversion coefficients for the number of dimensions of the high-dimensional data that cannot be mutually reduced,
Using the acquired plurality of transformation coefficients, the high-dimensional data is uniquely mapped to a one-dimensional space,
One-dimensional data obtained by the mapping is included as index data in an ascending or descending order, and an index having a hierarchical structure is generated.
Index generation method including the above.

(Supplementary Note 10) The mapping calculates a sum of products of element data of each dimension forming the high-dimensional data and each conversion coefficient acquired by the coefficient acquisition unit.
The index generation method according to attachment 9.

(Supplementary Note 11) Select prime numbers corresponding to the number of dimensions of the high-dimensional data.
Further including
The acquisition of the conversion coefficient is to obtain the logarithm of each prime number based on the product of the selected prime numbers as the plurality of conversion coefficients.
The index generation method according to attachment 10.

(Supplementary Note 12) Normalize the high-dimensional data to natural numbers.
Further including
The acquisition of the conversion coefficient acquires a prime number corresponding to the number of dimensions of the high-dimensional data as the plurality of conversion coefficients,
The mapping uses the element data of each dimension forming the normalized high-dimensional data as a power, and calculates a product of values obtained by raising each of the acquired transform coefficients to the power. ,
The index generation method according to attachment 9.

(Supplementary note 13) In the search method using the index generated by the index generation method according to any one of Supplementary notes 9 to 12, and executed by at least one computer,
Retrieve search target data having the same number of dimensions as the high-dimensional data,
Using the plurality of transform coefficients, the search target data is uniquely mapped to the one-dimensional space in the same manner as the mapping included in the index generation method,
When evaluating the similarity between the high-dimensional data and the search target data, between the search target one-dimensional data obtained by the mapping of the search target data and the one-dimensional data included in the index Calculate distance,
Search method including that.

(Appendix 14) Obtaining a distance condition from the search target data,
Obtaining upper bound data and lower bound data related to a query range in a high-dimensional space of the search subject data indicated by the search subject data and the distance condition;
Using the same method and the same plurality of transformation coefficients as the mapping of the search target data, the upper bound data and the lower bound data are uniquely mapped to the one-dimensional space,
From the index data included in the index, specify the index data within the range between the upper bound one-dimensional data and the lower bound one-dimensional data obtained by the mapping of the upper bound data and the lower bound data,
Calculating a distance between the identified index data and the one-dimensional data to be searched;
The identified index data is filtered by comparing the one-dimensional distance condition in the one-dimensional space obtained from the distance condition with the calculated distance, and the index data obtained by the filtering is extracted as a solution candidate To
The search method according to supplementary note 13, further including:

(Supplementary Note 15) An actual distance between the high-dimensional data corresponding to the index data extracted as the solution candidate and the search target data is calculated,
By comparing the calculated actual distance and the distance condition, high-dimensional data that is a solution to a range query based on the search target data and the distance condition is extracted.
The search method according to supplementary note 14, further including:

(Supplementary Note 16) Obtain data number information indicating the number of data k (k is a natural number),
Based on the position of the one-dimensional search target data in the order of the index data included in the index, a number that is a predetermined multiple of the number indicated by the data count information immediately before and after the one-dimensional search target data Identify index data for
Calculating an actual distance between each high-dimensional data corresponding to each identified index data and the search target data;
Specifying the kth smallest actual distance in the calculated actual distance as the distance condition;
From the high-dimensional data extracted as the solution of the range query based on the search target data and the distance condition, the top k high-dimensional data are extracted as the solution of the k nearest neighbor search in ascending order of the actual distance.
The search method according to supplementary note 15, further including:

(Supplementary Note 17) A program that causes at least one computer to execute the index generation method according to any one of Supplementary Notes 9 to 12.

(Supplementary Note 18) A program that causes at least one computer to execute the search method according to any one of Supplementary Notes 13 to 16.

1 High-dimensional data retrieval device (retrieval device)
10 CPU
11 Memory 20, 101 Data acquisition unit 21, 104 Index generation unit 23, 102 Coefficient acquisition unit 24, 103 Conversion unit 25 Rearrangement processing unit 27 Database (DB)
30, 201 Query acquisition unit 31 Search unit 32, 202 Search target conversion unit 33, 203 Distance calculation unit 34 Range search unit 35 Range acquisition unit 36 First target identification unit 37 Candidate extraction unit 38 First similarity calculation unit 40 Nearest Search unit 41 Second object specifying unit 42 Second similarity calculation unit 100 Index generation device 200 Search device

Claims (10)

A data acquisition unit for acquiring high-dimensional data;
A coefficient acquisition unit that acquires conversion coefficients corresponding to the number of dimensions of the high-dimensional data that cannot be mutually reduced;
Using a plurality of conversion coefficients acquired by the coefficient acquisition unit, a conversion unit that uniquely maps the high-dimensional data to a one-dimensional space;
An index generation unit that includes the one-dimensional data obtained by the conversion unit as index data in an ascending or descending order, and generates an index having a hierarchical structure;
An index generation device comprising: The conversion unit calculates a sum of products of element data of each dimension forming the high-dimensional data and each conversion coefficient acquired by the coefficient acquisition unit;
The index generation device according to claim 1. The coefficient acquisition unit selects a prime number corresponding to the number of dimensions of the high-dimensional data, and acquires the logarithm of each prime number based on the product of the selected prime numbers as the plurality of transform coefficients.
The index generation device according to claim 2. The coefficient acquisition unit acquires a prime number corresponding to the number of dimensions of the high-dimensional data as the plurality of conversion coefficients,
The conversion unit normalizes the high-dimensional data acquired by the data acquisition unit to a natural number, uses the element data of each dimension forming the normalized high-dimensional data as a power, and the coefficient acquisition unit Calculate the product of the values obtained by raising each obtained conversion coefficient to the power,
The index generation device according to claim 1. In the search apparatus using the said index produced | generated by the index production | generation apparatus of any one of Claim 1 to 4,
A query acquisition unit that acquires search target data having the same number of dimensions as the high-dimensional data;
A search target conversion unit that uniquely maps the search target data to the one-dimensional space in the same manner as the conversion unit, using the same plurality of conversion coefficients as the plurality of conversion coefficients acquired by the coefficient acquisition unit; ,
When evaluating the similarity between the high-dimensional data and the search target data, the search target one-dimensional data obtained by the search target conversion unit and the one-dimensional data included in the index as the index data A distance calculation unit for calculating a distance between;
A search device comprising: A first condition acquisition unit that acquires a distance condition from the search target data;
A range search unit that extracts high-dimensional data that is a solution to a range query based on the search target data and the distance condition by referring to the index data included in the index;
Further comprising
The range search unit
A range acquisition unit that acquires upper bound data and lower bound data related to a query range in a high-dimensional space of the search target data indicated by the search target data and the distance condition;
Including
The search target conversion unit uses the plurality of conversion coefficients to uniquely map the upper bound data and the lower bound data to the one-dimensional space,
The range search unit
Among the index data included in the index, the search target conversion unit specifies index data within a range between the upper bound one-dimensional data and the lower bound one-dimensional data obtained from the upper bound data and the lower bound data. 1st object specific part,
Further including
The distance calculation unit calculates a distance between the index data specified by the first target specifying unit and the search target one-dimensional data obtained by the search target conversion unit,
The range search unit
The index data specified by the first target specifying unit is filtered by comparing the one-dimensional distance condition in the one-dimensional space obtained from the distance condition with the distance calculated by the distance calculating unit, and the filtering A candidate extraction unit that extracts the index data obtained in step 1 as solution candidates,
Further including
The search device according to claim 5. The range search unit
A first similarity calculation unit that calculates an actual distance between high-dimensional data corresponding to the solution candidate index data extracted by the candidate extraction unit and the search target data;
Further including
Extracting high-dimensional data as a solution to the range query by comparing the actual distance calculated by the first similarity calculation unit and the distance condition;
The search device according to claim 6. A second condition acquisition unit for acquiring data number information indicating the data number k (k is a natural number);
By referring to the index data included in the index, a nearest neighbor search unit that extracts high-dimensional data serving as a solution of the k nearest neighbor search indicated by the search target data and the data number information;
Further comprising
The nearest neighbor search unit includes:
Based on the position of the search target one-dimensional data obtained by the search target conversion unit in the order in which the index data included in the index is arranged, the data count information is obtained immediately before and immediately after the search target one-dimensional data. A second target specifying unit that specifies a predetermined number of index data of the number shown,
A second similarity calculating unit that calculates an actual distance between each high-dimensional data corresponding to each index data specified by the second target specifying unit and the search target data;
Including
The k-th smallest actual distance calculated by the second similarity calculation unit is used as the distance condition together with the search target data, and is extracted by operating the range search unit. Extracting the top k high-dimensional data from the dimensional data in ascending order of the real distance as a solution of the k nearest neighbor search;
The search device according to claim 7. In an index generation method executed by at least one computer,
Acquire high-dimensional data,
Obtain conversion coefficients for the number of dimensions of the high-dimensional data that cannot be mutually reduced,
Using the acquired plurality of transformation coefficients, the high-dimensional data is uniquely mapped to a one-dimensional space,
One-dimensional data obtained by the mapping is included as index data in an ascending or descending order, and an index having a hierarchical structure is generated.
Index generation method including the above. A search method using the index generated by the index generation method according to claim 9 and executed by at least one computer,
Retrieve search target data having the same number of dimensions as the high-dimensional data,
Using the plurality of transform coefficients, the search target data is uniquely mapped to the one-dimensional space in the same manner as the mapping included in the index generation method,
When evaluating the similarity between the high-dimensional data and the search target data, between the search target one-dimensional data obtained by the mapping of the search target data and the one-dimensional data included in the index Calculate distance,
Search method including that.

Images