JP2023013868A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

To generate a group in which objects are classified.SOLUTION: An information processing device according to the present application comprises an acquisition unit and a generation unit. The acquisition unit acquires object information indicating a plurality of objects serving as a data search target. The generation unit generates first group information indicating a plurality of first groups in which the plurality of objects are classified, and second group information indicating a plurality of second groups, which are groups in the classification different from the plurality of first groups and in which objects belonging to the same first group in the plurality of first groups among the plurality of objects are classified into the same group, on the basis of the object information acquired by the acquisition unit.SELECTED DRAWING: Figure 24

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Conventionally, techniques for searching (retrieving) various types of information have been provided. For example, there is provided a technique of generating a graph in which nodes corresponding to search targets are connected by edges and performing a search using the generated graph. Also, such a technique is used, for example, for image retrieval.

Japanese Patent No. 6293335 Japanese Patent No. 6300982 Japanese Patent No. 6311000

Masajiro Iwasaki, "Neighborhood Retrieval Using Approximate k-Nearest Neighbor Graph Using Tree Structured Index", Transactions of Information Processing Society of Japan, 2011/2, Vol. 52, No. 2. pp.817-828.

However, the conventional technology described above has room for improvement in terms of efficiency of search processing. For example, in the conventional technology described above, the number of edges in the graph is adjusted to change the graph to a graph that enables efficient searches, thereby improving efficiency such as reducing the search processing time. There is a limit to improving the efficiency of search processing by modification. Therefore, in order to generate information that can be used for efficient search processing, such as speeding up search processing, it is necessary to introduce the concept of groups that classify objects to be searched, and generate groups that classify objects. is desired.

The present application has been made in view of the above, and aims to provide an information processing apparatus, an information processing method, and an information processing program for generating groups for classifying objects.

An information processing apparatus according to the present application includes an acquisition unit that acquires object information indicating a plurality of objects to be searched for data; and first group information indicating a first group of the plurality of first groups, and an object belonging to the same first group in the plurality of first groups among the plurality of objects, which is a group classified differently from the plurality of first groups. and a generation unit that generates second group information indicating a plurality of second groups that are groups classified into the same group.

According to one aspect of the embodiment, it is possible to generate groups for classifying objects.

FIG. 1 is a diagram illustrating an example of information processing according to the first embodiment. FIG. 2 is a diagram illustrating an example of data according to the first embodiment; FIG. 3 is a diagram illustrating an example of data according to the first embodiment; FIG. 4 is a diagram illustrating a configuration example of an information processing system according to the first embodiment. FIG. 5 is a diagram illustrating a configuration example of an information processing apparatus according to the first embodiment; 6 is a diagram illustrating an example of an object information storage unit according to the first embodiment; FIG. 7 is a diagram illustrating an example of a graph information storage unit according to the first embodiment; FIG. 8 is a diagram illustrating an example of a quantization information storage unit according to the first embodiment; FIG. 9 is a diagram illustrating an example of a codebook information storage unit according to the first embodiment; FIG. FIG. 10 is a flowchart illustrating an example of information processing according to the first embodiment. FIG. 11 is a flowchart illustrating an example of search processing according to the first embodiment. FIG. 12 is a diagram illustrating an example of information processing according to the second embodiment. FIG. 13 is a diagram illustrating a configuration example of an information processing apparatus according to the second embodiment. 14 is a diagram illustrating an example of a quantization information storage unit according to the second embodiment; FIG. FIG. 15 is a diagram illustrating an example of a blob information storage unit according to the second embodiment; FIG. 16 is a flowchart illustrating an example of search processing according to the second embodiment. FIG. 17 is a diagram illustrating an example of information processing according to a modification; 18 is a diagram illustrating an example of a graph information storage unit according to a modification; FIG. FIG. 19 is a flowchart illustrating an example of search processing according to the modification. FIG. 20 is a diagram illustrating another example of information processing according to the modification. FIG. 21 is a diagram illustrating an example of blob information generation processing. FIG. 22 is a flowchart illustrating an example of information processing according to the modification. FIG. 23 is a diagram illustrating an example of information processing according to the third embodiment; FIG. 24 is a diagram illustrating a configuration example of an information processing apparatus according to the third embodiment; 25 is a diagram illustrating an example of a blob connection graph information storage unit according to the third embodiment; FIG. FIG. 26 is a flowchart illustrating an example of information processing according to the third embodiment. FIG. 27 is a flowchart illustrating an example of search processing according to the third embodiment. FIG. 28 is a flowchart illustrating an example of search processing according to the third embodiment. FIG. 29 is a diagram illustrating an example of blob connection graph information according to a modification. 30 is a diagram illustrating an example of a blob information storage unit according to a modification; FIG. 31 is a diagram illustrating an example of a blob connection graph information storage unit according to a modification; FIG. FIG. 32 is a diagram illustrating an example of information processing according to the fourth embodiment; FIG. 33 is a diagram illustrating a configuration example of an information processing apparatus according to the fourth embodiment; FIG. 34 is a conceptual diagram regarding quantization according to the fourth embodiment. FIG. 35 is a diagram illustrating an example of a transposed index according to the fourth embodiment; FIG. 36 is a diagram showing a specific example of transposed indexes according to the fourth embodiment. FIG. 37 is a flow chart showing an example of information processing according to the fourth embodiment. FIG. 38 is a flow chart showing an example of information processing according to the fourth embodiment. FIG. 39 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device.

Hereinafter, modes for implementing an information processing apparatus, an information processing method, and an information processing program (hereinafter referred to as "embodiments") according to the present application will be described in detail with reference to the drawings. The information processing apparatus, information processing method, and information processing program according to the present application are not limited to this embodiment. Also, in each of the following embodiments, the same parts are denoted by the same reference numerals, and overlapping descriptions are omitted.

(embodiment)
[1. First Embodiment]
[1-1. information processing]
An example of information processing according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of information processing according to the first embodiment. The information processing apparatus 100 executes search processing using a graph index (simply referred to as a “graph”) in which a plurality of objects to be searched for data are structured in a graph. FIG. 1 shows a part of search processing when the information processing apparatus 100 performs neighborhood search using a graph in which nodes corresponding to vectors obtained by vectorizing an object to be searched for data are connected by edges. . The information processing apparatus 100 searches for nodes in the vicinity of a given search query (vector) by tracing the graph, with each node of the graph as an object to be searched. The information processing apparatus 100 extracts a predetermined number of nodes specified as the number of neighboring nodes to be extracted (hereinafter also referred to as “the number of searches”) as neighboring nodes of the search query. In the following, a case where image information is targeted for data retrieval will be described as an example, but data retrieval targets may be various objects such as moving image information and audio information.

The information processing apparatus 100 performs search processing on nodes corresponding to a huge amount of image information in units of millions to hundreds of millions. ) only. For example, the information processing apparatus 100 acquires information about a plurality of nodes (vectors) indicated by nodes N1, N7, N9, etc., as indicated by spatial information SP1 in FIG. When "node N* (* is an arbitrary number)" is described in this way, it indicates that the node is identified by the node ID "N*". For example, when "node N1" is described, the node is identified by the node ID "N1". Each white circle (o) in the spatial information SP1 indicates each node.

In the spatial information SP1 in FIG. 1, nodes that are mainly related to the explanation are denoted by reference numerals, but each of the unsigned white circles (○) is also a node, and many nodes are included in addition to the nodes shown in the figure. be Each node corresponds to each object (search target). For example, each of the multiple local features extracted from the image may be an object. Also, for example, various data in which distances between objects are defined may be objects.

For example, the spatial information SP1 in FIG. 1 may be Euclidean space. For example, the space information SP1 corresponds to the number of dimensions of the vector of the object, and is assumed to be a multidimensional space such as 100 or 1000 dimensions. FIG. 1 conceptually illustrates a state in which a vector (space) is divided into a plurality of partial regions (subspaces) by Cartesian product quantization as shown in spatial information SP1. will be described later.

Dotted lines connecting white circles (o), which are nodes in the spatial information SP1, indicate edges connecting the nodes. In the example of FIG. 1, in order to simplify the explanation, nodes are connected by undirected (bidirectional) edges (hereinafter also simply referred to as "edges"). The undirected edge here means an edge that allows data to be traced bi-directionally between connected nodes. For example, the white circle (◯) indicating the node N1 and the white circle (◯) indicating the node N4 are connected by a dotted line, and the node N1 and the node N4 are connected by an edge. That is, in the graph shown in the spatial information SP1, it is possible to trace between the node N1 and the node N4 in both directions. Specifically, in the spatial information SP1, the node N1 can be traced to the node N4, and the node N4 can be traced to the node N1.

In the example of FIG. 1, only the edges connecting the illustrated nodes are illustrated for the sake of illustration, but a large number of edges are included in addition to the illustrated edges. Thus, in the spatial information SP1 in FIG. 1, only a part of the edge is illustrated, but it is assumed to be a k-nearest neighbor graph, for example. Note that the spatial information SP1 may be various graphs.

Also, the edges of the graph are not limited to undirected edges, and may be directed edges. A directed edge can only be traced from the node that is the reference source of the directed edge to the node that is the reference destination. For example, when two nodes are connected by two directed edges: a first edge with one as a reference source and the other as a reference destination, and a second edge with one as a reference destination and the other as a reference source , the two nodes are connected by an undirected (bidirectional) edge.

From here, the search processing will be described with reference to FIG. In the example of FIG. 1, it is assumed that the information processing apparatus 100 has acquired a graph GR1 as shown in the spatial information SP1. Note that the information processing apparatus 100 may generate the graph GR1 using various conventional techniques as appropriate. First, before describing the retrieval process, direct product quantization of vectors (spaces) will be described.

FIG. 1 conceptually illustrates a state in which a vector (space) is divided into four subspaces AR11 to AR14 by direct product quantization as shown in spatial information SP1. The subspaces AR11 to AR14 shown in FIG. 1 are conceptual diagrams for explaining distances between vectors of nodes (objects), etc. Each of the subspaces AR11 to AR14 is a multidimensional space. For example, the partial space AR11 shown in FIG. 1 is shown in a two-dimensional manner because it is shown on a plane, but it is assumed to be a multi-dimensional space such as 100 or 1000 dimensions.

Each of the subspaces AR11 to AR14 represents a space corresponding to each division (partial vector) of the four-divided vector. Note that, for example, a 100-dimensional vector may be divided into 100 parts. For example, the subspace AR11 is a space (" Also referred to as "first subspace"). A subspace AR11 indicates a subspace (first subspace) corresponding to the division position at the beginning of the vector. In the case of the search query QE1, the partial space AR11 is a space corresponding to the first partial query QE1-1, which is the leading partial vector (first subvector).

Further, the subspace AR12 is a space (“second subspace”) having a dimension corresponding to the second subvector (also referred to as “second subvector”) from the top of the four partial vectors obtained by dividing the vector into four. also called). Subspace AR12 indicates a subspace (second subspace) corresponding to the second division position from the beginning of the vector. In the case of search query QE1, partial space AR12 is a space corresponding to second partial query QE1-2, which is the second partial vector (second sub-vector) from the beginning.

Further, the subspace AR13 is a space (“third subspace”) having a dimension corresponding to the third subvector (also referred to as “third subvector”) from the top of the four partial vectors obtained by dividing the vector into four. also called). Subspace AR13 indicates a subspace (third subspace) corresponding to the third division position from the beginning of the vector. In the case of the search query QE1, the partial space AR13 is a space corresponding to the third partial query QE1-3, which is the third partial vector (third subvector) from the top.

Further, the subspace AR14 is a space (" (also referred to as "fourth subspace"). Subspace AR14 indicates a subspace (fourth subspace) corresponding to the fourth division position from the beginning of the vector. In the case of search query QE1, partial space AR14 is a space corresponding to fourth partial query QE1-4, which is the fourth partial vector (fourth sub-vector) from the top.

In the example of FIG. 1, the partial spaces AR11 to AR14 are shown with similar shapes, but the shapes of the partial spaces AR11 to AR14 may be different, and the division of the regions in the partial spaces AR11 to AR14 may also be different. may Also, it is assumed that the connection relationship by edges between nodes is common in the subspaces AR11 to AR14. That is, the graph GR1 is a graph corresponding to the spatial information SP1 and is common to the subspaces AR11 to AR14.

Also, in the example of FIG. 1, a lookup table is generated for each divided partial vector (sub-vector). For example, clustering is performed for each of four sub-vectors, ie, a first sub-vector, a second sub-vector, a third sub-vector, and a fourth sub-vector, and a lookup table (codebook information) is generated individually.

In FIG. 1, the first subvectors are clustered into nine groups corresponding to the codebooks CD11 to CD19 as shown in the subspace AR11, and the vectors corresponding to each of the codebooks CD11 to CD19 are representative vectors ( centroid). For example, the first subvectors of nodes N7, N9, etc. are shown clustered into the group corresponding to codebook CD11. In this case, the first sub-vectors such as nodes N7 and N9 use the codebook information corresponding to the first sub-vectors (also referred to as "first codebook information") in the distance calculation (calculation), and the codebook CD11 vector quantized to a vector of

The second sub-vectors are clustered into a plurality of groups corresponding to codebooks CD21 to CD24, etc. (see FIGS. 2 and 9). Lloyd). In this case, the second sub-vector of each node is calculated using the codebook information corresponding to the second sub-vector (also referred to as "second codebook information") in the distance calculation (calculation), and the corresponding codebook vector is vector quantized to .

The third sub-vectors are clustered into a plurality of groups corresponding to the codebooks CD31 to CD34 (see FIGS. 2 and 9), and the vector corresponding to each of the codebooks CD31 to CD34 is the representative vector (centroid). calculated as In this case, the third sub-vector of each node is calculated using the codebook information corresponding to the third sub-vector (also referred to as "third codebook information") in the distance calculation (calculation). is vector quantized to .

The fourth subvectors are clustered into a plurality of groups corresponding to codebooks CD41 to CD44, etc. (see FIGS. 2 and 9), and vectors corresponding to each of codebooks CD24 to CD44, etc. are representative vectors (centroids). calculated as In this case, the fourth sub-vector of each node uses the codebook information corresponding to the fourth sub-vector (also referred to as "fourth codebook information") in the distance calculation (calculation) to obtain the corresponding codebook vector is vector quantized to .

It should be noted that the generation of codebook information, such as processing for obtaining representative vectors, is generated using various techniques as appropriate. Since the generation of codebook information is conventional technology, detailed description thereof will be omitted. Also, the lookup table is not limited to the above. For example, one lookup table (codebook information) may be used for the entire divided partial vectors (subvectors), but this point will be described later.

From here, the search processing for the search query QE1 will be described. First, the information processing device 100 acquires the search query QE1 (step S11). For example, the information processing device 100 acquires the search query QE1 from the terminal device 10 (see FIG. 4) used by the user.

Then, the information processing apparatus 100 divides the vector that is the search query QE1 into four. That is, the information processing device 100 divides the search query QE1 into four partial queries. In FIG. 1, the information processing apparatus 100 divides the search query QE1 into a first partial query QE1-1 that is a first sub-vector, a second partial query QE1-2 that is a second sub-vector, a second partial query QE1-2 that is a third sub-vector, a It is divided into four sub-vectors, the three-part query QE1-3 and the fourth sub-vector, the fourth partial query QE1-4. Specifically, the information processing apparatus 100 sets “45, 23, 2 . "98, 110..." is the third partial query QE1-3, and "12, 45, 4..." is the fourth partial query QE1-4.

The information processing apparatus 100 then calculates the distance between each sub-vector of the search query QE1 and the codebook vector corresponding to that sub-vector. The information processing apparatus 100 calculates the distance (difference) between each sub-vector of the search query QE1 and each codebook vector, as shown in the codebook information TB1 to TB4 in FIG. FIG. 2 is a diagram illustrating an example of data according to the first embodiment;

Codebook information TB1 indicates the first codebook information corresponding to the first sub-vector, and the information processing apparatus 100 stores each vector of the codebooks CD11 to CD19 corresponding to the first sub-vector and the first partial query QE1. Calculate the distance between -1. In FIG. 2, the information processing apparatus 100 calculates the distance DS11 as the distance between the codebook CD11 and the first partial query QE1-1, as shown in the codebook information TB1. Similarly, the information processing apparatus 100 calculates the distances between each of the codebooks CD12 to CD14 and the like and the first partial query QE1-1 as distances DS12 to DS14 and the like.

Similarly, each of codebook information TB2-TB4 indicates the second through fourth codebook information corresponding to each of the second through fourth sub-vectors. The information processing apparatus 100 calculates the distances DS21 to DS24 and the like between each of the codebooks CD21 to CD24 and the second partial query QE1-2, as shown in the codebook information TB2. The information processing apparatus 100 calculates the distances DS31 to DS34 and the like between each of the codebooks CD31 to CD34 and the third partial query QE1-3, as shown in the codebook information TB3. The information processing apparatus 100 calculates the distances DS41 to DS44 and the like between each of the codebooks CD41 to CD44 and the fourth partial query QE1-4 as shown in the codebook information TB4.

Although shown abstractly in FIG. 2 for the sake of explanation, the distances DS11 to DS44 and the like are assumed to be specific values. The distance is a value expressed in floating point, but scale-offset-compression ("https://www.unidata.ucar.edu/blogs/ developer/entry/compression_by_scaling_and_offset" etc.) may be compressed to a 1-byte integer type. The information processing apparatus 100 may calculate the distance between each codebook and the query at the timing when the search query QE1 is obtained, or at the timing when the information is required. In the search process, the information processing apparatus 100 uses the codebook information TB1 to TB4 as a lookup table to calculate the distance between each node and the search query QE1, that is, the approximate distance (also referred to as "first distance"). do. In this way, the information processing apparatus 100, when searching a graph in search processing, uses an approximate distance (second 1 distance) to perform processing, efficient search processing can be enabled.

The information processing apparatus 100 executes search processing targeting the search query QE1 (step S12). The information processing apparatus 100 obtains the search result of the search query QE1 by performing the search process as shown in FIG. 11 using the graph GR1 for the search query QE1. Details of the search processing shown in FIG. 11 will be described later. The information processing apparatus 100 performs search processing on the search query QE1, thereby extracting the number of nodes to be searched as nodes in the vicinity of the search query QE1.

In the search processing for the search query QE1, the information processing apparatus 100 selects a predetermined node from among the nodes as a node (hereinafter also referred to as a “starting node”) that serves as a starting point (starting point) for searching the graph GR1. do. For example, the information processing apparatus 100 selects the starting node using a predetermined index such as a tree structure index. In the example of FIG. 1, the information processing apparatus 100 selects the node N7 as the starting node. In order to simplify the explanation, FIG. 1 shows a case where only the node N7 is selected as the starting node using a predetermined index. Alternatively, the starting node may be selected by various methods such as random.

Here, in the search processing for the search query QE1, the information processing apparatus 100 parallelizes the distance between a node (hereinafter also referred to as a “connection node”) to which edges from one node are connected and the search query. Calculate (step S13). In FIG. 1, the information processing apparatus 100, as shown in batch processing information LT1, performs search query QE1 for nodes N9, N12, N54, N85, etc., which are nodes (connection nodes) to which edges from node N7 are connected. is calculated by parallelizing the distances from .

For example, as shown in node information INF1, the node N9 has a codebook CD12 for the first subvector, a codebook CD23 for the second subvector, a codebook CD35 for the third subvector, and a codebook CD47 for the fourth subvector. indicates that it is associated with Note that the size of the variable CD is determined by the size of the codebook, so if the codebook size is 16, for example, the variable CD can be 4 bits, and significant data compression is possible. Therefore, the information processing apparatus 100 calculates the distance between the node N9 and the search query QE1 using the distance DS12 of the codebook CD12, the distance DS23 of the codebook CD23, the distance DS35 of the codebook CD35, and the distance DS47 of the codebook CD47. Calculate For example, the information processing apparatus 100 calculates the sum of the distance DS12 of the codebook CD12, the distance DS23 of the codebook CD23, the distance DS35 of the codebook CD35, and the distance DS47 of the codebook CD47 as the distance between the node N9 and the search query QE1. calculate.

For example, as shown in node information INF2, the node N12 has a codebook CD14 for the first subvector, a codebook CD29 for the second subvector, a codebook CD31 for the third subvector, and a codebook CD45 for the fourth subvector. indicates that it is associated with For example, the information processing apparatus 100 calculates the sum of the distance DS14 of the codebook CD14, the distance DS29 of the codebook CD29, the distance DS31 of the codebook CD31, and the distance DS45 of the codebook CD45 as the distance between the node N12 and the search query QE1. calculate. Similarly, the information processing apparatus 100 uses the node information INF3 of the node N54 to calculate the distance between the node N54 and the search query QE1, and uses the node information INF4 of the node N85 to calculate the distance between the node N85 and the search query QE1. Calculate the distance.

For example, the information processing apparatus 100 collectively calculates the distances between each of the nodes N9, N12, N54, and N85 and the search query QE1 by parallelizing SIMD operations. Thereby, the information processing apparatus 100 can speed up the distance calculation by parallelizing the distance calculation, and can enable efficient search processing.

In order to simplify the explanation, FIG. 1 shows a case in which distances are calculated in parallel for four nodes, but the number of parallelized distances is determined based on the specifications of the information processing apparatus 100. . For example, when the number that the information processing apparatus 100 can collectively process by SIMD (also referred to as “batch-processable unit”) is “4”, the processing is the same as in FIG. When the number (batch processable unit) is “16”, the information processing apparatus 100 parallelizes and calculates distances for 16 nodes. Further, when the collectively processable unit is "32", the information processing apparatus 100 parallelizes and calculates the distances for 32 nodes. In this way, the information processing apparatus 100 enables efficient search processing by collectively performing distance calculations for a number of nodes corresponding to the number of nodes that can be processed collectively by SIMD (batch-processable unit). be able to.

The information processing apparatus 100 parallelizes the distance calculation of a plurality of nodes as described above, and performs the search processing as shown in FIG. are extracted as nodes near the search query QE1.

As described above, the information processing apparatus 100 calculates the approximate distance (first distance) between the vector-quantized vector of each node and the search query, and performs search processing using the first distance. can enable efficient search processing. In addition, the information processing apparatus 100 can perform efficient search processing by collectively performing distance calculations for a parallelizable number of nodes. As described above, the information processing apparatus 100 repeatedly uses a limited memory area (lookup table) by collectively calculating approximate distances between a node of a graph and a plurality of connection nodes. ), so high speed is possible.

In addition, in the example of FIG. 1, the information processing apparatus 100 performs search processing using vectors divided by direct product quantization. Search processing can be enabled. For example, the information processing apparatus 100 calculates the distance between short vectors by direct product quantization, and can reduce the size of the vector that is the target of distance calculation. Further, the information processing apparatus 100 can limit the memory space accessed by the lookup table used for distance calculation to the lookup table of divided sub-vectors. The information processing apparatus 100 can reduce the vector size while suppressing deterioration in search accuracy by direct product quantization. Note that FIG. 1 illustrates the processing in the case where the Cartesian product quantization is performed, but the case where the Cartesian product quantization is performed is only an example, and the information processing apparatus 100 can perform vector processing without performing the Cartesian product quantization. may be used to perform the search process.

[1-1-1. others〕
The processing described above is merely an example, and the information processing apparatus 100 may perform search processing using various information and methods for efficient search processing. Regarding this point, each item will be described in detail.

(Mode of storage of edges (connection nodes))
Regarding the information of edges (connection nodes) connected to each node, the following two storage modes (first storage mode and second storage mode) can be considered for the edge (connection node) storage mode (storage method) at the node. Therefore, the storage mode may be selected according to the usage mode.

For example, as a first storage mode, each node may be associated with an ID of a connection node (object) and stored, and the information of the quantized object may be stored in another table. For example, the data storage modes shown in FIGS. 7 and 8 correspond to the first storage mode. In the case of the first storage mode, memory usage can be reduced.

Also, for example, as a second storage mode, each node has a directly quantized object. In the case of the second storage mode, information obtained by quantizing a connection node (object) of a certain node is stored in association with each node. For example, a data storage mode in which quantized information (node information INF1 to INF4 in FIG. 2) of nodes N9, N12, N54, and N85, which are connection nodes of node N7, are stored in association with node N7. corresponds to the second storage mode. In the case of the second storage mode, since objects are accessed sequentially during retrieval, it is possible to suppress the occurrence of speed reduction.

(lookup table)
In the above example, each divided sub-vector is clustered separately and a lookup table (codebook information) is generated individually. There may be.

For example, all sub-vectors may be clustered to generate one lookup table. In the above example, for example, all four sub-vectors of the first sub-vector, the second sub-vector, the third sub-vector, and the fourth sub-vector may be clustered to generate one lookup table. .

Also, lookup tables may be generated for multiple classes by partially merging the individual subvectors. For example, subvectors with similar variances may be merged to form multiple classes, and lookup tables generated for each of the multiple classes. In the above example, for example, if two sub-vectors of the first sub-vector and the third sub-vector have similar variances, and two sub-vectors of the second sub-vector and the fourth sub-vector have similar variances, then the first The sub-vector and the third sub-vector may be merged into a first class, and the second and fourth sub-vectors may be merged into a second class. In this case, two lookup tables, a first class lookup table (codebook information) and a second class lookup table (codebook information), may be generated.

(transpose)
The holding of the data described above is merely an example, and the information processing apparatus 100 may hold data in various forms so as to enable efficient data reference and the like. For example, the information processing apparatus 100 may hold transposed data. This point will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of data according to the first embodiment;

For example, when data are held as shown in node information INF1 to INF4 and node information INF1 to INF4 are sequentially processed, codebook information TB1 to TB4, which are lookup tables, are repeated as shown in step S14 in FIG. It will be referred to.

Therefore, as shown in step S15, transposed data TR1 to TR4 are generated by transposing the node information INF1 to INF4. For example, the information processing apparatus 100 generates transposed data TR1 to TR4 obtained by transposing the node information INF1 to INF4.

In FIG. 3, first data TR1, which is a list of codebooks corresponding to the first sub-vectors of nodes N9, N12, N54, and N85 among the node information INF1 to INF4, is generated. Specifically, codebook CD12 corresponding to the first subvector of node N9, codebook CD14 corresponding to the first subvector of node N12, codebook CD13 corresponding to the first subvector of node N54, and node N85. First data TR1 is generated which is a list of codebook CD18 corresponding to the first subvector of .

Similarly, second data TR2, which is a list of codebooks corresponding to the second sub-vectors of nodes N9, N12, N54, and N85 among the node information INF1 to INF4, is generated. Further, third data TR3, which is a list of codebooks corresponding to the third sub-vectors of nodes N9, N12, N54, and N85 among the node information INF1 to INF4, is generated. Fourth data TR4, which is a list of codebooks corresponding to the fourth sub-vectors of nodes N9, N12, N54, and N85 among the node information INF1 to INF4, is generated.

Correspondence information is generated that indicates the correspondence between which data in the list of each of the transposed data TR1 to TR4 corresponds to which node. In the example of FIG. 3, in the list of transposed data TR1 to TR4, the first data corresponds to node N9, the second data corresponds to node N12, and the third data corresponds to node N54. Correspondingly, correspondence information is generated indicating that the fourth (last) data corresponds to node N85. The information processing apparatus 100 can specify which node each data in the list of the transposed data TR1 to TR4 corresponds to by referring to the correspondence information. For example, the information processing apparatus 100 generates correspondence information.

The information processing apparatus 100 performs processing using transposed data TR1 to TR4 obtained by transposing the node information INF1 to INF4. For example, when the information processing apparatus 100 refers to the lookup table using the transposed data TR1, it refers only to the codebook information TB1, and refers to only one codebook information TB1. Similarly, when the information processing apparatus 100 refers to the lookup table using the transposed data TR2, it refers only to the codebook information TB2, and refers to only one codebook information TB2. Only one piece of codebook information is similarly referred to for the transposed data TR3 and TR4. As a result, the information processing apparatus 100 can efficiently refer to the data, thereby enabling efficient search processing.

In this way, in order to refer to the lookup table for each subclass (subvector, etc.) during the batch distance calculation, the data of the neighboring objects (connection nodes) of the node are transposed, and the data is put into the node in the order in which it was compiled for each subclass. By having it, the locality of reference is further enhanced, and speedup becomes possible.

In order to simplify the explanation, FIG. 3 shows a case in which transposed data is generated for four nodes. determined by For example, when the batch processable unit of the information processing apparatus 100 is "4", the same processing as in FIG. 3 is performed. Transposed data is generated. Also, when the batch processable unit is "32", the transposed data is generated with 32 nodes as one unit. In this way, the information processing apparatus 100 can efficiently refer to data by using the transposed data generated according to the number of data that can be processed collectively by the information processing apparatus 100 by SIMD (batch processable unit). Therefore, efficient search processing can be enabled.

For example, most of the time during the search processing as described above is for distance calculation, and distance calculation can be speeded up by SIMD-based parallel calculation as described above. When the distance calculation is parallelized in this way, the time to fetch the object data occupies the search processing. If this fetch time can be reduced, further speedup will be realized. There is a method of prefetching to reduce the time to fetch data, but there is a limit.

On the other hand, the information processing apparatus 100 improves the locality of reference as described above and enables efficient data referencing, thereby suppressing data fetching and realizing further speedup. .

(search results)
When returning search results based on the above-described second distance (true distance), the following two methods, a first method and a second method, are conceivable.

For example, as a first method, the number of searches is more than the specified number of searches, and at the end of the search, the true distance calculation is performed, sorted by distance, and the specified number of objects are used as the search results. good. In this case, the information processing apparatus 100 extracts a larger number of nodes (also referred to as an "extended search number") than the designated search number (also referred to as a "first number"). 2”) is set as the number of searches, and by performing the search process as shown in FIG. do. Then, the information processing apparatus 100 calculates the second distance (true distance) for the neighborhood candidate nodes, and selects the first number of nodes from among the neighborhood candidate nodes with the shortest second distance as the neighborhood of the search query. node.

Further, for example, as a second method, based on the first distance (approximate distance) during the search, only when the node (object) enters the search range or the search range, the second distance (true ) and replace the approximate distance with the true distance to return search results based on the true distance.

Here, the search range is defined by "r" in FIG. 11, and the search range is defined by "r(1+ε)" using the search range coefficient "ε" in FIG. Range. For example, when calculating the second distance (true distance) when entering the search range, the information processing apparatus 100 calculates the node A second distance (true distance) of is calculated. Further, when calculating the second distance (true distance) when entering the search range, the information processing apparatus 100 calculates , compute the second distance (true distance) for that node.

For example, when increasing the accuracy, the condition may be the case where the search range is entered. Also, when speeding up the processing is desired, the condition may be that the object is within the search range. Note that the above is only an example, and which condition to use may be appropriately set according to the value of the search range coefficient "ε", the purpose of processing, and the like.

(Vector quantization)
Note that the information processing apparatus 100 may perform vector quantization in two stages, as disclosed in Patent Document 3. FIG. Then, the information processing apparatus 100 may calculate the distance between the search query and each node using the following formula (1).

Here, the value of the left side in the above formula (1) indicates, for example, the squared distance between the search query and the node. Also, for example, "x" in the above formula (1) corresponds to a query. Also, for example, "y" in the above formula (1) corresponds to a node. Also, for example, “q _c (y)” in the right side of the above equation (1) indicates a representative vector (centroid) of “y”. For example, when the information processing apparatus 100 does not have vector data of nodes, the information processing apparatus 100 may use the numerical value of the centroid of the partial area to which each node belongs for "y" in the above equation (1). Also, for example, “y−q _c (y)” indicates a residual vector. Also, for example, “q _p ” in the right side of the above equation (1) indicates a predetermined quantizer (function).

Also, for example, "j" in the right side of the above equation (1) may be the number of divided spaces. For example, in the example of FIG. 1, "j" in the right side of equation (1) above may be the number of divided spaces "4". Also, for example, “u _j ( )” on the right side of the above equation (1) indicates a partial residual vector between vectors in parentheses. For example, the information processing apparatus 100 may calculate the squared distance between the query and the node in each subspace using the above equation (1), and sum the squared distances between the query and the node to calculate the distance between the query and the node. good.

[1-2. Configuration of information processing system]
As shown in FIG. 4 , the information processing system 1 includes a terminal device 10 , an information providing device 50 and an information processing device 100 . The terminal device 10, the information providing device 50, and the information processing device 100 are connected via a predetermined network N so as to be communicable by wire or wirelessly. FIG. 4 is a diagram illustrating a configuration example of an information processing system according to the first embodiment. The information processing system 1 shown in FIG. 4 may include multiple terminal devices 10 , multiple information providing devices 50 , and multiple information processing devices 100 .

The terminal device 10 is an information processing device used by a user. The terminal device 10 accepts various operations by the user. In addition, below, the terminal device 10 may be described as a user. That is, hereinafter, the user can also be read as the terminal device 10 . The terminal device 10 described above is implemented by, for example, a smart phone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like.

The information providing device 50 is an information processing device that stores information for providing various information to users and the like. For example, the information providing device 50 stores object IDs based on character information and the like collected from various external devices such as web servers. For example, the information providing device 50 is an information processing device that provides an image search service to a user or the like. For example, the information providing device 50 stores each information for providing an image search service. For example, the information providing device 50 provides the information processing device 100 with vector information corresponding to images for which an image search service is to be performed. Further, by transmitting a query to the information processing device 100 , the information providing device 50 receives an object ID or the like indicating an image corresponding to the query from the information processing device 100 .

The information processing device 100 is a computer that provides a search service. The information processing apparatus 100 provides a designated search number of objects corresponding to the search query as search results. The information processing apparatus 100 uses a graph in which search target nodes (objects) are connected by edges to provide nodes (objects) near the search query as search results. The information processing apparatus 100 uses a graph in which a group of nodes corresponding to each of a plurality of objects are connected by edges to search for nodes in the vicinity of a search query. and the search query using vector quantized vector information of a plurality of nodes.

For example, upon receiving a query (search query) from the terminal device 10, the information processing device 100 searches for targets (objects) similar to the search query, and provides search results to the terminal device. Further, for example, the data provided by the information processing device 100 to the terminal device may be data itself such as image information, or information for referring to corresponding data such as a URL (Uniform Resource Locator). good too. Also, the search query and search target (object) may be any type of data such as image, voice, and text data.

[1-3. Configuration of Information Processing Device]
Next, the configuration of the information processing apparatus 100 according to the first embodiment will be described using FIG. FIG. 5 is a diagram showing a configuration example of the information processing apparatus 100 according to the first embodiment. As shown in FIG. 5, the information processing apparatus 100 has a communication section 110, a storage section 120, and a control section . The information processing apparatus 100 includes an input unit (for example, a keyboard, a mouse, etc.) that receives various operations from an administrator of the information processing apparatus 100, and a display unit (for example, a liquid crystal display, etc.) for displaying various information. may have.

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 110 is connected to a network (for example, network N in FIG. 4) by wire or wirelessly, and transmits and receives information to and from the terminal device 10 and the information providing device 50 .

(storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 according to the first embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123, and a codebook information storage unit 124, as shown in FIG. .

(Object information storage unit 121)
The object information storage unit 121 according to the first embodiment stores various information about objects. For example, the object information storage unit 121 stores object IDs and vector data. 6 is a diagram illustrating an example of an object information storage unit according to the first embodiment; FIG. The object information storage unit 121 shown in FIG. 6 includes items such as "object ID" and "vector information".

"Object ID" indicates identification information for identifying an object. "Vector information" indicates vector information corresponding to the object identified by the object ID. That is, in the example of FIG. 6, vector data (vector information) corresponding to an object is registered in association with an object ID that identifies the object.

For example, in the example of FIG. 6, the object (target) identified by the object ID "OB1" is associated with multidimensional vector information "10, 24, 51, 2...".

It should be noted that the object information storage unit 121 may store various types of information, not limited to the above, depending on the purpose.

(Graph information storage unit 122)
The graph information storage unit 122 according to the first embodiment stores various kinds of information about graphs. For example, the graph information storage unit 122 stores generated graphs. 7 is a diagram illustrating an example of a graph information storage unit according to the first embodiment; FIG. The graph information storage unit 122 shown in FIG. 7 has items such as "node ID", "object ID", and "connection node information".

"Node ID" indicates identification information for identifying each node (target) in the graph. "Object ID" indicates identification information for identifying an object. When the node ID and the object ID are common, the object ID may be stored in the “node ID” and the item “object ID” may not be included in the graph information storage unit 122 . For example, when used as an object ID and a node ID, the object ID may be stored in the "node ID" and the item "object ID" may not be included in the graph information storage unit 122. FIG.

"Connection node information" indicates information about a node (reference destination node) that can be traced from the corresponding node. For example, "connection node information" includes information such as "reference destination". "Reference destination" indicates information for identifying a reference destination (node) that is connected by an edge and that can be traced from the node. That is, in the example of FIG. 7, a node ID (object ID) that identifies a node is registered in association with a reference destination (node) that can be traced from that node by an edge. The “connection node information” may include information (edge ID) for identifying the edge connected to the reference destination.

The example of FIG. 7 indicates that the node (node N1) identified by the node ID "N1" corresponds to the object (target) identified by the object ID "OB1". Also, from the node N1, an edge is connected to a node (node N4) identified by the node ID "N4", indicating that the node N1 can be traced to the node N4.

It also indicates that the node (node N2) identified by the node ID "N2" corresponds to the object (target) identified by the object ID "OB2". Also, from the node N2, an edge is connected to the node (node N6) identified by the node ID "N6", indicating that the node N2 can be traced to the node N6.

Also, the node (node N7) identified by the node ID "N7" corresponds to the object (target) identified by the object ID "OB7". Further, edges are connected from the node N7 to the nodes N9, N12, N54 and N85, indicating that the nodes N9, N12, N54 and N85 can be traced from the node N2.

It should be noted that the graph information storage unit 122 may store various types of information, not limited to the above, depending on the purpose. For example, the graph information storage unit 122 may store lengths of edges connecting nodes (vectors). That is, the graph information storage unit 122 may store information indicating distances between nodes (vectors). Note that the graph information storage unit 122 may store graph information in various data structures without being limited to the above.

The graph may also include a program module that takes a query as input, searches for nodes by tracing edges in the graph, extracts and outputs nodes that are similar to the query. That is, the graph may be assumed to be used as a program module that performs search processing using the graph. For example, the graph GR1 may be a program that, when vector data is input as a query, extracts nodes corresponding to vector data similar to the vector data from the graph and outputs them. For example, the graph GR1 may be data used as a program module for retrieving similar images corresponding to the query image. For example, graph GR1 causes a computer to extract and output nodes similar to the query in the graph based on an input query.

(Quantization information storage unit 123)
The quantization information storage unit 123 according to the first embodiment stores various information regarding allocation processing. 8 is a diagram illustrating an example of a quantization information storage unit according to the first embodiment; FIG. In the example of FIG. 8, the quantization information storage unit 123 has items such as "node ID", "object ID", and "quantization information".

"Node ID" indicates identification information for identifying each node (target) in the graph. "Object ID" indicates identification information for identifying an object. When the node ID and the object ID are common, the object ID may be stored in the “node ID” and the quantization information storage unit 123 may not include the item “object ID”.

Also, "quantization information" indicates information on the quantized vector of each node (object). For example, "quantization information" includes information such as "element" and "codebook ID". "Element" indicates the placement of the corresponding object in the vector. In the example of FIG. 8, the "elements" include "#1", "#2", "#3", and "#4". In this case, the vector of each node (object) is divided into four, indicating that each divided partial vector is quantized by the codebook. Note that the number of divisions is not limited to 4. For example, when the number of divisions is 6, the "elements" include "#1", "#2", "#3", "#4", "#5", " #6” is included. "Codebook ID" indicates information for identifying the codebook corresponding to each element (partial vector).

In the example of FIG. 8, the vector of node N9 (object OB9) is quantized by a codebook (codebook CD12) in which the leading partial vector among the four partial vectors is identified by codebook ID "CD12". indicates that Further, the vector of node N9 (object OB9) is quantized by a codebook (codebook CD23) in which the second partial vector from the beginning of the four partial vectors is identified by codebook ID "CD23". indicates that

Further, the vector of node N9 (object OB9) is quantized by a codebook (codebook CD35) in which the third partial vector from the beginning of the four partial vectors is identified by codebook ID "CD35". indicates that The vector of node N9 (object OB9) is a codebook (codebook CD47 ) to be quantized.

It should be noted that the quantization information storage unit 123 may store various types of information, not limited to the above, depending on the purpose.

(Codebook information storage unit 124)
The codebook information storage unit 124 according to the first embodiment stores various information about codebooks. For example, the codebook information storage unit 124 stores codebook IDs and vector information of each codebook. 9 is a diagram illustrating an example of a codebook information storage unit according to the first embodiment; FIG. In the example of FIG. 9, the codebook information storage unit 124 stores a lookup table showing the correspondence between each codebook and vector.

The codebook information storage unit 124 stores codebook information TB1 used for quantizing the first partial vector among the partial vectors divided into four, and the second partial vector from the beginning among the partial vectors divided into four. Codebook information TB2 used for quantization is stored. Further, the codebook information storage unit 124 stores codebook information TB3 used for quantizing the third partial vector from the top of the partial vectors divided into four, the fourth partial vector from the top of the four partial vectors, and Stores codebook information TB4 and the like used for quantizing the th (that is, the last) partial vector.

In the example of FIG. 9, the codebook information TB1 includes a codebook (codebook CD11) identified by the codebook ID "CD11", a codebook (codebook CD12) identified by the codebook ID "CD12", and the like. Store codebook information. For example, codebook CD11 indicates that multidimensional vector information of "5, 13..." is associated. Also, the codebook CD12 indicates that multidimensional vector information of "27, 51..." is associated.

Note that the codebook information storage unit 124 may store various types of information, not limited to the above, depending on the purpose. For example, the codebook information storage unit 124 may store information indicating the difference (distance) between each codebook and the search query.

(control unit 130)
Returning to the explanation of FIG. 5 , the control unit 130 is a controller, and for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), etc. control the inside of the information processing apparatus 100 Various programs (corresponding to an example of an information processing program) stored in the storage device are executed by using the RAM as a work area. Also, the control unit 130 is a controller, and is implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 5, the control unit 130 includes an acquisition unit 131, a generation unit 132, a search processing unit 133, and a provision unit 134, and realizes or executes information processing functions and actions described below. do. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 5, and may be another configuration as long as it performs information processing described later.

(Acquisition unit 131)
Acquisition unit 131 acquires various types of information. For example, the acquisition unit 131 acquires various information from the storage unit 120 . For example, the acquisition unit 131 acquires various information from the object information storage unit 121, the graph information storage unit 122, the quantization information storage unit 123, the codebook information storage unit 124, and the like. The acquisition unit 131 also acquires various types of information from an external information processing device. The acquisition unit 131 acquires various types of information from the terminal device 10 and the information providing device 50 .

Acquisition unit 131 acquires a graph. For example, the information processing apparatus 100 may acquire spatial information SP1 in FIG. For example, the information processing device 100 may acquire a graph from an external device such as the information providing device 50 .

The acquisition unit 131 acquires search queries for a plurality of objects to be searched for data. For example, the acquisition unit 131 acquires information regarding the search query QE1. For example, the acquisition unit 131 acquires a search query related to image search. For example, the acquisition unit 131 acquires a query from the terminal device 10 to be used. For example, the acquisition unit 131 acquires a query from the information providing device 50 that has received the query from the terminal device 10 to be used.

(Generating unit 132)
The generator 132 generates various types of information. For example, the generation unit 132 generates various types of information (data) from information (data) stored in the storage unit 120 . For example, the generation unit 132 generates various information from information (data) stored in the object information storage unit 121, the graph information storage unit 122, the quantization information storage unit 123, the codebook information storage unit 124, and the like. .

The generation unit 132 may generate a graph as shown in the graph information storage unit 122. FIG. For example, the generator 132 generates spatial information SP1. The generation unit 132 may also generate information on vector quantization as shown in the quantization information storage unit 123 . For example, the generation unit 132 generates information obtained by vector-quantizing each object such as the node N1 (object OB1). In addition, the generation unit 132 may generate information about codebooks as shown in the codebook information storage unit 124 . The generator 132 may generate a codebook lookup table. For example, the generator 132 generates information about codebooks, such as codebook information TB1 to TB4. When the information processing device 100 acquires the information shown in the graph information storage unit 122, the quantization information storage unit 123, and the codebook information storage unit 124 from an external device such as the information providing device 50, the information processing device 100 The generator 132 may not be provided.

(Search processing unit 133)
The search processing unit 133 provides a search service for objects. The search processing unit 133 searches for various types of information. The search processing unit 133 searches for various types of information. For example, the search processing unit 133 searches for objects by searching the graph. For example, when the query acquired by the acquisition unit 131 is acquired, the search processing unit 133 searches the graph for objects similar to the query. For example, the search processing unit 133 extracts objects similar to the query by searching the graph. For example, the search processing unit 133 extracts objects similar to the query by searching the graph based on the processing procedure shown in FIG. Note that the information processing apparatus 100 may not have the search processing unit 133 when not providing a search service.

The search processing unit 133 selects various types of information in search processing. The search processing unit 133 extracts various types of information in search processing. The search processing unit 133 determines various types of information in search processing. The search processing unit 133 determines various types of information in search processing. The search processing unit 133 changes various information in the search process. The search processing unit 133 updates various information in the search process.

The search processing unit 133 selects a plurality of nodes according to a predetermined criterion in a search process for searching for nodes near the search query using a graph in which a group of nodes corresponding to each of a plurality of objects are connected by edges. and the search query using vector quantized vector information of a plurality of nodes. The search processing unit 133 parallelizes and collectively calculates the distances between a plurality of nodes and the search query. The search processing unit 133 parallel-processes the calculation of the distance between a search query and a plurality of nodes whose batch processing number is determined based on the specifications of the information processing apparatus 100 .

The search processing unit 133 calculates the distance between the plurality of nodes and the search query using a codebook in which each of the plurality of nodes is associated with the corresponding representative vector. The search processing unit 133 calculates the distance between a search query and a plurality of nodes to which edges from one node are connected. The search processing unit 133 searches a plurality of nodes and a search query using node information in which reference information indicating a representative vector associated with each of a plurality of nodes and each of the plurality of nodes is associated with one node and stored. Calculate the distance from

The search processing unit 133 calculates the distance between the plurality of nodes and the search query using the vector information of the plurality of nodes each divided into a plurality of partial vectors by direct product quantization. The search processing unit 133 uses a plurality of codebooks associated with a representative vector corresponding to a vector at each division position of a plurality of partial vectors obtained by dividing each of the plurality of nodes, and a search query and a plurality of nodes. Calculate the distance of The search processing unit 133 uses node information in which reference information indicating a representative vector associated with each of a plurality of nodes and a plurality of partial vectors of each of the plurality of nodes is stored in association with one node. , to calculate the distance between multiple nodes and the search query.

The search processing unit 133 uses reference information in which a list of representative vectors in which each of a plurality of partial vectors of each node is associated with each of a plurality of nodes is used to determine a plurality of nodes and a search query. Calculate the distance of The search processing unit 133 generates transposition information in which representative vectors corresponding to each of the partial vectors of each of the plurality of nodes are arranged in a list for each division position, and correspondence indicating the position to which each of the plurality of nodes corresponds in the list. The distance between the plurality of nodes and the search query is calculated using the reference information including the attached information. The search processing unit 133 calculates the distance between the plurality of nodes and the search query using one codebook in which each of the plurality of partial vectors of each of the plurality of nodes is associated with the corresponding representative vector.

The search processing unit 133 targets predetermined nodes among the nodes to be processed in the search processing. Calculate the distance. The search processing unit 133 extracts, as neighborhood candidate nodes, a second number of nodes that is larger than the first number of nodes extracted as neighborhood nodes of the search query in the search processing, and selects the neighborhood candidate nodes as targets for the second number of nodes. Calculate the distance. The search processing unit 133 extracts, from the neighborhood candidate nodes, the first number of nodes in descending order of the second distance as the neighborhood nodes of the search query.

The search processing unit 133 calculates the second distance for nodes whose first distance is within a predetermined threshold. The search processing unit 133 calculates a second distance for nodes within a search range indicating a target range to be extracted as neighboring nodes. The search processing unit 133 calculates the second distance for the nodes within the search range indicating the target range of the search process.

(Providing unit 134)
The providing unit 134 provides various information. For example, the providing unit 134 transmits various information to the terminal device 10 and the information providing device 50 . For example, the providing unit 134 provides object IDs corresponding to search queries as search results. The providing unit 134 transmits the search result to the terminal device 10 . The providing unit 134 provides the terminal device 10 with the object ID searched by the search processing unit 133 as a search result corresponding to the search query.

Further, the providing unit 134 may provide the information providing device 50 with the object ID searched by the search processing unit 133 . For example, the providing unit 134 provides the information providing device 50 with the object ID extracted by the search processing unit 133 . The providing unit 134 provides the information providing device 50 with the object ID extracted by the search processing unit 133 as information indicating the vector corresponding to the query.

[1-4. Information processing flow]
Next, the procedure of information processing by the information processing system 1 according to the first embodiment will be described with reference to FIG. 10 . FIG. 10 is a flowchart illustrating an example of information processing according to the first embodiment.

As shown in FIG. 10, the information processing apparatus 100 acquires search queries for a plurality of objects to be searched for data (step S101). In the example of FIG. 1, the information processing device 100 acquires the search query QE1.

Then, the information processing apparatus 100 uses a graph in which a group of nodes corresponding to each of a plurality of objects are connected by edges to search for nodes in the vicinity of the search query. is calculated using vector information of a plurality of nodes subjected to vector quantization (step S102). In the example of FIG. 1, the information processing apparatus 100 calculates the distances between the plurality of nodes N9, N12, N54, and N85 and the search query QE1 as vectors of the plurality of nodes N9, N12, N54, and N85 that have undergone vector quantization. Calculate using information.

[1-5. Search processing example]
Here, an example of search processing according to the first embodiment will be described with reference to FIG. 11 as an example. FIG. 11 is a flowchart illustrating an example of search processing according to the first embodiment. The search processing described below is performed by the search processing unit 133 of the information processing apparatus 100 . Also, the object referred to below may be read as a node. In addition, below, the information processing apparatus 100 (search processing unit 133) performs search processing. Note that the information processing apparatus 100 does not need to have the search processing unit 133 when the search service is not provided. The search query for the processing described below may be an additional node, a target node, an object specified by the user, or the like.

Here, the neighborhood object set N(G, y) is the set of neighborhood objects associated by the edge attached to node y. "G" may be predetermined graph data (for example, graph GR1 shown in spatial information SP1, etc.). For example, the information processing apparatus 100 executes k-nearest neighbor search processing.

For example, the information processing apparatus 100 sets the radius r of the hypersphere to ∞ (infinity) (step S300), and extracts the subset S from the existing object set (step S301). For example, the information processing apparatus 100 may extract the object (node) selected as the root node as the subset S. FIG. Also, for example, a hypersphere is a virtual sphere that indicates a search range. The objects included in the object set S extracted in step S301 are also included in the initial set of the object set R of the search results.

Next, the information processing apparatus 100 extracts an object having the shortest distance from the object y, where y is the search query object, among the objects included in the object set S, and sets it as an object s (step S302). For example, when only the object (node) selected as the root node is an element of S, the information processing apparatus 100 extracts the root node as the object s. Next, the information processing apparatus 100 excludes the object s from the object set S (step S303).

Next, the information processing apparatus 100 determines whether or not the distance d(s, y) between the object s and the object y exceeds r(1+ε) (step S304). Here, ε is an expansion factor, and r(1+ε) is a value that indicates the radius of the search range (only nodes within this range are searched; making it larger than the search range increases the accuracy). be. When the distance d(s, y) between the object s and the object y exceeds r(1+ε) (step S304: Yes), the information processing device 100 outputs the object set R as a neighborhood object set of the object y (step S305), the process ends.

If the distance d(s, y) between the object s and the search query object y does not exceed r(1+ε) (step S304: No), the information processing apparatus 100 calculates the neighborhood object set N(G, s) of the object s. One object not included in the object set C is selected from the objects that are elements of , and the selected object u is stored in the object set C (step S306). The object set C is provided for convenience in order to avoid duplicate searches, and is set to an empty set at the start of processing.

Next, the information processing apparatus 100 determines whether or not the distance d(u, y) between the object u and the object y is r(1+ε) or less (step S307). If the distance d(u, y) between the object u and the object y is less than or equal to r(1+ε) (step S307: Yes), the information processing apparatus 100 adds the object u to the object set S (step S308). If the distance d(u, y) between the object u and the object y is not less than r(1+ε) (step S307: No), the information processing apparatus 100 performs determination (processing) in step S309.

Next, the information processing apparatus 100 determines whether or not the distance d(u, y) between the object u and the object y is less than or equal to r (step S309). When the distance d(u, y) between the object u and the object y exceeds r (step S309: No), the information processing apparatus 100 performs determination (processing) of step S315. That is, when the distance d(u, y) between the object u and the object y is not equal to or less than r, the information processing apparatus 100 performs determination (processing) in step S315.

If the distance d(u, y) between the object u and the object y is less than or equal to r (step S309: Yes), the information processing apparatus 100 adds the object u to the object set R (step S310). Then, the information processing apparatus 100 determines whether or not the number of objects included in the object set R exceeds ks (step S311). The predetermined number ks is an arbitrarily determined natural number. For example, ks may be the number of searches or the number of extraction targets. Also, for example, if there is no upper limit on the number of objects to be extracted in a range search or the like, ks may be set to infinity. For example, ks=4 or the like may be used. When the number of objects included in the object set R does not exceed ks (step S311: No), the information processing apparatus 100 performs determination (processing) in step S313.

If the number of objects included in the object set R exceeds ks (step S311: Yes), the information processing apparatus 100 selects the object that is the longest (farthest) from the object y among the objects included in the object set R, It is excluded from the object set R (step S312).

Next, the information processing apparatus 100 determines whether or not the number of objects included in the object set R matches ks (step S313). If the number of objects included in the object set R does not match ks (step S313: No), the information processing apparatus 100 performs determination (processing) in step S315. If the number of objects included in the object set R matches ks (step S313: Yes), the information processing apparatus 100 determines that the object y has the longest (farthest) distance among the objects included in the object set R. The distance between the object and object y is set to a new r (step S314).

Then, the information processing apparatus 100 determines whether or not all objects have been selected from objects that are elements of the neighborhood object set N(G, s) of the object s and stored in the object set C (step S315). . If all the objects that are elements of the neighborhood object set N(G, s) of the object s have not been selected and stored in the object set C (step S315: No), the information processing apparatus 100 proceeds to step S306. and repeat the process.

When all objects are selected from the objects that are elements of the neighborhood object set N(G, s) of the object s and stored in the object set C (step S315: Yes), the information processing apparatus 100 returns the object set S is an empty set (step S316). If the object set S is not an empty set (step S316: No), the information processing apparatus 100 returns to step S302 and repeats the process. If the object set S is an empty set (step S316: Yes), the information processing apparatus 100 outputs the object set R and ends the process (step S317). For example, the information processing apparatus 100 may select an object (node) included in the object set R as a neighboring node corresponding to the additional node (input object y). For example, the information processing apparatus 100 may extract (select) objects (nodes) included in the object set R as neighboring nodes corresponding to the target node (input object y). Further, for example, the information processing apparatus 100 may provide objects (nodes) included in the object set R as a search result corresponding to the search query (input object y) to the terminal device or the like that performed the search.

[2. Second Embodiment]
A second embodiment will now be described. In the second embodiment, neighboring objects are grouped by clustering or the like, and each group (hereinafter also referred to as "blob") is collectively targeted for distance calculation. That is, in the second embodiment, batch distance calculation is performed for each blob. Note that descriptions of the same points as in the first embodiment will be omitted as appropriate. In the second embodiment, the information processing system 1 has an information processing device 100A instead of the information processing device 100. FIG.

[2-1. information processing]
First, an overview of information processing according to the second embodiment will be described with reference to FIG. FIG. 12 is a diagram illustrating an example of information processing according to the second embodiment.

In the example of FIG. 12, it is assumed that the information processing apparatus 100A has acquired a graph GR21 as shown in spatial information SP21. For example, the spatial information SP21 in FIG. 1 may be Euclidean space. For example, the space information SP21 corresponds to the number of dimensions of the vector of the object, and is assumed to be a multi-dimensional space such as 100 or 1000 dimensions. In the example of FIG. 12, for the sake of simplicity, description will be made based on a diagram in which direct product quantization is not performed, but vector (space) direct product quantization may be performed in the same manner as in FIG. .

First, each information shown in FIG. 12 will be described. Dotted lines connecting white circles (o) that are nodes in the spatial information SP21 of FIG. 12 indicate edges connecting the nodes. FIG. 12 shows an example of undirected edges as in FIG. 1, but the edges of the graph GR21 are not limited to undirected edges and may be directed edges. Since notes and edges are the same as in FIG. 1, detailed descriptions thereof are omitted.

In the spatial information SP21 of FIG. 12, the area surrounded by straight lines is a collection (grouping) of nearby objects by clustering or the like, and the area is called a blob. In the example shown below, a blob is the processing unit for batch distance calculation. FIG. 12 shows the case of clustering into 10 regions (blobs) of blobs BL1 to BL10. For example, blob BL1 indicates that it is a blob to which multiple nodes such as nodes N7, N9, N85, and N126 belong. The black dots in the blobs BL1 to BL10 indicate the centroids (representative vectors) of each blob.

Note that the blobs BL1 to BL10 are generated by appropriately using various methods related to clustering and the like. For example, the clustering for classifying into blobs may be k-means clustering. Also, for example, in clustering for classifying into blobs, instead of using the centroid obtained at the center coordinates (average) of each cluster in one iteration (assignment) with k-means in the next iteration, Alternatively, we can replace the centroids with the objects closest to each centroid before the next iteration. In this case, the centroid of the final cluster obtained by k-means is the existing object, ie, the node. As a result, clusters (blobs) are more likely to match edges (nearby nodes) of each node, and search performance can be further improved.

In the example of FIG. 12, the information processing device 100A performs search processing using information of a plurality of blobs BL1 to BL10 classified by clustering a plurality of nodes (objects) as shown in the spatial information SP21. .

From here, the search processing for the search query QE2 will be described. First, the information processing device 100A acquires the search query QE2 (step S21). For example, the information processing device 100A acquires the search query QE2 from the terminal device 10 (see FIG. 4) used by the user.

The information processing device 100A then calculates the distance between the vector of the search query QE2 and the vector of the codebook. Although illustration is omitted, when direct product quantization is not performed, the information processing device 100A calculates the distance (difference) between the vector of the search query QE2 and the vector of the codebook for quantizing the entire vector. . The information processing device 100A stores the distance (difference) between the vector of the search query QE2 and the vector of each codebook as codebook information TB21 in association with information for identifying each codebook. Note that when direct product quantization is performed, the information processing device 100A, similarly to FIG. Calculate the distance (difference) between Calculation of the distance between the search query vector and the codebook vector is the same as in FIGS. 1 and 2, and detailed description thereof will be omitted.

The information processing device 100A executes search processing for the search query QE2 (step S22). The information processing device 100A obtains the search result of the search query QE2 by performing the search process as shown in FIG. 16 using the graph GR21 for the search query QE2. Details of the search processing shown in FIG. 16 will be described later. The information processing apparatus 100A extracts the number of nodes to be searched as nodes in the vicinity of the search query QE2 by performing the search process on the search query QE2.

The information processing apparatus 100A selects a predetermined node from among the nodes as a node (originating node) that serves as a starting point for searching the graph GR21 in the retrieval process for the retrieval query QE2. In the example of FIG. 12, the information processing device 100A selects the node N9 as the starting node. In the example of FIG. 12, the information processing apparatus 100A starts search processing with the node N9 as the processing target, for example.

Here, the information processing apparatus 100A parallelizes and calculates the distances between the search query and a plurality of nodes belonging to the blob to which the node to be processed belongs in the search processing for the search query QE2 (step S23). In FIG. 12, the information processing apparatus 100A, as shown in batch processing information LT2, performs search query QE2 and are parallelized and calculated.

The information processing device 100A calculates the distance between each of the nodes N7, N9, N85, and N126 and the search query QE2 using the distance between each codebook indicated by the codebook information TB21 and the search query QE2. For example, the information processing device 100A refers to the codebook information TB21 and calculates the distance of the codebook corresponding to the vector of the node N7 as the distance between the node N7 and the search query QE2. Note that distance calculation when direct product quantization is performed is the same as in FIGS.

As with the information processing apparatus 100, the batch processable unit is determined based on the specifications of the information processing apparatus 100A. For example, when the number that can be processed collectively by the information processing apparatus 100A by SIMD (batch processable unit) is "4", the distances are parallelized and calculated for four nodes in the same manner as in FIG. For example, the information processing device 100A collectively calculates the distances between each of the nodes N7, N9, N85, and N126 and the search query QE2 by parallelizing SIMD operations. Thereby, the information processing apparatus 100A can speed up the distance calculation by parallelizing the distance calculation, and can enable efficient search processing. Further, the information processing apparatus 100A uses the graph GR21 to trace the nodes (for example, nodes N12 and N64) to which the edges from the node N7 are connected, and target the blobs (for example, blobs BL2 and BL3) to which those nodes belong. Then, batch distance calculation is performed and search processing is performed. Note that the information processing apparatus 100A prevents the same blob from being repeatedly subjected to batch distance calculation in the search process, but the details will be described later.

In order to simplify the explanation, FIG. 12 shows a case where distances are calculated in parallel for four nodes, but the number of parallelized distances is determined based on the specifications of the information processing apparatus 100A. . Since this point is the same as in FIGS. 1, 2, etc., detailed description is omitted. Further, the information processing apparatus 100A may use transposed data as in the example shown in FIG.

As described above, the information processing device 100A calculates the approximate distance (second distance) between the vector-quantized vector of each node and the search query, and performs search processing using the second distance. can enable efficient search processing. In addition, the information processing apparatus 100A can perform efficient search processing by collectively performing distance calculations for a parallelizable number of nodes. As described above, the information processing apparatus 100A can perform efficient search processing by collectively calculating blobs as a unit of collective processing.

In the search process, the information processing apparatus 100A traces the graph GR21 and performs the above-described process on the nodes to be processed, thereby extracting the number of nodes to be searched as nodes in the vicinity of the search query QE2. For example, the information processing device 100A obtains search results by either the first method or the second method described above, similarly to the information processing device 100 .

The processing described above is merely an example, and the information processing apparatus 100A may perform search processing using various information and methods for efficient search processing. Regarding this point, each item will be described in detail. For example, the information processing apparatus 100A performs clustering and generates blobs so that the number of nodes belonging to each blob is the number that the information processing apparatus 100 can collectively process by SIMD (batch processing unit). good too.

Further, the information processing device 100A may use the information about the blob to prevent redundant processing from being performed. The information processing apparatus 100A may have a blob distance calculation flag (hereinafter simply referred to as a "flag") indicating whether the distance calculation for each blob has been performed, and a table indicating to which blob each object belongs. good. In this case, the information processing apparatus 100A may use a flag to manage whether or not each blob has been processed. For example, the information processing apparatus 100A identifies the blob to which the node belongs from the table immediately before processing the neighboring nodes one by one, determines whether the blob has undergone batch distance calculation, and if not, performs batch calculation. and may process individual nodes. This point will also be described in FIGS. 15 and 16. FIG. As a result, the information processing device 100A can prevent redundant processing from being performed.

[2-2. Configuration of Information Processing Device]
Next, the configuration of the information processing apparatus 100A according to the second embodiment will be described using FIG. FIG. 13 is a diagram illustrating a configuration example of an information processing apparatus according to the second embodiment. As shown in FIG. 13, the information processing device 100A has a communication section 110, a storage section 120A, and a control section 130A. In addition, in the information processing apparatus 100A, the description of the points similar to those of the information processing apparatus 100 will be omitted as appropriate.

(Storage unit 120A)
The storage unit 120A is realized by, for example, a semiconductor memory device such as a RAM or flash memory, or a storage device such as a hard disk or an optical disk. 13, the storage unit 120A according to the second embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123A, a codebook information storage unit 124, a blob and an information storage unit 125 .

(Quantization information storage unit 123A)
A quantization information storage unit 123A according to the second embodiment stores various kinds of information regarding allocation processing. 14 is a diagram illustrating an example of a quantization information storage unit according to the second embodiment; FIG. In the example of FIG. 14, the quantization information storage unit 123A has items such as "node ID", "object ID", "blob ID", and "quantization information".

"Node ID" indicates identification information for identifying each node (target) in the graph. "Object ID" indicates identification information for identifying an object. When the node ID and the object ID are common, the object ID may be stored in the "node ID" and the item "object ID" may not be included in the quantization information storage unit 123A.

"Blob ID" indicates information for identifying a blob to which a node (object) belongs.

Also, "quantization information" indicates information on the quantized vector of each node (object). For example, "quantization information" includes information such as "element" and "codebook ID". "Element" indicates the placement of the corresponding object in the vector. In the example of FIG. 14, the "elements" include "#1", "#2", "#3", and "#4". In this case, the vector of each node (object) is divided into four, indicating that each divided partial vector is quantized by the codebook.

The example of FIG. 14 indicates that the node N1 (object OB1) belongs to the blob (blob BL9) identified by the blob ID "BL9". For example, when a vector is divided into four by direct product quantization, the quantization information of node N1 stores information similar to the four codebooks indicating the quantization information of node N1 shown in FIG. Further, for example, when direct product quantization is not performed, information indicating one codebook for quantizing the vector of node N1 is stored in the quantization information of node N1.

Note that the quantization information storage unit 123A may store various types of information, not limited to the above, depending on the purpose.

(Blob information storage unit 125)
A blob information storage unit 125 according to the second embodiment stores information about blobs. For example, the blob information storage unit 125 stores various information for identifying objects associated with each blob. FIG. 15 is a diagram illustrating an example of a blob information storage unit according to the second embodiment; In the example of FIG. 15, the blob information storage unit 125 includes items such as "blob ID", "node ID", and "vector information".

“Blob ID” indicates identification information for identifying a blob. "Node ID" indicates a node (object) associated with the blob identified by the blob ID. "Vector information" indicates the vector information of the blob. For example, "vector information" indicates the vector corresponding to the blob's centroid.

The example shown in FIG. 15 indicates that the nodes (objects) associated with the blob (blob BL1) identified by the blob ID "BL1" are nodes N7, N9, N85, N126, and the like. Blob BL1 indicates that multidimensional vector information of "51, 4, 102, 33..." is associated.

It also indicates that the nodes (objects) associated with the blob (blob BL9) identified by the blob ID "BL9" are nodes N1, N4, N5, and the like. Blob BL9 indicates that multidimensional vector information of "12, 55, 12, 6..." is associated.

Note that the blob information storage unit 125 may store various types of information, not limited to the above, depending on the purpose. The blob information storage unit 125 may store a flag indicating whether or not each blob has been processed for distance calculation. For example, the blob information storage unit 125 stores a value indicating that the blob has not been processed for distance calculation (also referred to as “unprocessed flag value”) and a value indicating that it has been processed for distance calculation (“processed flag value”). flag value”) to the flag value of each blob. For example, the blob information storage unit 125 sets the value of the flag of each blob to a value (for example, 0) indicating that the distance calculation for that blob has not been processed at the start of the search process, and The value of the flag of the blob that has become is changed to a value (for example, 1) indicating that the distance calculation has been completed.

For example, the information processing apparatus 100A refers to the value of the flag of the blob to be processed, and determines whether or not to execute batch distance calculation processing for that blob. The information processing apparatus 100A refers to the value of the flag of each blob, and if the value of the flag of that blob is the unprocessed flag value, determines that the blob has not yet undergone batch distance calculation processing. Perform a batch distance calculation of the nodes belonging to . On the other hand, when the value of the flag of the blob is the processed flag value, the information processing device 100A determines that the batch distance calculation has been completed for the blob, and performs the batch distance calculation of the nodes belonging to the blob. Not performed.

(Control section 130A)
Returning to the description of FIG. 13, the control unit 130A is a controller, and for example, various programs (information processing programs) stored in a storage device inside the information processing apparatus 100A are controlled by a CPU, MPU, GPU, or the like. (equivalent to one example) is implemented by executing the RAM as a work area. Also, the control unit 130A is a controller, and is realized by an integrated circuit such as ASIC or FPGA, for example.

As shown in FIG. 13, the control unit 130A has an acquisition unit 131, a generation unit 132A, a search processing unit 133A, and a provision unit 134, and implements or executes the information processing functions and actions described below. do. Note that the internal configuration of the control unit 130A is not limited to the configuration shown in FIG. 13, and may be another configuration as long as it performs information processing to be described later.

(Generation unit 132A)
The generation unit 132A generates various types of information in the same manner as the generation unit 132 does.

The generation unit 132A may generate information regarding vector quantization as shown in the quantization information storage unit 123 . For example, the generation unit 132 generates information indicating that the node N1 (object OB1) belongs to the blob BL9. In addition, the generation unit 132 may generate information about blobs as shown in the blob information storage unit 125 . For example, the generation unit 132 generates information indicating that nodes N7, N9, N85, N126, etc. belong to the blob BL1. When the information processing apparatus 100 acquires the information shown in the graph information storage unit 122, the quantization information storage unit 123A, the codebook information storage unit 124, and the blob information storage unit 125 from an external device such as the information providing device 50, The information processing apparatus 100 may not have the generation unit 132A.

(Search processing unit 133A)
133 A of search process parts perform the various processes regarding a search process like the search process part 133. FIG.

The search processing unit 133A uses information of a plurality of blobs into which a plurality of objects are classified in the search processing of searching for nodes in the vicinity of the search query for a node group corresponding to each of a plurality of objects. Distances between multiple nodes belonging to the blob and the search query are calculated using vector quantized vector information of the multiple nodes. The search processing unit 133A parallelizes and collectively calculates distances between a plurality of nodes and a search query. The search processing unit 133A parallel-processes the calculation of the distance between a search query and a plurality of nodes whose batch processing number is determined based on the specifications of the information processing device 100A.

The search processing unit 133A calculates the distance between the search query and a plurality of nodes belonging to one blob adjacent to the blob to which the search query applies. The search processing unit 133A calculates the distance between the plurality of nodes and the search query in the search processing for searching for nodes in the vicinity of the search query using a graph in which a plurality of nodes are connected by edges. The search processing unit 133A calculates the distance between a search query and a plurality of nodes belonging to one blob to which a connection node connecting edges from a target node to be processed in the search process belongs.

The search processing unit 133A calculates the distance between a plurality of nodes belonging to one blob other than the blob to which the target node belongs and the search query. The search processing unit 133A uses a blob graph in which each of a plurality of nodes is connected to other blobs other than the blob to which each of the plurality of nodes belongs. Calculate distances between multiple nodes and a search query. In the graph before conversion in which a plurality of nodes are connected by edges, the search processing unit 133A finds a blob to which the node to which the edge is connected from one of the plurality of nodes belongs, other than the blob to which the one node belongs. In a search process for searching for nodes in the vicinity of a search query using a blob graph, which is a transformed graph generated by connecting edges from one node to blobs of Calculate the distance of

The search processing unit 133A calculates the distance between a plurality of nodes belonging to one blob, which is a blob in which edges from a target node to be processed in the search process are connected, and the search query. The search processing unit 133A determines whether one blob is to be processed using information indicating processed blobs that have already been processed. When the one blob is a processed blob, the search processing unit 133A determines that the one blob is not to be processed.

The search processing unit 133A targets a predetermined node among the nodes to be processed in the search processing, and, unlike the first distance, which is a vector-quantized distance, a second distance that is not vector-quantized. Calculate the distance. The search processing unit 133A extracts, as neighborhood candidate nodes, a second number of nodes that is larger than the first number of nodes to be extracted as neighborhood nodes of the search query in the search processing, and selects the neighborhood candidate nodes as targets for the second number of nodes. Calculate the distance. The search processing unit 133A extracts the first number of nodes from the neighborhood candidate nodes with the shortest second distance as nodes in the neighborhood of the search query.

The search processing unit 133A calculates the second distance for nodes whose first distance is within a predetermined threshold. The search processing unit 133A calculates a second distance for nodes within a search range indicating a target range to be extracted as neighboring nodes. The search processing unit 133A calculates a second distance for nodes within a search range indicating the target range of search processing.

[2-3. Search processing example]
Here, an example of search processing according to the second embodiment will be described with reference to FIG. 16 as an example. FIG. 16 is a flowchart illustrating an example of search processing according to the second embodiment. The search processing described below is performed by the search processing unit 133A of the information processing device 100A. Note that the description of the same points as those of the first embodiment, such as FIG. 11, will be omitted as appropriate. For example, in FIG. 16, the same steps as those in FIG. 11 are denoted by the same step numbers, and the description thereof is appropriately omitted.

Here, the neighborhood object set N(G, y) is the set of neighborhood objects associated by the edge attached to node y. "G" may be predetermined graph data (for example, graph GR21 shown in spatial information SP21, etc.). For example, the information processing device 100A executes k-nearest neighbor search processing.

For example, the search process shown in FIG. 16 differs from the search process shown in FIG. 11 in that the process shown in step S304a is performed after step S304. In this way, in the search processing shown in FIG. 16, when a blob that has not yet been accessed is reached in a normal graph search, the distance between the object (node) in that blob and the query is calculated collectively. , to process each node.

Specifically, in the search process of FIG. 16, the information processing apparatus 100A performs all of the following ("-" in step S304a in FIG. (hereinafter referred to as “first processing” to “fourth processing”) are executed (step S304a). In step S304a, the information processing apparatus 100A executes processing (first processing) for collectively calculating the distances of the objects in the blob. Further, in step S304a, the information processing apparatus 100A executes a process (second process) of setting a blob distance calculation flag. Further, in step S304a, the information processing apparatus 100A executes a process (third process) of storing in C all the objects in the blob for which the batch distance calculation has been performed. Further, in step S304a, the information processing apparatus 100A performs the process (fourth process) of step S307 to step S314 one by one with all objects of the blob as u. After that, the information processing apparatus 100A performs the processes after step S306.

[2-4. Modification]
Modifications will now be described. In a modification of the second embodiment, graphs including the concept of blobs may be used. For example, in the modification of the second embodiment, a graph in which blobs are referenced by edges from nodes may be used. Note that descriptions of the same points as those of the first embodiment and the second embodiment will be omitted as appropriate. The information processing apparatus 100A according to the modification has a graph information storage unit 122A instead of the graph information storage unit 122. FIG.

[2-4-1. information processing]
First, with reference to FIG. 17, an outline of information processing according to the modification will be described. FIG. 17 is a diagram illustrating an example of information processing according to a modification;

The information processing device 100A converts the graph GR21 in which the nodes are connected by edges into a graph GR31 in which the edge referenced from the node is a blob (step S31). That is, as shown in FIG. 17, the information processing apparatus 100A generates a graph GR31 that is a graph (blob graph) into which the concept of blobs is introduced from a graph GR21 that is a normal graph.

In the example of FIG. 17, the information processing device 100A generates a graph GR31 including directed edges from nodes to blobs, as indicated by spatial information SP31. In the graph GR31, edges between nodes are deleted, and the graph GR31 includes directed edges from nodes to blobs and does not include edges between nodes. The arrowed lines shown in FIG. 17 indicate directed edges from the node at the base of the arrow to the blob at the tip of the arrow. That is, the arrow line shown in FIG. 17 indicates a directed edge having the node at the arrowhead as the reference source and the blob at the arrowhead as the reference destination. For example, FIG. 17 shows that from node N1, edges are connected to two blobs, blob BL8 and blob BL10.

For example, the information processing device 100A converts the edge of each node from the edge to the connection node (neighboring node) to the edge to the blob to which the connection node belongs. Note that the information processing apparatus 100A does not generate an edge to the blob to which each node belongs. In the example of FIG. 17, the information processing apparatus 100A does not generate an edge from the node N9 to the blob BL1 and an edge from the node N126 to the blob BL1 because the nodes N9 and N126 belong to the same blob BL1. On the other hand, in the example of FIG. 17, the information processing device 100A generates an edge from the node N9 to the blob BL4 and an edge from the node N18 to the blob BL1 because the node N9 and the node N18 belong to different blobs BL1 and BL4. .

In the example of FIG. 17, the information processing device 100A performs search processing using a graph GR31 including directed edges from nodes (objects) to blobs, as indicated by spatial information SP31. For example, the information processing device 100A obtains the search result of the search query QE2 by performing the search process as shown in FIG. 19 using the graph GR21 for the search query QE2. The details of the search processing shown in FIG. 19 will be described later. Note that the search processing in FIG. 17 is common to FIG. 12 in that the search processing is performed using the concept of blobs, and is the same as the search processing shown in FIG. 12 except for processing related to edges.

[2-4-2. graph〕
Next, with reference to FIG. 18, the outline of the graph according to the modification will be described. 18 is a diagram illustrating an example of a graph information storage unit according to a modification; FIG. For example, the graph information storage unit 122A shown in FIG. 18 stores graphs (blob graphs) in which the concept of blobs is introduced. The graph information storage unit 122A shown in FIG. 18 has items such as "node ID", "object ID", and "blob information". Thus, the graph information storage unit 122A according to the modification differs from the graph information storage unit 122 of FIG. 7 in that it has "blob information" instead of "connection node information". Note that the description of the same points as those of the graph information storage unit 122 in FIG. 7 will be omitted as appropriate.

Also, "blob information" indicates information about a blob (referred to blob) that can be traced from the corresponding node. For example, "blob information" includes information such as "reference destination". "Reference destination" indicates information for identifying a reference destination (blob) that is connected by an edge and can be traced from the node. That is, in the example of FIG. 18, a node ID (object ID) that identifies a node is registered in association with a reference destination (blob) that can be traced from that node by an edge. The "blob information" may include information (edge ID) for identifying the edge connected to the reference destination.

The example of FIG. 18 indicates that the node (node N1) identified by the node ID "N1" corresponds to the object (target) identified by the object ID "OB1". Also, an edge is connected from the node N1 to the blob (blob BL8) identified by the blob ID "BL8", indicating that the node N1 can be traced to the blob BL8. Also, an edge is connected from the node N1 to the blob (blob BL10) identified by the blob ID "BL10", indicating that the node N1 can be traced to the blob BL10.

It also indicates that the node (node N2) identified by the node ID "N2" corresponds to the object (target) identified by the object ID "OB2". Also, an edge is connected from the node N2 to the blob (blob BL5) identified by the blob ID "BL5", indicating that the node N1 can be traced to the blob BL5.

It should be noted that the graph information storage unit 122A may store various types of information, not limited to the above, depending on the purpose.

The graph may also include a program module that takes a query as input, searches for nodes by tracing edges in the graph, extracts and outputs nodes similar to the query. That is, the graph may be assumed to be used as a program module that performs search processing using the graph. For example, the graph GR31 may be a program that, when vector data is input as a query, extracts nodes corresponding to vector data similar to the vector data from the graph and outputs them. For example, the graph GR31 may be data used as a program module for retrieving similar images corresponding to the query image. For example, the graph GR31 causes a computer to extract and output nodes similar to the query in the graph based on the input query.

[2-4-3. Search processing example]
From here, an example of search processing according to the modified example will be described using FIG. 19 as an example. FIG. 19 is a flowchart illustrating an example of search processing according to the modification. The search processing described below is performed by the search processing unit 133A of the information processing device 100A. 11, 16, etc., description of the same points as those of the first embodiment or the second embodiment will be omitted as appropriate. For example, in FIG. 19, the same steps as those in FIG. 11 are denoted by the same step numbers, and the description thereof is appropriately omitted.

In FIG. 19, the set N(G, y) differs from the search processing in FIGS. 11 and 16 in that it is a set of blobs associated by edges attached to node y. In FIG. 19, N(G, s) and C are blob sets. 11 is read as "blob set N" in FIG. 19, and "object set C" in FIG. 11 is read as "blob set C" in FIG. . Also, the wording of "object" in the processing related to the change to the blob can be appropriately read as "blob". “G” may be graph (blob graph) data (for example, graph GR31 shown in spatial information SP31, etc.) into which the concept of blob is introduced. For example, the information processing device 100A executes k-nearest neighbor search processing.

For example, in the search processing shown in FIG. 19, the processing in step S306 from FIG. 11 is changed as follows. In the search process of FIG. 19, the information processing device 100A selects one blob that is not included in the blob set C from among N(G, s), stores the selected blob in the blob set C, The objects in the blob are collectively calculated, and the set is defined as B (also referred to as "target blob object set B") (step S306).

19 differs from the search process shown in FIG. 11 in that the process shown in step S306a is performed after step S306. Specifically, in the search process of FIG. 19, the information processing apparatus 100A selects one object u from the target blob object set B (step S306a). After that, the information processing apparatus 100A performs the processes after step S307.

19 differs from the search process shown in FIG. 11 in that the process shown in step S314a is performed before step S315. Specifically, in the search process of FIG. 19, the information processing apparatus 100A determines whether or not all objects have been selected from the target blob object set B (step S314a).

When all the objects in the target blob object set B have been selected (step S314a: Yes), the information processing apparatus 100A performs the process of step S315. On the other hand, if all the objects have not been selected from the target blob object set B (step S314a: No), the information processing apparatus 100A returns to step S306a and repeats the process.

[2-4-4. Other example of generation processing]
In the above example, the case where the edge of each node is converted from the edge to the connection node to the edge to the blob to which the connection node belongs to generate the blob graph, but the information processing apparatus 100A can convert various information. It may be used as appropriate to generate a blob graph. For example, the information processing device 100A may generate a blob graph using a neighborhood search index (also simply referred to as an “index”) that searches for a plurality of objects. It should be noted that descriptions of the same points as those described above will be omitted as appropriate.

An example of generating a blob graph will be described with reference to FIG. FIG. 20 is a diagram illustrating another example of information processing according to the modification. In the following description, a case where a graph is used as an example of an index will be described, but the index may be of any type as long as it searches a plurality of objects. For example, the index may be an index (hash index) using a hash-related description such as a hash table, an index having a tree structure (tree index), or the like.

Information indicating an object or a node corresponding to the object (also referred to as "object node") may be referred to as first information, and information used to classify a plurality of objects may be referred to as second information. Information indicating a blob may be referred to as third information, and a graph in which object nodes are connected by edges may be referred to as fourth information. For example, a graph GR21 in FIG. 20, in which object nodes such as the node N1 are connected by edges, corresponds to the fourth information. Also, unlike the fourth information, the graph GR31 in FIG. 17 and the graph GR32 in FIG. 20 are blob graphs (also referred to as “group graphs”) in which object nodes and blobs are connected by edges. That is, the group graph is a graph in which edges are connected from object nodes to blobs, the object nodes are the reference sources, and the blobs (groups) are the reference destinations.

The information processing device 100A uses, as an index (fourth information), the graph GR21 in which the object nodes are connected by edges to generate a graph GR32, which is a group graph in which blobs are referenced by the edges from the object nodes (step S41). In FIG. 20, the information processing device 100A generates a graph GR32 including directed edges from nodes to blobs, as indicated by spatial information SP32.

For example, the information processing device 100A selects one object from a plurality of objects, searches for nearby objects of the one object using the graph GR21, and selects one object node corresponding to the one object to which the nearby object belongs. Graph GR32 is generated by a connection process that connects edges to other groups. For example, the information processing device 100A randomly selects one object from a plurality of objects, performs connection processing on the selected one object, and generates the graph GR32.

For example, the information processing device 100A performs the following processes (1-1) to (1-4) to generate the graph GR32. Note that the object may be read as a node corresponding to the object.

(1-1): Obtain object NX at random (selection)
(1-2): Constant search for k neighboring objects of object NX (1-3): Obtain blobs to which each neighboring object belongs (1-4): Generate edges to blobs obtained from object NX

For example, in process (1-1), the information processing device 100A randomly acquires one object NX from a plurality of objects stored in the object information storage unit 121. FIG.

For example, in the process (1-2), the information processing apparatus 100A performs the search process as shown in FIG. are extracted as neighboring objects of the object NX.

For example, in the process (1-3), the information processing device 100A refers to the blob information stored in the blob information storage unit 125 and acquires the blob associated with the neighboring object (node) of the object NX.

For example, in processing (1-4), the information processing device 100A generates an edge to the blob acquired from the object NX. For example, the information processing device 100A generates the graph GR32 by associating information indicating the acquired blob with the object NX in the quantization information storage unit 123A.

For example, the information processing device 100A repeats the above processes (1-1) to (1-4) until there are no more objects to be processed, thereby generating the graph GR32. If the blob shown in the spatial information SP21 is generated by searching, the graph GR32 may be generated using the search result when generating the blob. An example of generating blobs by searching will be described later.

For example, the information processing device 100A performs search processing using a graph GR32 including directed edges from nodes (objects) to blobs, as indicated by spatial information SP32. For example, the information processing apparatus 100A obtains the search result of the search query QE2 by performing the search process as shown in FIG. 19 using the graph GR32 for the search query QE2.

Also, in the above example, a case where blobs are generated by clustering has been shown, but blobs may be generated by various methods other than clustering. For example, the information processing device 100A may generate information (third information) indicating a blob using second information such as an index.

The information processing device 100A generates the third information indicating the blob using the graph GR21 as an index as shown in FIG. FIG. 21 is a diagram illustrating an example of blob information generation processing. Spatial information SP20 in FIG. 21 shows a state before blob information in spatial information SP21 is generated, and graph GR21 in FIG. 21 is similar to graph GR21 in FIG. Note that the processing shown in FIG. 21 shows a part of the first method described later, and the details will be described later.

For example, the information processing device 100A generates third information that classifies a plurality of objects into a plurality of blobs by searching using the graph GR21. For example, the information processing device 100A selects one node (also referred to as a “processing target node”) from a plurality of nodes in the graph GR21, and selects at least one of neighboring nodes that are nodes connected to the processing target node by edges. Third information is generated by a classification process of classifying objects corresponding to each of the classification target node, which is a node of the part, and the processing target node into one group. For example, the information processing apparatus 100A selects a node to be processed from a plurality of nodes by a predetermined process, performs classification processing on the selected node to be processed, and generates the third information.

For example, the information processing device 100A may select a node to be processed based on the number of neighboring nodes (connection nodes) connected to the node by edges. For example, the information processing device 100A may select processing target nodes in order from the node with the largest number of neighboring nodes. A method of selecting nodes to be processed in descending order of the number of neighboring nodes and generating third information indicating blobs is also referred to as a “first method”. Further, for example, the information processing apparatus 100A may select the processing target nodes in order from the node with the smallest number of neighboring nodes (connection nodes). A method of selecting nodes to be processed in ascending order of the number of neighboring nodes and generating blobs is also referred to as a “second method”. Further, for example, the information processing device 100A may randomly select a node to be processed. A method of randomly selecting a node to be processed and generating third information indicating a blob is also called a “third method”. Below, an outline of each processing procedure of the first method, the second method, and the third method will be described.

[2-4-4-1. First method]
First, the first method will be explained. For example, in the case of the first method, the information processing device 100A performs the following processes (2-1) to (2-3) to generate third information indicating blobs.

(2-1): Acquire a node ND from a node having a large number of neighboring nodes (number of edges) (2-2): Consider the neighboring node of the node ND and the node ND as one blob (2-3): Delete blob nodes and their edges

For example, the information processing device 100A performs processes (2-1) to (2-3) in all nodes.

For example, in processing (2-1), the information processing device 100A sequentially acquires one node ND by referring to the graph GR21. The information processing device 100A may acquire one node ND using list information of nodes arranged in descending order of the number of neighboring nodes.

For example, in the process (2-2), the information processing device 100A refers to the graph to be processed (graph GR21, etc.) and generates information regarding the neighboring nodes of the node ND and the node ND as one blob. Note that the information processing apparatus 100A may not set all neighboring nodes as blobs, but may set neighboring nodes equal to or smaller than a constant K as blobs. For example, the information processing apparatus 100A may treat K neighboring nodes and the node ND as one blob instead of treating all neighboring nodes of the node ND as a blob together with the node ND.

For example, in the process (2-3), the information processing apparatus 100A deletes the node determined as the blob in the process (2-2) and the edges of the node so that redundant nodes are not selected. For example, the information processing apparatus 100A deletes the node (and the edge) if the blob node is a neighboring node of another node. Note that the above method is only an example, and the process (2-3) is not limited to the method of deleting edges and the like, and any method that can generate a blob may be used. For example, the information processing apparatus 100A may generate a blob by a method of holding a list of nodes that already belong to the blob and managing nodes to be excluded. For example, the information processing apparatus 100A may manage nodes that already belong to the blob by setting a predetermined flag (deletion flag or the like) on the node that is set as the blob. In this way, when using a flag, the information processing apparatus 100A refers to the flag of each node during processing to determine whether the node is to be processed.

A specific example of the first method described above will be described with reference to FIG. In FIG. 21, the information processing device 100A acquires the node N7 having the largest number of neighboring nodes, 4, from the graph GR21 (step S51). In FIG. 21, the node N1 in the graph GR21 also has four neighboring nodes, but the information processing device 100A acquires the node N7. Note that when there are a plurality of nodes having the same number of neighboring nodes, the information processing apparatus 100A may randomly acquire a node from the plurality of nodes.

Then, the information processing apparatus 100A sets the four nodes N9, N12, N54, and N126, which are neighboring nodes of the node N7, and the five nodes N7 as one blob BL11 (step S52). The information processing device 100A generates third information BLT1 indicating that five nodes N7, N9, N12, N54, and N126 belong to one blob BL11. Then, the information processing device 100A deletes the five nodes N7, N9, N12, N54, and N126 and the edges of each node from the graph GR21. As a result, the graph GR21 is updated to the graph GR21-1 as shown in the spatial information SP20-1.

Then, the information processing device 100A acquires the node N1 having four neighboring nodes from the graph GR21 (step S53). Then, the information processing apparatus 100A sets the four nodes N4, N5, N88, and N99, which are neighboring nodes of the node N1, and the five nodes of the node N1 as one blob BL12 (step S54). The information processing device 100A generates third information BLT2 indicating that five nodes N1, N4, N5, N88, and N99 belong to one blob BL12. Then, the information processing device 100A deletes the five nodes N1, N4, N5, N88, and N99 and the edges of each node from the graph GR21-1.

The information processing apparatus 100A repeats the above-described processing until there are no more nodes to be processed, and generates third information indicating a plurality of blobs.

[2-4-4-2. Second method]
Next, the second method will be explained. For example, in the case of the second method, the information processing device 100A performs the following processes (3-1) to (3-3) to generate third information indicating blobs.

(3-1): Acquire a node ND from a node having a smaller number of neighboring nodes (number of edges) (3-2): Consider the neighboring node of the node ND and the node ND as one blob (3-3): Delete blob nodes and their edges

For example, the information processing device 100A performs processes (3-1) to (3-3) in all nodes.

For example, in the process (3-1), the information processing device 100A sequentially acquires one node ND by referring to the graph GR21. The information processing apparatus 100A may acquire one node ND using list information of nodes arranged in ascending order of the number of neighboring nodes.

For example, in the process (3-2), the information processing device 100A refers to the graph to be processed (graph GR21, etc.), and generates information regarding the neighbor node of the node ND and the node ND as one blob. Note that the information processing apparatus 100A may not set all neighboring nodes as blobs, but may set neighboring nodes equal to or smaller than a constant K as blobs. For example, the information processing apparatus 100A may treat K neighboring nodes and the node ND as one blob instead of treating all neighboring nodes of the node ND as a blob together with the node ND.

For example, in the process (3-3), the information processing apparatus 100A deletes the node determined as a blob in the process (3-2) and the edges of the node so that redundant nodes are not selected. For example, the information processing apparatus 100A deletes the node (and the edge) if the blob node is a neighboring node of another node. Note that the second method is the same as the first method, except that the nodes are acquired in order from the one with the smallest number of neighboring nodes (number of edges), and thus a description of specific examples will be omitted. For example, the process (3-3) is not limited to the method of deleting edges or the like, as in the first method, and may be any method as long as it can generate blobs. For example, the information processing apparatus 100A may generate a blob by a method of holding a list of nodes that already belong to the blob and managing nodes to be excluded, as in the first method.

[2-4-4-3. Third method]
Next, the third method will be explained. For example, in the case of the third method, the information processing device 100A performs the following processes (4-1) to (4-3) to generate third information indicating blobs.

(4-1): Sequentially acquire nodes ND at random (4-2): Set neighboring nodes of node ND and nodes ND as one blob (4-3): Delete blob nodes and their edges

For example, the information processing device 100A performs processes (4-1) to (4-3) in all nodes. Note that the third method is the same as the first and second methods except that the order in which nodes are acquired is random, and detailed description thereof will be omitted. For example, the information processing device 100A may generate the third information in a random order (third method) if all edges have the same number.

[2-4-4-4. Fourth method]
Note that the above-described first to third methods are merely examples, and the information processing apparatus 100A may generate blobs through various processes. For example, the information processing device 100A may generate a blob by performing a search using a neighborhood search index (index) for searching a plurality of objects. A method of generating a blob by searching using an index is also called a “fourth method”.

The fourth method will be described below. It should be noted that descriptions of the same points as those described above will be omitted as appropriate. In the following, a case where a graph is used as an example of an index will be described as an example, but as described above, the index may be of any type as long as it searches for a plurality of objects. For example, the index may be any index such as a hash index, tree index, or the like.

From here, a case will be described where the information processing apparatus 100A generates the third information indicating the blob by performing a search using the graph GR21 as shown in FIG. 21 as an index. For example, the information processing device 100A performs the following processes (5-1) to (5-4) to generate the graph GR32.

(5-1): Obtain object NX at random (selection)
(5-2): Search constant k objects near object NX (5-3): Make object NX near object and object NX into one blob (5-4): Execute processing for avoiding overlapping processing

For example, in process (5-1), the information processing device 100A randomly acquires one object NX from the graph GR21.

For example, in the process (5-2), the information processing device 100A performs the search process as shown in FIG. are extracted as neighboring objects of the object NX. Note that the information processing device 100A may scan (search) all objects and extract nearby objects of the object NX.

For example, in the process (5-3), the information processing device 100A generates information regarding the neighboring nodes of the neighboring objects of the object NX and the object NX as one blob.

For example, in the process (5-4), the information processing apparatus 100A executes duplicate process avoidance process so that one object does not belong to a plurality of blobs due to duplicate searches. For example, the information processing device 100A deletes the blob object from the index. For example, the information processing device 100A deletes the blob object from the graph GR21.

Note that the information processing apparatus 100A is not limited to the above, and may perform processing for avoiding duplicate processing by any method as long as one object does not belong to a plurality of blobs. For example, the information processing apparatus 100A may set a predetermined flag (deletion flag or the like) on an object set as a blob to prevent an object that already belongs to the blob from being processed. Thus, when using flags, the information processing apparatus 100A refers to the flags of each object during processing to determine whether the object is to be processed. For example, the information processing device 100A may perform processing for avoiding overlapping processing by a method using a flag when using the search results to generate the above-described group graph.

[2-4-5. Information processing device according to modification]
In the information processing apparatus 100A according to the modified example of the second embodiment, the acquisition unit 131 also performs the following processes. The acquisition unit 131 acquires first information indicating a plurality of objects to be searched for data and second information used to classify the plurality of objects. The acquisition unit 131 acquires second information, which is an index for searching a plurality of objects. Acquisition unit 131 acquires second information, which is a graph in which a plurality of nodes corresponding to each of a plurality of objects are connected by edges. The acquisition unit 131 acquires fourth information, which is an index for searching a plurality of objects. Acquisition unit 131 acquires fourth information, which is a graph in which a plurality of object nodes corresponding to each of a plurality of objects are connected by edges.

In addition, in the information processing device 100A according to the modification of the second embodiment, the generation unit 132A also performs the following processing. The generation unit 132A uses the second information acquired by the acquisition unit 131 to classify a plurality of objects indicated by the first information, and classifies a plurality of objects used for batch processing in search processing targeting a plurality of objects. to generate the third information indicating the group of The generator 132A uses the second information to generate the third information that classifies the plurality of objects into the plurality of groups. The generation unit 132A generates third information for classifying a plurality of objects into a plurality of groups by searching using the second information.

The generation unit 132A selects one node from a plurality of nodes in the graph, selects a node to be classified, which is at least part of neighboring nodes, which are nodes connected to the one node by an edge, and the one node. Third information is generated by classification processing for classifying the object groups corresponding to each into one group. The generation unit 132A generates the third information by a classification process of classifying, into one group, a classification target node, which is a predetermined number of nodes in the neighborhood of one node, and an object group corresponding to each of the one node. Generate. The generation unit 132A generates the third information by a classification process of classifying the classification target nodes, which are all neighboring nodes of one node, and the object groups corresponding to each of the one nodes into one group.

The generation unit 132A selects one node from the plurality of nodes based on the number of neighboring nodes that are nodes connected to each of the plurality of nodes by edges, The third information is generated by classification processing for classifying the object groups corresponding to each into one group. The generation unit 132A selects one node in descending order of the number of neighboring nodes, and classifies the classification target node of the one node and the object group corresponding to each of the one node into one group. Generate third information. The generation unit 132A selects one node in ascending order of the number of neighboring nodes, and classifies the classification target node of the one node and the object group corresponding to each of the one node into one group. Generate third information.

The generation unit 132A randomly selects one node from a plurality of nodes, and classifies the classification target node of the one node and the object group corresponding to each of the one node into one group, thereby generating the third information. to generate The generation unit 132A excludes from the graph a group of processed nodes that are the nodes to be classified and the group of nodes classified into one group, and classifies the group using the graph after excluding the group of processed nodes. The third information is generated by repeating the process. The generation unit 132A generates the third information by repeating the classification process using the graph after excluding the processed node group until the plurality of objects are classified into any group. The generation unit 132A generates third information indicating a plurality of mutually exclusive groups of objects belonging to each.

The generation unit 132A uses the fourth information to generate an edge from an object node, which is a node corresponding to at least a part of objects among a plurality of objects, to a group other than the group to which the object node belongs among the plurality of groups. generates a connected group graph. The generation unit 132A selects one object from a plurality of objects, searches for a neighboring object of the one object in the plurality of objects using the fourth information, and selects an object node corresponding to the one object. , generates a group graph by a linking process that links edges to other groups to which neighboring objects belong. The generation unit 132A randomly selects one object from a plurality of objects, searches for neighboring objects of the one object, and generates an edge from one object node corresponding to the one object to another group to which the neighboring object belongs. A group graph is generated by a concatenation process that concatenates. The generator 132A generates a group graph using the fourth information.

[2-4-6. Information processing flow]
Next, the procedure of information processing according to the modification will be described with reference to FIG. 22 . FIG. 22 is a flowchart illustrating an example of information processing according to the modification.

As shown in FIG. 22, the information processing apparatus 100A acquires first information indicating a plurality of objects to be searched for data (step S401). For example, the information processing device 100A acquires first information indicating a plurality of objects from the object information storage unit 121. FIG.

The information processing device 100A also acquires second information used to classify a plurality of objects (step S402). For example, the information processing device 100A acquires the graph as the second information from the graph information storage unit 122A.

Then, the information processing apparatus 100A uses the second information to classify the plurality of objects indicated by the first information, and indicates a plurality of groups used for batch processing in the search processing targeting the plurality of objects. Third information is generated (step S403). For example, the information processing device 100A generates third information indicating a plurality of blobs whose objects belong to each other and are mutually exclusive.

[3. Third Embodiment]
In the above example, a graph (also called "object graph") in which nodes corresponding to objects (object nodes) are connected by edges, and a graph (blob graph) in which object nodes and groups (blobs) are connected by edges Although two types of graphs have been described, the information processing system 1 may use various types of graphs. For example, unlike an object graph and a blob graph, the information processing system 1 generates a graph connecting groups (blobs) (hereinafter also referred to as a "group connection graph" or "blob connection graph"), and generates a blob connection graph. may be used for searching.

This point will be described below as a third embodiment. In the third embodiment, the information processing system 1 has an information processing device 100B instead of the information processing device 100 or the information processing device 100A. It should be noted that the description of the points similar to those described above in the first embodiment, the second embodiment, etc., will be omitted as appropriate.

The information processing device 100B uses an index (index information) to link edges from a first group (first blob) to which one object belongs to a second group (second blob) to which neighboring objects of one object belong. generate a group-connected graph. In the following, a case where a graph is used as an example of an index will be described as an example, but as described above, the index may be of any type as long as it searches for a plurality of objects. For example, the index may be any index such as a hash index, tree index, or the like.

[3-1. information processing]
First, with reference to FIG. 23, an outline of information processing according to the third embodiment will be described. FIG. 23 is a diagram illustrating an example of information processing according to the third embodiment;

In the example of FIG. 23, it is assumed that the information processing apparatus 100B has acquired a graph GR21 as shown in the spatial information SP21. Note that the information processing apparatus 100B may generate the graph GR21 by appropriately using various techniques related to graph generation. Since the graph GR21 is the same as the graph GR21 shown in FIG. 12 and the like, the explanation is omitted.

The information processing device 100B generates information indicating the blobs BL1 to BL10 and the like by any method. The information processing device 100B may classify each node into one of the blobs BL1 to BL10 by arbitrary clustering such as k-means. Note that the information processing apparatus 100B may generate a blob by any method as long as it can generate a blob. For example, the information processing device 100B may generate information indicating the blobs BL1 to BL10 and the like by any one of the first to fourth methods described above.

The information processing device 100B generates a blob connection graph using the graph GR21 and blob information as indicated in the spatial information SP21 (step S61). In the example of FIG. 23, the information processing device 100B, as shown in the spatial information SP41, creates a graph GR41, which is a blob-connected graph in which a blob (first blob) and another blob (second blob) are connected by edges. to generate The arrow lines shown in FIG. 23 indicate directed edges from the blob at the arrowhead (first blob) to the blob at the arrowhead (second blob). That is, the arrow lines shown in FIG. 23 indicate directed edges having the blob at the arrowhead as the reference source and the blob at the arrowhead as the reference destination. For example, FIG. 23 shows that from blob BL1, edges are connected to five blobs: blob BL2, blob BL3, blob BL4, blob BL5 and blob BL8.

For example, the information processing device 100B searches for the graph GR21 and generates a graph GR41 based on the search result. For example, the information processing device 100B performs the following processes (6-1) to (6-4) to generate a graph GR41, which is a blob connection graph. Note that the following objects may be read as nodes corresponding to the objects.

(6-1): Obtain object NX at random (selection)
(6-2): Search constant k neighboring objects of object NX (6-3): Acquire blobs to which each neighboring object belongs (6-4): Generate edge from blob to which object NX belongs to acquired blob

For example, in process (6-1), the information processing device 100B randomly acquires one object NX from a plurality of objects stored in the object information storage unit 121. FIG.

For example, in the process (6-2), the information processing device 100B performs the search process as shown in FIG. are extracted as neighboring objects of the object NX.

For example, in the process (6-3), the information processing device 100B refers to the blob information stored in the blob information storage unit 125 and acquires the blob associated with the neighboring object (node) of the object NX.

For example, in the process (6-4), the information processing device 100B generates an edge to the blob acquired from the blob to which the object NX belongs. The information processing device 100B generates edges from the blob to which the object NX belongs to the blobs to which all neighboring objects belong. For example, the information processing device 100B associates the information (blob ID) indicating the blob acquired as the reference destination of the blob to which the object NX belongs with the information (blob ID) indicating the blob to which the object NX belongs, and generates the blob connection graph information. By registering in the storage unit 126, a graph GR41 is generated.

For example, the information processing device 100B repeats the above processes (6-1) to (6-4) until there are no more objects to be processed, thereby generating the graph GR41. Note that when blob Y has already been registered as a reference destination of blob X, the information processing apparatus 100B skips registration even if an object belonging to blob Y is searched (extracted) again as a neighboring object of an object belonging to blob X. By doing so, it is possible to prevent the same blob from being redundantly registered as the reference destination of each blob.

For example, the information processing apparatus 100B performs search processing using a graph GR41 connecting blobs with edges, as indicated by spatial information SP41. For example, the information processing apparatus 100B obtains the search result of the search query QE2 by performing the search processing as shown in FIGS. 27 and 28 using the graph GR41, which is the blob connection graph, for the search query QE2.

The method of generating the blob connection graph described above is merely an example, and the information processing apparatus 100B may generate the blob connection graph by various methods. For example, the information processing device 100B may generate a blob connection graph without searching for graphs. When the blobs shown in the spatial information SP21 are generated by searching, a blob connection graph such as the graph GR41 may be generated by using the search results at the time of generating the blobs.

For example, the information processing device 100B may convert the connection relation of object nodes in the graph GR21 into the connection relation of the blobs to which the object nodes belong to generate the blob connection graph. For example, an edge is connected from the node N1 belonging to the blob BL9 to a node N88 belonging to the blob BL8 and a node belonging to the blob BL10, and an edge is connected from the node N4 belonging to the blob BL9 to a node belonging to the blob BL7. . Therefore, the information processing apparatus 100B generates a blob connection graph in which the edges from the blob BL9 to the three blobs of the blob BL7, the blob BL8, and the blob BL10 are connected. In this manner, the information processing apparatus 100B may generate a blob connection graph that connects blobs with edges based on edges that connect object nodes.

[3-2. Configuration of Information Processing Device]
Next, the configuration of the information processing apparatus 100B according to the third embodiment will be described using FIG. FIG. 24 is a diagram illustrating a configuration example of an information processing apparatus according to the third embodiment; As shown in FIG. 24, the information processing device 100B has a communication section 110, a storage section 120B, and a control section 130B. In the information processing device 100B, description of the same points as those of the information processing device 100 or the information processing device 100A will be omitted as appropriate.

(storage unit 120B)
The storage unit 120B is realized by, for example, a semiconductor memory device such as a RAM or flash memory, or a storage device such as a hard disk or an optical disk. 24, the storage unit 120B according to the third embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123, a codebook information storage unit 124, a blob It has an information storage unit 125 and a blob connection graph information storage unit 126 .

(Blob connection graph information storage unit 126)
The blob connection graph information storage unit 126 according to the third embodiment stores various types of information regarding the blob connection graph. For example, the blob connection graph information storage unit 126 stores the generated blob connection graph. 25 is a diagram illustrating an example of a blob connection graph information storage unit according to the third embodiment; FIG. The blob connection graph information storage unit 126 shown in FIG. 25 has items such as "blob ID" and "connected blob information".

“Blob ID” indicates identification information for identifying each blob (group) in the graph. "Connection blob information" indicates information about a blob (reference destination blob) that can be traced from the corresponding blob. For example, "connection blob information" includes information such as "reference destination". "Reference destination" indicates information for identifying a reference destination (blob) that is connected by an edge and can be traced from the blob. That is, in the example of FIG. 25, a blob ID that identifies a blob is registered in association with a reference destination (blob) that can be traced from the blob by an edge. The “connection blob information” may include information (edge ID) for identifying the edge connected to the reference destination.

In the example of FIG. 25, the blob (blob BL1) identified by the blob ID "BL1" is identified by each of the blob IDs "BL2", "BL3", "BL4", "BL5", and "BL8". It shows that the edge is connected to 5 blobs. That is, it indicates that each of the five blobs BL2, BL3, BL4, BL5, and BL8 can be traced from the blob BL1.

Note that the blob connection graph information storage unit 126 is not limited to the above, and may store various types of information depending on the purpose. Also, the blob-connected graph may include a program module that takes a query as input, searches for objects (object nodes) by tracing edges in the blob-connected graph, extracts objects similar to the query, and outputs them. That is, the blob connection graph may be assumed to be used as a program module that performs search processing using the blob connection graph. For example, the graph GR41, which is a blob-connected graph, may be a program that, when vector data is input as a query, extracts objects corresponding to vector data similar to the vector data using the blob-connected graph and outputs them. . For example, the graph GR41 may be data used as a program module for searching for similar images corresponding to the query image. For example, graph GR41 causes a computer to extract and output objects similar to the query in the blob-connected graph based on an input query.

(Control section 130B)
Returning to the description of FIG. 24, the control unit 130B is a controller, and for example, various programs (information processing programs) stored in the storage device inside the information processing apparatus 100B are controlled by the CPU, MPU, GPU, or the like. (equivalent to one example) is implemented by executing the RAM as a work area. Also, the control unit 130B is a controller, and is implemented by an integrated circuit such as ASIC or FPGA, for example.

As shown in FIG. 24, the control unit 130B has an acquisition unit 131, a generation unit 132B, a search processing unit 133B, and a provision unit 134, and implements or executes the information processing functions and actions described below. do. Note that the internal configuration of the control unit 130B is not limited to the configuration shown in FIG. 24, and may be another configuration as long as it performs information processing to be described later.

The acquisition unit 131 according to the third embodiment acquires blob information indicating a plurality of blobs into which a plurality of objects to be searched for is classified, and index information indicating an index for searching the plurality of objects. do. The acquisition unit 131 acquires index information indicating a graph in which a plurality of nodes corresponding to each of a plurality of objects are connected by edges. The acquiring unit 131 acquires blob information indicating a plurality of blobs into which a plurality of objects are classified by clustering processing. The acquiring unit 131 acquires blob information indicating a plurality of blobs whose objects are mutually exclusive. Acquisition unit 131 acquires a query.

(Generating unit 132B)
The generation unit 132B generates various information in the same manner as the generation unit 132 or the generation unit 132A.

The generation unit 132B uses the index information acquired by the acquisition unit 131 to determine the edge from the first blob to which one of the plurality of objects belongs to the second blob to which the neighboring object of the one object belongs. Generates a connected blob-connected graph. The generating unit 132B selects one object from among a plurality of objects, searches the plurality of objects for objects in front of the object using the index information, and selects a neighborhood object from the first blob to which the one object belongs. A blob connectivity graph is generated by connecting the edges to the second blob to which the object belongs.

The generation unit 132B randomly selects one object from a plurality of objects, searches for neighboring objects of the one object, and connects edges from the first blob to which the one object belongs to the second blob to which the neighboring objects belong. generates a blob-connected graph. The generation unit 132B uses the graph to generate a blob connection graph in which edges are connected from the first blob to the second blob.

The generating unit 132B uses the graph to search for nearby objects of one object, and generates a blob connection graph by connecting edges from the first blob to which the one object belongs to the second blob to which the nearby object belongs. do. The generation unit 132B connects the edge from the first blob to the second blob to which the neighboring object corresponding to the neighboring node, which is the node connected by the edge to the one node corresponding to the one object in the graph, belongs, Generate a blob-connected graph.

The generation unit 132B generates a blob connection graph by connecting, from the first blob, a first representative point, which is a representative point of the first blob, and a second representative point, which is a representative point of the second blob, with an edge. to generate The generation unit 132B generates a blob connection graph by connecting the first representative point, which is the centroid of the first blob, and the second representative point, which is the centroid of the second blob, with edges. The generation unit 132B creates a distance between the first representative point, which is the central point calculated based on the objects belonging to the first blob, and the second representative point, which is the central point calculated based on the objects belonging to the second blob. are connected by edges to generate a blob-connected graph.

(Search processing unit 133B)
The search processing unit 133B performs various types of processing related to search processing, like the search processing unit 133 or the search processing unit 133A.

The search processing unit 133B performs search processing using the blob connection graph generated by the generation unit 132B. The search processing unit 133B performs a search process of searching for nearby objects of the search query using the blob connection graph. The search processing unit 133B performs search processing for searching for nearby objects of the search query by tracing edges connecting blobs in the blob connection graph.

In the search processing, the search processing unit 133B uses blob information indicating a plurality of blobs to calculate the distance between the target object, which is an object belonging to one blob, and the search query, and converts the vector information of the vector-quantized target object. calculated using The search processing unit 133B parallelizes and collectively calculates the distance between the target object and the search query. The search processing unit 133B parallel-processes the calculation of the distance between the search query and the target object of the batch processing number determined based on the specifications of the information processing apparatus.

[3-3. Information processing flow]
Next, the procedure of information processing according to the third embodiment will be described with reference to FIG. 26 . FIG. 26 is a flowchart illustrating an example of information processing according to the third embodiment.

As shown in FIG. 26, the information processing apparatus 100B acquires group information indicating a plurality of groups in which a plurality of objects to be searched for data are classified (step S501). For example, the information processing apparatus 100B acquires blob information indicating a plurality of blobs into which nodes (object nodes) corresponding to a plurality of objects to be searched for are classified.

The information processing apparatus 100B acquires index information indicating an index that searches for a plurality of objects (step S502). For example, the information processing apparatus 100B acquires, as an index, a graph in which nodes (object nodes) corresponding to a plurality of objects are connected by edges.

The information processing device 100B uses the index information to create a group connection graph in which edges are connected from a first group to which one of a plurality of objects belongs to a second group to which a neighboring object of the one object belongs. is generated (step S503). For example, the information processing apparatus 100B uses the index information to extract a blob in which edges are connected from a first blob to which one of a plurality of objects belongs to a second blob to which a neighboring object of the one object belongs. Generate a connected graph.

[3-4. Search processing example]
Here, an example of search processing according to the third embodiment will be described with reference to FIGS. 27 and 28 as an example. 27 and 28 are flowcharts showing an example of search processing according to the third embodiment. The search processing described below is performed by the search processing unit 133B of the information processing device 100B. It should be noted that descriptions of the same points as the search processing described in the first embodiment and the second embodiment will be omitted as appropriate.

In FIGS. 27 and 28, N(G, s) and C are sets of blobs (blob sets). Also, "G" may be graph (blob-connected graph) data in which blobs are connected by edges (for example, graph GR41 shown in spatial information SP41, etc.). For example, the information processing device 100B executes k-nearest neighbor search processing.

First, the processing (main processing) shown in FIG. 27 will be described. For example, the information processing apparatus 100B sets the radius r of the hypersphere to ∞ (infinity) and sets the blob set C to an empty set (Φ) (step S601). Then, the information processing apparatus 100B extracts the partial blob set B from the existing blob set (all blobs) (step S602). For example, the information processing device 100B may extract the blob to which the object (node) selected as the root node belongs as the partial blob set B. FIG. Further, for example, the information processing apparatus 100B may extract blobs as the partial blob set B at random.

Then, the information processing apparatus 100B executes determination processing (step S603). The information processing apparatus 100B executes determination processing of steps S701 to S717 shown in FIG. Details of the determination processing shown in FIG. 28 will be described later.

Then, the information processing device 100B sets the partial blob set B to the blob set S (step S604).

The information processing apparatus 100B selects the blob s with the smallest query distance d from among the blobs included in the blob set S (step S605). The “query distance” here is the shortest distance between all objects in the blob and the query object (query). For example, the information processing apparatus 100B selects the blob s from the blob set S that has the shortest query distance to the object y, where y is an object to be used as a query (search query). Note that the query distance is not limited to the above, and may be, for example, the distance between the query and a representative point (centroid or the like) of the blob. The information processing apparatus 100B then excludes the blob s from the blob set S (step S606).

Then, the information processing device 100B determines whether or not the query distance d of the blob s exceeds r(1+ε) (step S607). If the query distance d of the blob s exceeds r(1+ε) (step S607: Yes), the information processing device 100B outputs the object set R as the neighborhood object set of the object y (step S608), and ends the process.

When the query distance d of the blob s does not exceed r(1+ε) (step S607: No), the information processing device 100B determines whether the blob s is included in the blob set C (step S609).

If the blob s is included in the blob set C (step S609: Yes), the information processing apparatus 100B returns to step S605 and repeats the process.

If the blob s is not included in the blob set C (step S609: No), the information processing device 100B adds the blob s to the blob set C (step S610).

Then, the information processing device 100B sets the neighboring blob set N(G, s) of the blob s to the partial blob set B (step S611). A neighboring blob set N(G, s) is, for example, a set of blobs (neighboring blobs) associated with the blob s. For example, the neighborhood blob set N(G, s) is the set of blobs (neighborhood blobs) whose edges from blob s are connected. Then, the information processing device 100B executes determination processing (step S612). Although the details will be described later, for example, the information processing apparatus 100B executes determination processing of steps S701 to S717 shown in FIG. 28 in the same manner as step S603 described above.

Then, the information processing device 100B determines whether or not the blob set S is an empty set (Φ) (step S613). If the blob set S is not an empty set (step S613: No), the information processing apparatus 100B returns to step S605 and repeats the process. If the blob set S is an empty set (step S613: Yes), the information processing device 100B outputs the object set R and ends the process (step S614). For example, the information processing device 100B may select an object (node) included in the object set R as a neighboring node corresponding to the additional node (input object y). For example, the information processing device 100B may extract (select) objects (nodes) included in the object set R as neighboring nodes corresponding to the target node (input object y). Further, for example, the information processing device 100B may provide objects (nodes) included in the object set R as search results corresponding to the search query (input object y) to the terminal device or the like that performed the search.

From here, the processing (determination processing) shown in FIG. 28 will be described. First, the information processing apparatus 100B sets a sub-blob set B in the blob set T (step S701).

Then, the information processing apparatus 100B acquires one blob b from the blob set T, and deletes the blob b from the blob set T (step S702). Then, the information processing apparatus 100B sets min, which is a variable used for determination processing, to ∞ (infinity) (step S703).

Then, the information processing apparatus 100B acquires one object u from the blob b, and deletes the object u from the blob b (step S704).

The information processing apparatus 100B then determines whether or not the distance d(u, y) between the object u and the object y is less than min (step S705).

When the distance d(u, y) between the object u and the object y is less than min (step S705: Yes), the information processing device 100B sets (updates) the value of min to the distance d(u, y). (Step S706), the process of step S707 is performed.

When the distance d(u, y) between the object u and the object y is not less than min (step S705: No), the information processing apparatus 100B performs the process of step S707 without performing the process of step S706.

The information processing device 100B determines whether or not the distance d(u, y) between the object u and the object y is r(1+ε) or less (step S707).

If the distance d(u, y) between the object u and the object y is less than or equal to r(1+ε) (step S707: Yes), the information processing device 100B adds the blob b to the blob set S (step S708), The process of step S709 is performed.

When the distance d(u, y) between the object u and the object y is not less than r(1+ε) (step S707: No), the information processing apparatus 100B does not perform the process of step S708, but performs the process of step S709. .

The information processing device 100B determines whether or not the distance d(u, y) between the object u and the object y is less than or equal to r (step S709).

If the distance d (u, y) between the object u and the object y is less than or equal to r (step S709: Yes), the information processing device 100B adds the object u to the object set R (step S710). process.

If the distance d(u, y) between the object u and the object y is not less than or equal to r (step S709: No), the information processing apparatus 100B performs the process of step S711 without performing the process of step S710.

The information processing device 100B determines whether or not the number of objects included in the object set R exceeds ks (step S711). The predetermined number ks is an arbitrarily determined natural number. For example, ks may be the number of searches or the number of extraction targets. Also, for example, if there is no upper limit on the number of objects to be extracted in a range search or the like, ks may be set to infinity. For example, ks=4 or the like may be used.

When the number of objects included in the object set R exceeds ks (step S711: Yes), the information processing device 100B excludes from the object set R the object farthest from the object y among the objects included in the object set R. (Step S712), the process of step S713 is performed.

When the number of objects included in the object set R does not exceed ks (step S711: No), the information processing apparatus 100B does not perform the process of step S712, but performs the process of step S713.

The information processing device 100B determines whether or not the number of objects included in the object set R matches ks (step S713).

If the number of objects included in the object set R matches ks (step S713: Yes), the information processing device 100B determines the distance between the object farthest from the object y among the objects included in the object set R and the object y. is set to r (step S714), and the process of step S715 is performed.

When the number of objects included in the object set R does not match ks (step S713: No), the information processing apparatus 100B does not perform the process of step S714, but performs the process of step S715.

The information processing apparatus 100B determines whether the blob b is an empty set (Φ) (step S715). If the blob b is not an empty set (step S715: No), the information processing apparatus 100B returns to step S704 and repeats the process.

If the blob b is an empty set (step S715: Yes), the information processing apparatus 100B sets min as the blob query distance of the blob b in the sub-blob set B (step S716). For example, the information processing device 100B sets min as the query distance to be the shortest distance among the distances between all the objects of the blob b and the query.

The information processing device 100B determines whether the blob set T is an empty set (Φ) (step S717). If the blob set T is not an empty set (step S717: No), the information processing apparatus 100B returns to step S702 and repeats the process.

Also, if the blob set T is an empty set (step S717: Yes), the determination process ends.

[3-5. Modification]
The blob connection graph is not limited to the example described above, and may be configured in various manners. For example, the blob-connected graph may be a graph in which representative points of each group (blob) are connected by edges. This point will be described below as a modification of the third embodiment. Note that descriptions of the same points as those of the first, second, and third embodiments will be omitted as appropriate. An information processing apparatus 100B according to the modification of the third embodiment has a blob information storage unit 125A and a blob connection graph information storage unit 126A instead of the blob information storage unit 125 and the blob connection graph information storage unit 126. FIG.

[3-5-1. information processing]
First, with reference to FIG. 29, an overview of information processing according to the modification will be described. FIG. 29 is a diagram illustrating an example of blob connection graph information according to a modification. The example in FIG. 29 shows a case where blobs are generated by clustering.

In the example of FIG. 29, the information processing device 100B generates a blob connection graph in which the centroids of each blob are connected. The information processing apparatus 100B generates a graph GR42, which is a blob-connected graph in which centroids C1 to C10 corresponding to each of the blobs BL1 to BL10 are connected by edges, as shown in the spatial information SP41. The information processing device 100B generates the graph GR42 by appropriately using various conventional techniques related to graph generation.

For example, the information processing apparatus 100B converts the centroid C1 of the blob BL1 into the centroid C2 of the blob BL2, the centroid C3 of the blob BL3, the centroid C4 of the blob BL4, the centroid C7 of the blob BL7, and the centroid C8 of the blob BL8. generate a graph GR42 edge-connected to the five centroids of . Further, for example, in the information processing apparatus 100B, the centroid C2 of the blob BL2 is edge-connected to three centroids, that is, the centroid C1 of the blob BL1, the centroid C3 of the blob BL3, and the centroid C4 of the blob BL4. Generate graph GR42.

In addition, the information processing apparatus 100B may generate the graph GR42 by any method. The information processing device 100B may generate the graph GR42 using various indexes. For example, the information processing device 100B may generate a graph GR42 using a graph GR21 in which object nodes are connected. In this case, the information processing apparatus 100B can generate the graph GR42 by the same processing as the graph GR41 in FIG. 23, so detailed description thereof will be omitted. For example, the information processing device 100B may generate the graph GR42 by the same processing as the example shown in FIG. Graph GR42 shows undirected (bidirectional) edges as an example, but may be directed edges.

In this manner, the information processing apparatus 100B according to the modification generates a graph of centroids as a blob-connected graph. Then, the information processing apparatus 100B performs a normal search by centroid using the generated blob connection graph, and performs distance calculation for each blob. In this manner, the information processing apparatus 100B can simplify processing by representing blobs with centroids. If the blobs are not clustered, the information processing apparatus 100B may calculate the center point based on the objects belonging to each blob and use the calculated center point as the representative point.

In the example of FIG. 29, the information processing apparatus 100B performs search processing using a graph GR42 in which centroids, which are representative points of blobs, are connected by edges as shown in spatial information SP42. For example, the information processing device 100B obtains the search result of the search query QE2 by performing the search processing as shown in FIGS. 27 and 28 using the graph GR21 for the search query QE2. This point is the same as the example described above, and detailed description thereof will be omitted.

[3-5-2. information〕
Next, an outline of information related to the modification will be described with reference to FIGS. 30 and 31. FIG. 30 is a diagram illustrating an example of a blob information storage unit according to a modification; FIG. 31 is a diagram illustrating an example of a blob connection graph information storage unit according to a modification; FIG.

First, the blob information storage unit 125A according to the modification shown in FIG. 30 will be described. A blob information storage unit 125A according to the modification shown in FIG. 30 includes items such as "blob ID", "node ID", "vector information", and "centroid ID". Thus, the blob information storage unit 125A according to the modification differs from the blob information storage unit 125 of FIG. 15 in that it has the "centroid ID". Note that description of the same points as those of the blob information storage unit 125 in FIG. 15 will be omitted as appropriate.

“Centroid ID” indicates identification information for identifying the centroid of each blob. "Vector information" indicates the vector information of the centroid, which is the representative point of the blob.

The example shown in FIG. 30 indicates that the centroid of the blob BL1 identified by the blob ID "BL1" is the centroid (centroid C1) identified by the centroid ID "C1". The centroid of blob BL9 identified by blob ID "BL9" is the centroid (centroid C9) identified by centroid ID "C9".

Note that the blob information storage unit 125A may store various types of information, not limited to the above, depending on the purpose.

Next, the blob connection graph information storage unit 126A according to the modification shown in FIG. 31 will be described. The blob connection graph information storage unit 126A according to the modification shown in FIG. 30 has items such as "centroid ID" and "connected centroid information". Note that the description of the same points as those of the blob connection graph information storage unit 126 in FIG. 25 will be omitted as appropriate.

"Centroid ID" indicates identification information for identifying the centroid that is the representative point of each blob in the graph. "Connected centroid information" indicates information about a centroid (reference destination centroid) that can be traced from the corresponding centroid. For example, "connection centroid information" includes information such as "reference destination". "Reference destination" indicates information for identifying a reference destination (centroid) that is connected by an edge and can be traced from the centroid. That is, in the example of FIG. 31, a centroid ID that identifies a centroid is registered in association with a reference destination (centroid) that can be traced from the centroid by an edge. The "connection centroid information" may include information (edge ID) for identifying the edge connected to the reference destination.

In the example of FIG. 31, from the centroid (centroid C1) identified by the centroid ID "C1", each of the centroid IDs "C2", "C3", "C4", "C7", and "C8" indicates that the edge is connected to five centroids identified by . That is, from centroid C1, each of the five centroids of centroids C2, C3, C4, C7, and C8 can be traced.

Note that the blob connection graph information storage unit 126A is not limited to the above, and may store various types of information depending on the purpose. Also, the blob-connected graph may include a program module that takes a query as input, searches for objects (object nodes) by tracing edges in the blob-connected graph, extracts objects similar to the query, and outputs them. That is, the blob connection graph may be assumed to be used as a program module that performs search processing using the blob connection graph. For example, the graph GR42, which is a blob-connected graph, may be a program that, when vector data is input as a query, extracts objects corresponding to vector data similar to the vector data using the blob-connected graph and outputs them. . For example, the graph GR42 may be data used as a program module for retrieving similar images corresponding to the query image. For example, graph GR42 causes a computer to extract and output objects similar to the query in the blob-connected graph based on an input query.

As described above, in each embodiment and modification, the information processing system 1 performs the following processes. For example, the information processing system 1 generates blobs by vector clustering. For example, the information processing system 1 generates a graph index using the centroids of blobs as nodes.

For example, the information processing system 1 acquires a certain number of arbitrary vectors and performs clustering for direct product quantization. That is, the information processing system 1 divides into partial vectors, performs clustering on a partial vector basis, and generates a codebook.

For example, the information processing system 1 performs Cartesian product quantization on objects belonging to blob units to generate transposed indices. For example, the information processing system 1 generates a graph including edges from objects to blobs (blob graph). For example, the information processing system 1 searches for nearby objects using a general direct product quantization technique. The general Cartesian product quantization method here is, for example, after calculating the distance between the object and all the centroids of the blob and obtaining a certain number of upper blobs, the distance of the object in the blob is Cartesian product quantized may be obtained by obtaining the distance and acquiring the nearby object. As described above, when the centroid graph is generated, the information processing system 1 can search and acquire a certain number of blobs using the graph. Alternatively, the information processing system 1 may generate a graph in which blobs are connected by edges (blob connection graph) instead of the above-described graph including edges from objects to blobs (blob graph).

[4. Fourth Embodiment]
In the above example, the case of using blobs, which are an example of groups for classifying objects, is shown, but the groups for classifying objects are not limited to blobs, and various graphs for classifying objects may be used. For example, in addition to blobs (first group), clusters, which are second groups that are units for quantizing objects, may be used.

This point will be described below as a fourth embodiment. In the fourth embodiment, the information processing system 1 has an information processing device 100C instead of the information processing device 100, the information processing device 100A, or the information processing device 100B. Note that descriptions of points similar to those described above in the first, second, and third embodiments will be omitted as appropriate.

The information processing device 100C stores blob information (first group information) indicating a plurality of blobs that classify a plurality of objects, and cluster information (second group information) indicating a cluster that is a group classified differently from the blobs. Generate.

[4-1. information processing]
First, with reference to FIG. 32, an outline of information processing according to the fourth embodiment will be described. FIG. 32 is a diagram illustrating an example of information processing according to the fourth embodiment; Note that FIG. 32 shows a case where the information processing apparatus 100C classifies objects into a plurality of blobs and classifies them into clusters based on the blobs. will be described later.

First, the information processing device 100C generates blob information that classifies a plurality of objects into a plurality of blobs (step S71). In the example of FIG. 32, the information processing device 100C generates blob information that classifies a plurality of objects into blobs BL1 to BL10 and the like by any method. The information processing device 100C may classify each node into one of the blobs BL1 to BL10 by arbitrary clustering such as k-means. Note that the information processing apparatus 100C may generate the blob by any method as long as the blob can be generated. For example, the information processing device 100C may generate a blob using an index such as a graph. For example, the information processing device 100C may generate information indicating the blobs BL1 to BL10 and the like by any one of the first to fourth methods described above.

In FIG. 32, the information processing device 100C generates the graph GR41, which is a blob-connected graph, as shown in the spatial information SP41, but the graph GR41 may not be generated. For example, when the information processing device 100C generates the graph GR41, the graph GR41 is generated by processing using the same information as in FIG. 23, but the detailed description is omitted because the description is the same as in FIG.

Then, the information processing device 100C generates cluster information for classifying a plurality of blobs into a plurality of clusters (step S72). For example, the information processing device 100C generates cluster information indicating clusters for classifying a plurality of blobs such as blobs BL1 to BL10 by any method. The information processing device 100C may classify the blobs BL1 to BL10 into any one of a plurality of clusters such as blobs BL1 to BL10 by arbitrary clustering such as k-means. Note that the information processing apparatus 100C may generate the blob by any method as long as the blob can be generated. For example, the information processing device 100C may generate a blob using an index such as a graph.

In the example of FIG. 32, the information processing device 100C generates cluster information for classifying blobs BL1 to BL10 into clusters CL1 to CL4. For example, the information processing device 100C classifies the blob BL1 and the blob BL7 into the cluster CL1. That is, the information processing device 100C classifies the object belonging to the blob BL1 and the object belonging to the blob BL7 into the cluster CL1. In FIGS. 32 and 34, the boundaries of the blobs are shown to be on the boundaries of the clusters, but they may not match depending on the division method, and depending on the method, the space may be divided by the boundaries. Not exclusively.

The information processing device 100C also classifies the blob BL2 and the blob BL4 into the cluster CL2. That is, the information processing device 100C classifies the object belonging to the blob BL2 and the object belonging to the blob BL4 into the cluster CL2.

The information processing device 100C also classifies the blob BL3, the blob BL8, and the blob BL10 into the cluster CL3. That is, the information processing device 100C classifies the objects belonging to the blob BL3, the objects belonging to the blob BL8, and the objects belonging to the blob BL10 into the cluster CL3.

The information processing device 100C also classifies the blob BL5, the blob BL6, and the blob BL9 into the cluster CL4. That is, the information processing device 100C classifies the objects belonging to the blob BL5, the objects belonging to the blob BL6, and the objects belonging to the blob BL9 into the cluster CL4.

For example, the information processing device 100C executes processing related to quantization using cluster information. The information processing device 100C performs processing related to quantization for a plurality of objects using cluster information. For example, the information processing device 100C performs quantization for each object belonging to each of the clusters CL1 to CL4, which will be described later.

As described above, the information processing device 100C stores blob information indicating a plurality of blobs that classify a plurality of objects to be searched for data, and cluster information indicating a plurality of clusters classified differently from the plurality of blobs. Generate. Accordingly, the information processing device 100C can generate two types of groups for classifying a plurality of objects, and can generate groups for classifying objects.

[4-1-1. Other processing examples]
Note that the processing shown in FIG. 32 is merely an example, and the information processing apparatus 100C may generate the blob information and the cluster information by any form of processing as long as the information can be generated.

For example, the information processing device 100C may generate blob information to be classified into blobs after generating cluster information to be classified into clusters. In this case, the information processing device 100C may generate cluster information for classifying each object into one of a plurality of clusters by clustering a plurality of objects into clusters. For example, the information processing device 100C may classify each object into one of the clusters CL1 to CL4 by arbitrary clustering such as k-means.

The information processing device 100C may divide at least one of the clusters CL1 to CL4 to generate two or more blobs, thereby generating blob information indicating the blob. For example, the information processing device 100C divides each cluster and generates blobs by clustering the objects belonging to each cluster by arbitrary clustering such as k-means. For example, the information processing device 100C may generate the blob BL1 and the blob BL7 by dividing the cluster CL1 into two blobs. For example, the information processing device 100C clusters the objects belonging to the cluster CL1 with two clusters using k-means, divides the cluster CL1, and generates the blob BL1 and the blob BL7.

For example, the information processing device 100C may generate the blob BL2 and the blob BL4 by dividing the cluster CL2 into two blobs. For example, the information processing device 100C may generate a blob BL3, a blob BL8, and a blob BL10 by dividing the cluster CL3 into three blobs. For example, the information processing device 100C may generate blob BL5, blob BL6 and blob BL9 by dividing cluster CL4 into three blobs.

[4-2. Configuration of Information Processing Device]
Next, the configuration of the information processing apparatus 100C according to the fourth embodiment will be described using FIG. FIG. 33 is a diagram illustrating a configuration example of an information processing apparatus according to the fourth embodiment; As shown in FIG. 33, the information processing device 100C has a communication section 110, a storage section 120C, and a control section 130B. In the information processing device 100C, description of the same points as those of the information processing device 100, the information processing device 100A, or the information processing device 100B will be omitted as appropriate.

(Storage unit 120C)
120 C of memory|storage parts are implement|achieved by semiconductor memory elements, such as RAM and a flash memory, or memory|storage devices, such as a hard disk and an optical disk, for example. 33, the storage unit 120C according to the fourth embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123, a codebook information storage unit 124, a blob It has an information storage unit 125 , a blob connection graph information storage unit 126 , and a cluster information storage unit 127 .

A cluster information storage unit 127 according to the fourth embodiment stores information about clusters that are the second group. The cluster information storage unit 127 stores information on clusters that are units of quantization. The cluster information storage unit 127 may store identification information (cluster ID, etc.) identifying each cluster in association with information (blob ID, etc.) indicating a blob that is the first group belonging to the cluster. Further, the cluster information storage unit 127 may store identification information (cluster ID, etc.) identifying each cluster in association with information (object ID, etc.) indicating an object belonging to the cluster. In addition, the cluster information storage unit 127 associates identification information (cluster ID, etc.) for identifying each cluster with information (base point ID, etc.) for specifying a base point such as the centroid of the cluster and its vector. You can remember.

(Control unit 130C)
The control unit 130C is a controller. For example, various programs (corresponding to an example of an information processing program) stored in a storage device inside the information processing apparatus 100C are operated by the CPU, MPU, GPU, or the like in the RAM. It is realized by being executed as a region. Also, the control unit 130C is a controller, and is realized by an integrated circuit such as ASIC or FPGA, for example.

As shown in FIG. 33, the control unit 130C has an acquisition unit 131, a generation unit 132C, a search processing unit 133C, and a provision unit 134, and implements or executes the information processing functions and actions described below. do. Note that the internal configuration of the control unit 130C is not limited to the configuration shown in FIG. 33, and may be another configuration as long as it performs information processing to be described later.

An acquisition unit 131 according to the fourth embodiment acquires object information indicating a plurality of objects to be searched for data. The acquisition unit 131 acquires index information indicating an index that searches for a plurality of objects.

The acquisition unit 131 selects a plurality of blobs that classify a plurality of objects to be searched for data, and a group that is classified differently from the plurality of blobs, and among the plurality of objects, the plurality of blobs belong to the same blob. Acquire processing information indicating a plurality of clusters, which are groups that classify objects into the same group. The acquisition unit 131 acquires processing information indicating a first number of blobs for classifying a plurality of objects and a cluster for classifying the plurality of objects into a second number smaller than the first number. The acquiring unit 131 acquires processing information indicating a plurality of clusters in which object groups belonging to each of two or more blobs among a plurality of blobs are classified into one cluster.

The acquisition unit 131 acquires processing information indicating a plurality of blobs used for executing a first process, which is batch processing in search processing targeting a plurality of objects. The acquisition unit 131 acquires processing information indicating a plurality of clusters used for performing a second process, which is a process related to quantization targeting a plurality of objects.

The acquiring unit 131 acquires processing information in which a plurality of first identification information identifying each of the plurality of blobs and a plurality of second identification information identifying each of the plurality of clusters are associated with each other. The acquisition unit 131 acquires processing information in which one piece of second identification information is associated with two or more pieces of first identification information.

The acquisition unit 131 acquires processing information associated with the first identification information of the blob to which each of the plurality of objects belongs. The acquisition unit 131 acquires processing information in which quantized object information obtained by quantizing each of a plurality of objects is associated with first identification information. The acquiring unit 131 associates the quantized object information generated by vector quantization that quantizes each partial vector obtained by dividing the vector corresponding to each of the plurality of objects by direct product quantization with the first identification information. Get the processing information received. The obtaining unit 131 acquires processing information associated with the first identification information from the quantized object information generated based on each partial centroid corresponding to each partial area of each centroid corresponding to quantization. get. The acquisition unit 131 obtains a quantized object generated based on residual vectors generated by each centroid corresponding to quantization and each partial centroid corresponding to each partial region of each centroid. The information acquires processing information associated with the first identification information.

(Generation unit 132C)
The generation unit 132C generates various information in the same manner as the generation unit 132, the generation unit 132A, or the generation unit 132B.

Based on the object information acquired by the acquisition unit 131, the generation unit 132C generates blob information indicating a plurality of blobs that classify a plurality of objects. The generation unit 132C generates cluster information indicating a plurality of clusters, which are groups that are classified differently from the plurality of blobs and that classify objects belonging to the same blob among the plurality of objects into the same group. to generate

The generation unit 132C generates blob information for classifying a plurality of objects into a first number of blobs and cluster information for classifying a plurality of objects into a second number of clusters smaller than the first number. The generation unit 132C generates cluster information indicating a plurality of clusters in which object groups belonging to each of two or more blobs among the plurality of blobs are classified into one cluster.

The generation unit 132C generates blob information indicating a plurality of blobs used for batch processing in search processing targeting a plurality of objects. The generation unit 132C generates blob information indicating a plurality of blobs used to collectively calculate distances for a plurality of objects. The generation unit 132C generates blob information indicating a plurality of blobs used for parallel processing of calculating the distance between the search query and the object used in the search process.

The generation unit 132C generates cluster information indicating a plurality of clusters used for performing quantization-related processing on a plurality of objects. The generator 132C generates cluster information indicating a plurality of clusters that are units of quantization. The generation unit 132C generates cluster information indicating a plurality of clusters used for quantizing vectors corresponding to each of a plurality of objects.

The generation unit 132C classifies a plurality of clusters after classifying a plurality of blobs. The generation unit 132C generates blob information indicating a plurality of blobs by clustering a plurality of objects. The generator 132C generates blob information indicating a plurality of blobs using the index information. The generating unit 132C generates cluster information indicating a plurality of clusters using blob information indicating blobs. The generation unit 132C generates cluster information indicating a plurality of clusters by clustering a plurality of blobs.

After classifying a plurality of clusters, the generation unit 132C classifies a plurality of blobs. The generation unit 132C generates cluster information indicating a plurality of clusters by clustering a plurality of objects. The generating unit 132C generates blob information indicating a plurality of blobs using cluster information indicating clusters. The generation unit 132C divides at least one cluster out of the plurality of clusters to generate two or more blobs, thereby generating blob information indicating the plurality of blobs. The generating unit 132C generates first group information indicating a plurality of first groups by clustering objects belonging to each second group and dividing each second group into two or more first groups.

(Search processing unit 133C)
The search processing unit 133C functions as a processing unit that executes a first process for objects belonging to blobs and a second process for objects belonging to clusters. 133 C of search process parts perform the various processes regarding search processing like the search process part 133, the search process part 133A, or the search process part 133B. 133 C of search process parts perform the process using the information produced|generated by the production|generation part 132C.

The search processing unit 133C uses the processing information acquired by the acquisition unit 131 to perform a first process targeting objects belonging to blobs and a second process targeting objects belonging to clusters. The search processing unit 133C executes the second process on the object group belonging to one cluster. The search processing unit 133C executes the first process on a group of objects belonging to one blob.

The search processing unit 133C executes a first process of collectively calculating distances for a group of objects belonging to one blob. The search processing unit 133C executes first processing, which is parallel processing of calculating the distance between each object group belonging to one blob and the search query used in the search processing.

The search processing unit 133C executes the second process on the object group belonging to one cluster. The search processing unit 133C executes a second process of quantization on a group of objects belonging to one cluster. The search processing unit 133C executes a second process of quantizing vectors corresponding to each object group belonging to one cluster.

When two or more blobs belong to one cluster, the search processing unit 133C executes the second process on the object group belonging to the two or more blobs. When two or more blobs belong to one cluster, the search processing unit 133C executes quantization on all objects belonging to the two or more blobs.

The search processing unit 133C refers to the processing information and executes the first process and the second process. The search processing unit 133C refers to the processing information to specify an object to be processed. In the search processing for a plurality of objects, the search processing unit 133C performs the first processing on a blob of one of the first identification information among the two or more pieces of first identification information, and After generating reference information to be referred to in search processing for a cluster of information, first No reference information is generated during processing.

The search processing unit 133C performs search processing using the blob connection graph. The search processing unit 133C performs a search process of searching for nearby objects of the search query using the blob connection graph. The search processing unit 133C performs search processing for searching for nearby objects of the search query by tracing edges connecting blobs in the blob connection graph.

In the search processing, the search processing unit 133C uses blob information indicating a plurality of blobs to calculate the distance between the target object, which is an object belonging to one blob, and the search query, and converts the vector information of the vector-quantized target object. calculated using The search processing unit 133C parallelizes and collectively calculates the distance between the target object and the search query. The search processing unit 133C parallel-processes the calculation of the distance between the search query and the target object of the batch processing number determined based on the specifications of the information processing device.

The search processing unit 133C performs search processing using the blob connection graph and the processing shown in FIGS. 27 and 28. FIG. For example, the search processing unit 133C performs search processing using a graph GR41 connecting blobs with edges, as indicated by spatial information SP51. For example, the search processing unit 133C obtains the search result of the search query QE3 by performing search processing as shown in FIGS. For example, the search processing unit 133C executes the first process for the blob to be processed by tracing the blob connection graph.

The search processing unit 133C generates the reference information for the cluster to which the blob to be processed belongs, if the reference information has not yet been generated. For example, the search processing unit 133C generates a distance lookup table (LUT) as reference information. For example, the search processing unit 133C generates a distance lookup table for each cluster in the search process. That is, in the search processing, the search processing unit 133C generates a distance lookup table for each cluster rather than for each blob. Thereby, the information processing device 100C can suppress an increase in search time.

The search processing unit 133C uses the generated reference information to collectively execute distance calculation for objects belonging to the blob to be processed. For example, the search processing unit 133C does not generate the reference information when the reference information has already been generated for the cluster to which the blob to be processed belongs. Then, the search processing unit 133C uses the already generated reference information to collectively execute the distance calculation for the objects belonging to the blob to be processed.

[4-3. Example of processing and information]
Examples of processing and information used for the processing in the fourth embodiment will now be described.

First, the concept of quantization will be described with reference to FIG. FIG. 34 is a conceptual diagram regarding quantization according to the fourth embodiment.

For example, base point CL1 indicates the base point used for quantization of cluster CL1. In this case, base point CL1 indicates the centroid of cluster CL1, which includes blob BL1 and blob BL7. Also, for example, a base point CL2 indicates a base point used for quantization of the cluster CL2. In this case, base point CL2 indicates the centroid of cluster CL2, which includes blob BL2 and blob BL4.

For example, base point CL3 indicates the base point used for quantization of cluster CL3. In this case, origin CL3 indicates the centroid of cluster CL3, which includes blob BL3, blob BL8 and blob BL10. Also, for example, a base point CL4 indicates a base point used for quantization of the cluster CL4. In this case, origin CL4 indicates the centroid of cluster CL4, which includes blob BL5, blob BL6 and blob BL9.

In this case, the information processing device 100C obtains residual vectors in each of the four clusters CL1 to CL4. Then, the information processing device 100C performs quantization using the residual vectors found in each of the four clusters CL1 to CL4. For example, the information processing device 100C quantizes objects belonging to the cluster CL1 using residual vectors from the base point CL1. Although a detailed description of the quantization using the residual vector is omitted, the information processing device 100C performs quantization using the residual vector by appropriately using the technique disclosed in Patent Document 3, for example.

Note that FIG. 34 shows a state in which a vector is not divided into partial vectors for the sake of illustration, but the information processing device 100C derives a base point for each partial space divided by direct product quantization, Quantization may be performed using residual vectors based on the derived base points. For example, when the information processing device 100C divides a vector into four partial vectors and performs direct product quantization, there is a base point for each subspace corresponding to each partial vector, and using the residual vector from the base point, Quantize.

An example of object quantization and processing by quantization by the information processing apparatus 100C will be described below with reference to FIG. FIG. 34 illustrates a case where the query object (search query) is the search query QE3 as an example. In addition, in the quantization of objects (vectors), direct product quantization, etc., which will be described below, detailed descriptions of the same points as those disclosed in Patent Document 3 will be omitted as appropriate.

As shown in the spatial information SP52 of FIG. 34, the distance between the search query QE3 and each vector corresponding to each object belonging to the cluster CL1 such as the node N7 is different, but the information processing device 100C can perform Do not use a vector corresponding to each object belonging to . In this case, the distance between the search query QE3 and each vector is regarded as, for example, the distance between the search query QE3 and the vector corresponding to the base point CL1 of the cluster CL1. That is, the vectors corresponding to each object belonging to cluster CL1 are quantized into vectors corresponding to base point CL1. Specifically, the information processing device 100C obtains a residual vector that is the difference of each vector from the base point CL1, and quantizes the residual vector by direct product quantization (further divides into subspaces and quantizes).

For example, the information processing device 100C further divides each cluster into a plurality of partial areas, and generates partial centroids corresponding to the plurality of partial areas as a codebook. For example, the information processing device 100C may generate partial centroids corresponding to a plurality of partial regions into which each cluster is divided, based on residual vector information regarding each cluster.

For example, the information processing device 100C may determine the number of partial areas in each cluster based on the number of objects included in each cluster, or may determine the number of partial areas based on a predetermined set value. good too. For example, the information processing device 100C may appropriately use various conventional techniques regarding clustering to determine the partial regions. For example, the information processing device 100C calculates a residual vector from the object included in each partial area of the cluster CL1 and the base point CL1 of the cluster CL1, and generates the partial centroid of the partial area from the residual vector. For example, the information processing device 100C uses residual vector information between the base point of the cluster and the partial centroid of each partial area. For example, the information processing device 100C uses information of the residual vector between the base point CL1 of the cluster CL1 and the partial centroid of each partial area thereof. For example, the information processing device 100C stores information on the partial centroid of the partial area to which the object belongs in association with each object.

For example, the information processing device 100C can quantize the position of each object to the position of the centroid of the partial area to which the vector belongs, by using the information described above. Thereby, the information processing device 100C can refine the distance between the search query QE3 and each object to the distance between the search query QE3 and the vector corresponding to the partial centroid of the partial area to which each object belongs.

For example, the information processing device 100C uses information indicating the vector of the base point of each cluster and the residual vector related to the partial centroid. Thereby, the information processing device 100C can calculate the distance to the partial centroid for the search query QE3, for example. Therefore, the information processing device 100C can, for example, set the distance between the search query QE3 and each object to a partial centroid that is closer than the base point of the cluster.

The information processing device 100C may divide the vector of each object into a plurality of partial vectors for processing. That is, the information processing device 100C may perform processing using a technique related to so-called direct product quantization. Since this point is the same as that of the above-described first embodiment and the like, redundant description will be omitted as appropriate.

For example, in the case of FIG. 34, the information processing device 100C divides the vector of the search query QE3 into a plurality of partial vectors (also referred to as "partial vectors of the search query QE3"). In this case, the spatial information SP52 is also divided into a plurality of partial spaces (also referred to as "partial spaces of the spatial information SP52"). For example, the information processing device 100C may divide the vector of the search query QE3 into four as in FIG. In this case, the spatial information SP52 is also divided into four subspaces and each cluster is divided into a plurality of subspaces.

For example, the information processing device 100C stores information indicating partial centroids (codebooks) corresponding to areas to which each object belongs in four partial spaces, in association with each object. For example, the information processing device 100C calculates the distance between each object and the search query QE3 by totaling the distances between the four codebooks corresponding to each object and the search query QE3.

Next, an outline of the transposed index will be described with reference to FIGS. 35 and 36. FIG. FIG. 35 is a diagram illustrating an example of a transposed index according to the fourth embodiment; FIG. 36 is a diagram showing a specific example of transposed indexes according to the fourth embodiment.

First, the transposed index IND1 shown in FIG. 35 will be described. The transposed index IND1 shown in FIG. 35 includes "blob ID", "cluster ID", "number of quantized objects", "quantized object #1", "quantized object #2", "quantized object #3", and "quantized object #4". In FIG. 35, four objects from "quantization object #1" to "quantization object #4" are illustrated, but "quantization object #5", "quantization object #6", and "quantization object #6" are shown in transposition index IND1. , etc., corresponding to the number of objects belonging to each blob.

"Blob ID" indicates identification information for identifying a blob that is the first group. "Cluster ID" indicates identification information for identifying a cluster that is the second group. "Number of quantized objects" indicates the number of objects belonging to the blob. "Quantized object #1" to "quantized object #4" indicate quantized objects. For example, "quantization object #1" to "quantization object #4" store a list of information indicating codebooks corresponding to partial vectors obtained by direct product quantization of objects.

In the example of FIG. 35, the blob with the blob ID "1" belongs to the cluster with the cluster ID "2", indicating that the number of belonging objects is four. Also, for the blob with the blob ID "1", "4, 2, 8, 10" is stored in the "quantization object #1", the object is composed of four partial vectors, and the codebook "4 ”, “2”, “8”, and “10”. For example, for the object corresponding to “quantized object #1” with blob ID “1”, the first partial vector among the four partial vectors is quantized with codebook “4”, and the second partial vector is quantized with codebook “4”. It indicates that the partial vector is quantized with codebook '2', the third partial vector is quantized with codebook '8', and the last partial vector is quantized with codebook '10'.

Note that the transposed index IND1 is not limited to the above, and may store various information depending on the purpose. For example, the transposed index IND1 may store information indicating each quantized object (such as an object ID) in association with each quantized object.

Next, the transposed index IND2 shown in FIG. 36 will be described. FIG. 36 shows a specific example of the transposed index corresponding to the example shown in FIG. 32 and the like. The description of the same points as the transposed index IND1 will be omitted as appropriate. The transposed index IND2 shown in FIG. 36 includes items such as "blob ID", "cluster ID", "number of quantized objects", and "quantized object #1". Although FIG. 36 shows only "quantized object #1", it contains items corresponding in number to the number of objects belonging to each blob, similar to the transposed index IND1.

Similar to the spatial information SP51 shown in FIG. 32, the transposed index IND2 indicates that the blob BL1 identified by the blob ID "BL1" belongs to the cluster CL1 with the cluster ID "CL1". It also indicates that the number of objects belonging to the blob BL1 is "NM1". In FIG. 36, an abstract code such as "NM1" is used, but in the "number of quantized objects", a value indicating a specific number (for example, 10, 100, etc.) is stored in the same manner as the transposed index IND1. .

In addition, blob BL1 stores "CD51, CD64, CD77, CD82" in "quantization object #1", and the object is composed of four partial vectors, which are codebooks "CD51", "CD64", and "CD64", respectively. It indicates that it is quantized with "CD77" and "CD82".

32, the transposed index IND2 indicates that the blob BL2 identified by the blob ID "BL2" belongs to the cluster CL2 with the cluster ID "CL2". 32, the transposed index IND2 indicates that the blob BL7 identified by the blob ID "BL7" belongs to the cluster CL1 with the cluster ID "CL1".

Note that the transposed index IND2 is not limited to the above, and may store various information depending on the purpose. For example, the transposed index IND2 may store information indicating each quantized object (such as an object ID) in association with each quantized object. In this way, information (cluster ID) for identifying a cluster is added to the transposed index, and the information processing device 100C calculates a residual vector using the transposed index. For example, the residual vector is used during index generation and retrieval, but since the process is the same as in the prior art such as Patent Document 3, detailed description thereof will be omitted as appropriate.

[4-4. Information processing flow]
Next, the procedure of information processing according to the fourth embodiment will be described with reference to FIGS. 37 and 38. FIG. 37 and 38 are flowcharts showing an example of information processing according to the fourth embodiment.

First, FIG. 37 will be described. For example, FIG. 37 is a flowchart illustrating an example of group generation processing. As shown in FIG. 37, the information processing device 100C acquires object information indicating a plurality of objects to be searched for data (step S601). For example, the information processing device 100</b>C acquires object information indicating a plurality of objects from the object information storage unit 121 .

The information processing device 100C generates first group information indicating a plurality of first groups for classifying a plurality of objects based on the object information (step S602). For example, the information processing device 100C generates blob information indicating a plurality of blobs classifying a plurality of objects based on the object information.

The information processing device 100C is a group that is classified differently from the plurality of first groups, and is a group that classifies objects belonging to the same first group among the plurality of first groups into the same group. Second group information indicating a plurality of second groups is generated (step S603). For example, the information processing device 100C generates cluster information indicating a plurality of clusters, at least one of which includes two or more blobs. Note that the information processing device 100C may generate cluster information indicating a cluster corresponding to the blob on a one-to-one basis.

Next, FIG. 38 will be described. For example, FIG. 38 is a flowchart showing an example of processing using groups. As shown in FIG. 38, the information processing device 100C includes a plurality of first groups that classify a plurality of objects to be searched for data, and a group that is classified differently from the plurality of first groups. Among objects, processing information indicating a plurality of second groups, which are groups in which objects belonging to the same first group in a plurality of first groups are classified into the same group, is acquired (step S701). For example, the information processing device 100C acquires the blob information stored in the blob information storage unit 125 and the cluster information stored in the cluster information storage unit 127 as processing information. For example, the information processing device 100C acquires the transposed index IND1 or the transposed index IND2.

The information processing apparatus 100C uses the processing information to perform a first process for objects belonging to the first group and a second process for objects belonging to the second group (step S702). For example, the information processing device 100C performs batch processing on objects belonging to blobs, and performs quantization on objects belonging to clusters.

[5. effect〕
As described above, the information processing apparatus 100C according to the fourth embodiment has the acquisition unit 131 and the generation unit 132C. The acquisition unit 131 acquires object information indicating a plurality of objects to be searched for data. Based on the object information acquired by the acquisition unit 131, the generation unit 132C generates first group information indicating a plurality of first groups for classifying a plurality of objects, and a group classified differently from the plurality of first groups. and generating second group information indicating a plurality of second groups, which are groups in which objects belonging to the same first group in the plurality of first groups are classified into the same group among the plurality of objects.

As described above, the information processing apparatus 100C according to the fourth embodiment provides first group information indicating a plurality of first groups for classifying a plurality of objects to be searched for data, and the plurality of first groups. By generating second group information indicating a plurality of second groups to be classified, two types of groups for classifying a plurality of objects can be generated, and groups for classifying objects can be generated.

The generation unit 132C generates first group information for classifying a plurality of objects into a first group of a first number, and second group information for classifying a plurality of objects into a second group of a second number smaller than the first number. to generate

Thus, the information processing apparatus 100C can generate two types of groups each having a different number of groups, and can generate groups for classifying objects.

The generating unit 132C generates second group information indicating a plurality of second groups in which object groups belonging to each of two or more first groups among the plurality of first groups are classified into one second group.

Thus, the information processing apparatus 100C can generate two types of groups having an inclusive relationship, and can generate groups for classifying objects.

The generation unit 132C generates first group information indicating a plurality of first groups used for batch processing in search processing targeting a plurality of objects.

In this way, the information processing device 100C can generate two types of groups including the first group used for batch processing in search processing, and can generate groups for classifying objects.

The generation unit 132C generates first group information indicating a plurality of first groups used to collectively calculate distances for a plurality of objects.

In this way, the information processing device 100C can generate two types of groups including the first group used to collectively calculate distances, and can generate groups for classifying objects.

The generation unit 132C generates first group information indicating a plurality of first groups used for parallel processing of calculating the distance between the search query and the object used in the search process.

In this way, the information processing device 100C can generate two types of groups including the first group used for parallel processing of calculating the distance between the search query used in the search process and the object. Groups can be created to classify.

The generation unit 132C generates second group information indicating a plurality of second groups used for performing processing related to quantization for a plurality of objects.

In this way, the information processing device 100C can generate two types of groups including the second group used for performing processing related to quantization for a plurality of objects, and generates groups for classifying objects. can do.

The generation unit 132C generates second group information indicating a plurality of second groups that are units of quantization.

In this way, the information processing device 100C can generate two types of groups including the second group that is the unit of quantization, and can generate groups for classifying objects.

The generation unit 132C generates second group information indicating a plurality of second groups used for quantizing vectors corresponding to each of a plurality of objects.

Thus, the information processing device 100C can generate two types of groups including the second group used for quantizing vectors corresponding to each of a plurality of objects, and generates groups for classifying objects. can do.

The generation unit 132C performs the classification of the plurality of second groups after the classification of the plurality of first groups.

In this manner, the information processing apparatus 100C can generate groups for classifying objects by performing processing for generating group information in the order of the classification of the first group and the classification of the second group.

132 C of production|generation parts produce|generate the 1st group information which shows several 1st groups by clustering several objects.

In this manner, the information processing apparatus 100C can generate groups for classifying objects by classifying a plurality of first groups by clustering a plurality of objects.

The acquisition unit 131 acquires index information indicating an index that searches for a plurality of objects. The generator 132C generates first group information indicating a plurality of first groups using the index information.

In this manner, the information processing device 100C can generate groups for classifying objects by classifying a plurality of first groups using indexes.

The generator 132C generates second group information indicating a plurality of second groups using the first group information indicating the first group.

In this way, the information processing apparatus 100C can classify the second groups based on the classification of the first groups by generating the second group information indicating the plurality of second groups using the first group information. , can generate groups that classify objects.

The generation unit 132C generates second group information indicating a plurality of second groups by clustering the plurality of first groups.

In this way, the information processing device 100C clusters the first groups and generates second group information indicating a plurality of second groups, thereby classifying the second groups based on the classification of the first groups. Groups can be created to classify objects.

The generation unit 132C performs the classification of the plurality of first groups after the classification of the plurality of second groups.

In this manner, the information processing apparatus 100C can generate groups for classifying objects by performing processing for generating group information in the order of the classification of the second group and the classification of the first group.

The generation unit 132C generates second group information indicating a plurality of second groups by clustering a plurality of objects.

In this manner, the information processing apparatus 100C can generate groups for classifying objects by classifying a plurality of second groups by clustering a plurality of objects.

The generating unit 132C generates first group information indicating a plurality of first groups using the second group information indicating the second group.

In this way, the information processing device 100C can classify the first groups based on the classification of the second groups by generating the first group information indicating the plurality of first groups using the second group information. , can generate groups that classify objects.

The generation unit 132C divides at least one of the plurality of second groups to generate two or more first groups, thereby generating first group information indicating the plurality of first groups.

In this way, the information processing device 100C divides the second group and generates first group information indicating a plurality of first groups, thereby classifying the first group based on the classification of the second group. Groups can be created to classify objects.

The generating unit 132C generates first group information indicating a plurality of first groups by clustering objects belonging to each second group and dividing each second group into two or more first groups.

In this way, the information processing apparatus 100C clusters objects belonging to each second group, divides each second group into two or more first groups, and generates first group information indicating a plurality of first groups. By generating, it is possible to classify the first group based on the classification of the second group, and it is possible to generate groups for classifying the objects.

[6. Hardware configuration]
The information processing apparatuses 100, 100A, 100B, and 100C according to the respective embodiments described above are implemented by a computer 1000 configured as shown in FIG. 39, for example. FIG. 39 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface (I/F) 1500, an input/output interface (I/F) 1600, and a media interface (I/F). ) 1700.

The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400 and controls each section. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. Communication interface 1500 receives data from other devices via network N, sends the data to CPU 1100, and transmits data generated by CPU 1100 to other devices via network N. FIG.

The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, through an input/output interface 1600 . CPU 1100 acquires data from an input device via input/output interface 1600 . CPU 1100 also outputs the generated data to an output device via input/output interface 1600 .

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1200 . CPU 1100 loads such a program from recording medium 1800 onto RAM 1200 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the information processing apparatus 100 according to the first embodiment, the CPU 1100 of the computer 1000 implements the functions of the control unit 130 by executing programs loaded on the RAM 1200 . The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired via the network N from another device.

As described above, some of the embodiments of the present application have been described in detail based on the drawings, but these are merely examples, and various modifications, It is possible to carry out the invention in other forms with modifications.

[7. others〕
Further, among the processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being performed manually can be performed manually. All or part of this can also be done automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the one shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

Further, each process described in each embodiment described above can be appropriately combined within a range in which the contents of the process are not inconsistent.

Also, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the acquisition unit can be read as acquisition means or an acquisition circuit.

1 information processing system 10 terminal device 50 information providing device 100, 100A, 100B, 100C information processing device 120, 120A, 120B, 120C storage unit 121 object information storage unit 122 graph information storage unit 123, 123A quantization information storage unit 124 code Book information storage units 125, 125A Blob information storage units 126, 126A Blob connection graph information storage units 130, 130A, 130B, 130C Control unit 131 Acquisition units 132, 132A, 132B, 132C Generation units 133, 133A, 133B, 133C Search processing Department (Processing Department)
134 Provider

Claims (21)

an acquisition unit for acquiring object information indicating a plurality of objects to be searched for data;
First group information indicating a plurality of first groups for classifying the plurality of objects, based on the object information acquired by the acquisition unit, and a group classified differently from the plurality of first groups, a generation unit for generating second group information indicating a plurality of second groups, which are groups in which objects belonging to the same first group in the plurality of first groups are classified into the same group among the plurality of objects;
An information processing device comprising: The generating unit
the first group information for classifying the plurality of objects into a first group of a first number; and the second group information for classifying the plurality of objects into a second group of a second number smaller than the first number. 2. The information processing apparatus according to claim 1, wherein a is generated. The generating unit
generating the second group information indicating the plurality of second groups in which object groups belonging to each of two or more first groups among the plurality of first groups are classified into one second group; 3. The information processing apparatus according to claim 1 or 2. The generating unit
4. The method according to any one of claims 1 to 3, wherein said first group information indicating said plurality of first groups used for batch processing in said search processing targeting said plurality of objects is generated. The information processing device according to . The generating unit
5. The information processing apparatus according to claim 4, wherein said first group information indicating said plurality of first groups used for collectively calculating distances for said plurality of objects is generated. The generating unit
4. The first group information indicating the plurality of first groups used for parallel processing of calculating the distance between the search query and the object used in the search processing is generated. 6. The information processing device according to 5. The generating unit
7. The method according to any one of claims 1 to 6, wherein said second group information indicating said plurality of second groups used for performing processing related to quantization targeting said plurality of objects is generated. The information processing device described. The generating unit
8. The information processing apparatus according to claim 7, wherein the second group information indicating the plurality of second groups that are the units of the quantization is generated. The generating unit
9. Information according to claim 7 or claim 8, wherein said second group information is generated to indicate said plurality of second groups used to quantize vectors corresponding to each of said plurality of objects. processing equipment. The generating unit
The information processing apparatus according to any one of claims 1 to 9, wherein the classification into the plurality of second groups is performed after the classification into the plurality of first groups. The generating unit
11. The information processing apparatus according to claim 10, wherein said first group information indicating said plurality of first groups is generated by clustering said plurality of objects. The acquisition unit
obtaining index information indicating an index for searching the plurality of objects;
The generating unit
11. The information processing apparatus according to claim 10, wherein said index information is used to generate said first group information indicating said plurality of first groups. The generating unit
The information according to any one of claims 10 to 12, wherein the first group information indicating the first group is used to generate the second group information indicating the plurality of second groups. processing equipment. The generating unit
The information processing apparatus according to any one of claims 10 to 13, wherein said second group information indicating said plurality of second groups is generated by clustering said plurality of first groups. The generating unit
The information processing apparatus according to any one of claims 1 to 9, wherein the classification into the plurality of first groups is performed after the classification into the plurality of second groups. The generating unit
16. The information processing apparatus according to claim 15, wherein said second group information indicating said plurality of second groups is generated by clustering said plurality of objects. The generating unit
17. The information processing apparatus according to claim 15, wherein the first group information indicating the plurality of first groups is generated using the second group information indicating the second group. The generating unit
The first group information indicating the plurality of first groups is generated by dividing at least one second group out of the plurality of second groups to generate two or more first groups. The information processing apparatus according to any one of claims 15 to 17. The generating unit
The first group information indicating the plurality of first groups is generated by clustering the objects belonging to each second group and dividing each second group into two or more first groups. The information processing apparatus according to any one of claims 15 to 18. A computer-executed information processing method comprising:
an obtaining step of obtaining object information indicating a plurality of objects to be searched for data;
First group information indicating a plurality of first groups for classifying the plurality of objects, based on the object information acquired in the acquisition step, and a group classified differently from the plurality of first groups, a generating step of generating second group information indicating a plurality of second groups, which are groups into which objects belonging to the same first group in the plurality of first groups are classified into the same group among the plurality of objects;
An information processing method comprising: an acquisition procedure for acquiring object information indicating a plurality of objects to be searched for data;
First group information indicating a plurality of first groups for classifying the plurality of objects, based on the object information obtained by the obtaining procedure, and a group classified differently from the plurality of first groups, a generation step of generating second group information indicating a plurality of second groups, which are groups in which objects belonging to the same first group in the plurality of first groups are classified into the same group among the plurality of objects;
An information processing program characterized by causing a computer to execute

Images