JP2024027034A - Information processing apparatus, information processing method, and information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To create groups into which objects are classified.

SOLUTION: An information processing apparatus according to the present application has an acquisition unit and a generation unit. The acquisition unit acquires object information indicating a plurality of objects to be subjected to data search. The generation unit generates, based on the object information acquired by the acquisition unit, first group information indicating a plurality of first groups that are a first number of groups into which the plurality of objects are classified, and second group information indicating a plurality of second groups that are groups different in classification from the plurality of first groups and are a second number of groups larger than the first number into which the plurality of objects are classified.

SELECTED DRAWING: Figure 44

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Conventionally, techniques for searching (searching) various types of information have been provided. For example, there is a technology that generates a graph in which nodes corresponding to a search target are connected by edges, and performs a search using the generated graph. Further, such a technique is used for, for example, image search.

Patent No. 6293335 Patent No. 6300982 Patent No. 6311000

Masajiro Iwasaki "Neighborhood search using approximate k-nearest neighbor graph using tree-structured index", Information Processing Society of Japan Transactions, 2011/2, Vol. 52, No. 2. pp.817-828.

However, with the above-mentioned conventional technology, there is room for improvement in terms of improving the efficiency of search processing. For example, in the above-mentioned conventional technology, by adjusting the number of edges of the graph and changing the graph to one that allows efficient searching, efficiency is achieved by reducing search processing time. There is a limit to how much efficiency can be achieved in search processing through changes. Therefore, in order to generate information that can be used for efficient search processing, such as speeding up search processing, the concept of groups for classifying objects to be searched is introduced, and groups for classifying objects are generated. is desired.

The present application has been made in view of the above, and aims to provide an information processing device, an information processing method, and an information processing program that generate groups for classifying objects.

The information processing device according to the present application includes an acquisition unit that acquires object information indicating a plurality of objects that are targets of data search, and a second object that classifies the plurality of objects based on the object information acquired by the acquisition unit. first group information indicating a plurality of first groups that are one number group; and groups that have a different classification from the plurality of first groups and are larger than the first number for classifying the plurality of objects. A generation unit that generates second group information indicating a plurality of second groups that are a second number of groups.

According to one aspect of the embodiment, it is possible to generate groups for classifying objects.

FIG. 1 is a diagram illustrating an example of information processing according to the first embodiment. FIG. 2 is a diagram showing an example of data according to the first embodiment. FIG. 3 is a diagram showing an example of data according to the first embodiment. FIG. 4 is a diagram illustrating a configuration example of an information processing system according to the first embodiment. FIG. 5 is a diagram illustrating a configuration example of an information processing apparatus according to the first embodiment. FIG. 6 is a diagram illustrating an example of an object information storage unit according to the first embodiment. FIG. 7 is a diagram illustrating an example of a graph information storage unit according to the first embodiment. FIG. 8 is a diagram illustrating an example of a quantization information storage unit according to the first embodiment. FIG. 9 is a diagram illustrating an example of a codebook information storage unit according to the first embodiment. FIG. 10 is a flowchart illustrating an example of information processing according to the first embodiment. FIG. 11 is a flowchart illustrating an example of search processing according to the first embodiment. FIG. 12 is a diagram illustrating an example of information processing according to the second embodiment. FIG. 13 is a diagram illustrating a configuration example of an information processing apparatus according to the second embodiment. FIG. 14 is a diagram illustrating an example of a quantization information storage unit according to the second embodiment. FIG. 15 is a diagram illustrating an example of a blob information storage unit according to the second embodiment. FIG. 16 is a flowchart illustrating an example of search processing according to the second embodiment. FIG. 17 is a diagram illustrating an example of information processing according to a modification. FIG. 18 is a diagram illustrating an example of a graph information storage unit according to a modification. FIG. 19 is a flowchart illustrating an example of a search process according to a modification. FIG. 20 is a diagram illustrating another example of information processing according to the modification. FIG. 21 is a diagram illustrating an example of blob information generation processing. FIG. 22 is a flowchart illustrating an example of information processing according to a modification. FIG. 23 is a diagram illustrating an example of information processing according to the third embodiment. FIG. 24 is a diagram illustrating a configuration example of an information processing apparatus according to the third embodiment. FIG. 25 is a diagram illustrating an example of a blob connection graph information storage unit according to the third embodiment. FIG. 26 is a flowchart illustrating an example of information processing according to the third embodiment. FIG. 27 is a flowchart illustrating an example of search processing according to the third embodiment. FIG. 28 is a flowchart illustrating an example of search processing according to the third embodiment. FIG. 29 is a diagram illustrating an example of blob connection graph information according to a modification. FIG. 30 is a diagram illustrating an example of a blob information storage unit according to a modification. FIG. 31 is a diagram illustrating an example of a blob connection graph information storage unit according to a modification. FIG. 32 is a diagram illustrating an example of information processing according to the fourth embodiment. FIG. 33 is a diagram illustrating a configuration example of an information processing apparatus according to the fourth embodiment. FIG. 34 is a conceptual diagram regarding quantization according to the fourth embodiment. FIG. 35 is a diagram illustrating an example of a transposed index according to the fourth embodiment. FIG. 36 is a diagram showing a specific example of a transposed index according to the fourth embodiment. FIG. 37 is a flowchart illustrating an example of information processing according to the fourth embodiment. FIG. 38 is a flowchart illustrating an example of information processing according to the fourth embodiment. FIG. 39 is a diagram illustrating an example of information processing according to the fifth embodiment. FIG. 40 is a diagram illustrating an example of information processing according to the fifth embodiment. FIG. 41 is a diagram illustrating a configuration example of an information processing apparatus according to the fifth embodiment. FIG. 42 is a flowchart illustrating an example of information processing according to the fifth embodiment. FIG. 43 is a flowchart illustrating an example of search processing according to the fifth embodiment. FIG. 44 is a diagram illustrating an example of information processing according to the sixth embodiment. FIG. 45 is a diagram illustrating a configuration example of an information processing apparatus according to the sixth embodiment. FIG. 46 is a diagram illustrating an example of a transposed index according to the sixth embodiment. FIG. 47 is a flowchart illustrating an example of information processing according to the sixth embodiment. FIG. 48 is a flowchart illustrating an example of search processing according to the sixth embodiment. FIG. 49 is a diagram illustrating an example of generation of the first group and the second group. FIG. 50 is a diagram illustrating an example of generation of the first group and the second group. FIG. 51 is a diagram illustrating an example of generation of the first group and the second group. FIG. 52 is a flowchart illustrating an example of information processing according to the sixth embodiment. FIG. 53 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus, an information processing method, and an embodiment of an information processing program (hereinafter referred to as "embodiments") according to the present application will be described in detail below with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited to this embodiment. Further, in each of the embodiments below, the same parts are given the same reference numerals, and redundant explanations will be omitted.

(Embodiment)
[1. First embodiment]
[1-1. Information processing〕
An example of information processing according to the first embodiment will be described using FIG. 1. FIG. 1 is a diagram illustrating an example of information processing according to the first embodiment. The information processing apparatus 100 executes a search process using a graph index (also simply referred to as a "graph") that is a graph structure of a plurality of objects that are data search targets. FIG. 1 shows part of a search process when the information processing device 100 performs a neighborhood search using a graph in which nodes corresponding to vectors of the object to be searched are connected by edges. . The information processing apparatus 100 uses each node of the graph as an object to be searched, and searches for nodes near a given search query (vector) by tracing the graph. Through the search process, the information processing apparatus 100 extracts a predetermined number of nodes (hereinafter also referred to as "search number") designated as the number of nearby nodes to be extracted as nodes in the vicinity of the search query. In the following, a case where image information is the object of data search will be described as an example, but the object of data search may be various objects such as video information and audio information.

The information processing device 100 performs search processing for nodes corresponding to a huge amount of image information in units of millions to hundreds of millions, but in the drawing, only a portion of them (several tens of nodes such as node N1 in FIG. 1) are shown. ) are shown. For example, the information processing apparatus 100 acquires information regarding a plurality of nodes (vectors) such as nodes N1, N7, N9, etc., as shown in spatial information SP1 in FIG. When "node N* (* is an arbitrary numerical value)" is written in this way, it indicates that the node is identified by the node ID "N*". For example, when "node N1" is written, the node is identified by the node ID "N1". Each white circle (◯) in the spatial information SP1 indicates each node.

In the spatial information SP1 in FIG. 1, codes are mainly attached to nodes related to the explanation, but each white circle (○) without a code is also a node, and many nodes are included in addition to the nodes shown. It will be done. Further, each node corresponds to each object (search target). For example, each of the plurality of local features extracted from the image may be an object. Further, for example, various data in which distances between objects are defined may be objects.

For example, the spatial information SP1 in FIG. 1 may be a Euclidean space. For example, it is assumed that the spatial information SP1 corresponds to the number of dimensions of the vector of the object, and is a multidimensional space such as 100 dimensions or 1000 dimensions. Note that FIG. 1 conceptually illustrates a state in which a vector (space) is divided into four subregions (subspaces) by direct product quantization as shown in spatial information SP1, but regarding the point of direct product quantization, will be described later.

A dotted line connecting white circles (◯) that are nodes in the spatial information SP1 indicates an edge connecting the nodes. In the example of FIG. 1, to simplify the explanation, a case is shown in which nodes are connected by undirected (bidirectional) edges (hereinafter also simply referred to as "edges"). Note that the undirected edge here means an edge that allows data to be traced in both directions between connected nodes. For example, the white circle (◯) indicating the node N1 and the white circle (◯) indicating the node N4 are connected by a dotted line, indicating that the nodes N1 and N4 are connected by an edge. That is, in the graph shown in the spatial information SP1, it is possible to trace the path between the node N1 and the node N4 in both directions. Specifically, the spatial information SP1 can be traced from the node N1 to the node N4, and from the node N4 to the node N1.

Note that in the example of FIG. 1, for convenience of illustration, only edges connecting the illustrated nodes are illustrated, but many edges other than the illustrated edges are included. In this way, the spatial information SP1 in FIG. 1 shows only a portion of the edges, but it is assumed that the spatial information SP1 in FIG. 1 is, for example, a k-nearest neighbor graph. Note that the spatial information SP1 may be various types of graphs.

Furthermore, the edges of the graph are not limited to undirected edges, but may be directed edges. In the case of a directed edge, it is possible to trace only from the node that is the reference source of the directed edge to the reference destination node. For example, if two nodes are connected by two directed edges: a first edge where one is the reference source and the other is the reference destination, and a second edge where one is the reference destination and the other is the reference source. , the two nodes are connected by an undirected (bidirectional) edge and have the same state.

The search process will now be described using FIG. 1. In the example of FIG. 1, it is assumed that the information processing apparatus 100 has already acquired a graph GR1 as shown in the spatial information SP1. Note that the information processing apparatus 100 may generate the graph GR1 using various conventional techniques as appropriate. First, prior to explaining the search process, vector (space) product quantization will be explained.

FIG. 1 conceptually illustrates a state in which a vector (space) is divided into four subspaces AR11 to AR14 by direct product quantization as shown in spatial information SP1. The subspaces AR11 to AR14 shown in FIG. 1 are shown as conceptual diagrams for explaining distances between vectors of nodes (objects), etc., but each of the subspaces AR11 to AR14 is a multidimensional space. For example, the subspace AR11 shown in FIG. 1 is illustrated in a two-dimensional manner because it is illustrated on a plane, but it is assumed that it is a multidimensional space, such as a 100-dimensional or 1000-dimensional space.

Each of the partial spaces AR11 to AR14 indicates a space corresponding to each division (partial vector) of the vector divided into four. Note that, for example, a 100-dimensional vector may be divided into 100 parts. For example, the subspace AR11 is a dimension space (" (also referred to as "first subspace"). Subspace AR11 indicates a subspace (first subspace) corresponding to the first division position of the vector. In the case of search query QE1, subspace AR11 is a space corresponding to first subquery QE1-1, which is the first subvector (first subvector).

Furthermore, the subspace AR12 is a dimensional space ("second subspace") corresponding to the second subvector from the beginning (also referred to as "second subvector") among the four subvectors obtained by dividing the vector into four. ). Subspace AR12 indicates a subspace (second subspace) corresponding to the second division position from the beginning of the vector. In the case of search query QE1, subspace AR12 is a space corresponding to second subquery QE1-2, which is the second subvector (second subvector) from the beginning.

Further, the subspace AR13 is a dimensional space ("third subspace") corresponding to the third subvector from the beginning (also referred to as "third subvector") among the four subvectors obtained by dividing the vector into four. ). Subspace AR13 indicates a subspace (third subspace) corresponding to the third division position from the beginning of the vector. In the case of search query QE1, subspace AR13 is a space corresponding to third partial query QE1-3, which is the third partial vector (third subvector) from the beginning.

Furthermore, the subspace AR14 is a dimension space (" 4th subspace). Subspace AR14 indicates a subspace (fourth subspace) corresponding to the fourth division position from the beginning of the vector. In the case of search query QE1, subspace AR14 is a space corresponding to fourth subquery QE1-4, which is the fourth subvector (fourth subvector) from the beginning.

In the example of FIG. 1, the subspaces AR11 to AR14 are shown with similar shapes, but the shapes of the subspaces AR11 to AR14 may be different, and the manner in which the regions in each of the subspaces AR11 to AR14 are divided may also be different. It's okay. Further, it is assumed that the connection relationship between nodes by edges is common to subspaces AR11 to AR14. That is, the graph GR1 is a graph corresponding to the spatial information SP1, and is common to the subspaces AR11 to AR14.

Furthermore, in the example of FIG. 1, a lookup table is generated for each divided partial vector (subvector). For example, clustering is performed for each of four subvectors: a first subvector, a second subvector, a third subvector, and a fourth subvector, and a lookup table (codebook information) is generated individually.

In FIG. 1, the first subvectors are clustered into nine groups corresponding to codebooks CD11 to CD19, as shown in subspace AR11, and the vectors corresponding to each of codebooks CD11 to CD19 are representative vectors ( centroid). For example, the first subvectors such as nodes N7 and N9 indicate that they are clustered into a group corresponding to codebook CD11. In this case, the first subvectors such as nodes N7 and N9 use the codebook information (also referred to as "first codebook information") corresponding to the first subvector in the distance calculation (calculation). is vector quantized into a vector.

Furthermore, the second sub-vectors are clustered into a plurality of groups corresponding to codebooks CD21 to CD24, etc. (see FIGS. 2 and 9), and the vectors corresponding to each of codebooks CD21 to CD24, etc. are representative vectors (centre Lloyd's). In this case, the second subvector of each node is calculated using the codebook information (also referred to as "second codebook information") corresponding to the second subvector in distance calculation. vector quantized.

The third sub-vectors are clustered into multiple groups corresponding to codebooks CD31 to CD34, etc. (see FIGS. 2 and 9), and the vectors corresponding to each of codebooks CD31 to CD34, etc. are representative vectors (centroids). It is calculated as In this case, the third subvector of each node is calculated using the codebook information (also referred to as "third codebook information") corresponding to the third subvector in distance calculation. vector quantized.

The fourth sub-vectors are clustered into multiple groups corresponding to codebooks CD41 to CD44, etc. (see FIGS. 2 and 9), and the vectors corresponding to each of codebooks CD24 to CD44, etc. are representative vectors (centroids). It is calculated as In this case, the fourth subvector of each node is calculated using the codebook information (also referred to as "fourth codebook information") corresponding to the fourth subvector in distance calculation. vector quantized.

Note that the codebook information, such as the process of determining the representative vector, is generated using various techniques as appropriate. Since generation of codebook information is a conventional technique, detailed explanation will be omitted. Further, the lookup table is not limited to the above, and for example, one lookup table (codebook information) may be used for all divided partial vectors (subvectors), but this point will be described later.

From here, a search process targeting the search query QE1 will be explained. First, the information processing device 100 obtains the search query QE1 (step S11). For example, the information processing device 100 acquires the search query QE1 from the terminal device 10 (see FIG. 4) used by the user.

Then, the information processing device 100 divides the vector that is the search query QE1 into four. That is, the information processing device 100 divides the search query QE1 into four partial queries. In FIG. 1, the information processing device 100 divides the search query QE1 into a first partial query QE1-1 which is a first subvector, a second partial query QE1-2 which is a second subvector, and a second partial query QE1-2 which is a third subvector. It is divided into four subvectors: 3 partial queries QE1-3 and a fourth partial query QE1-4, which is a fourth subvector. Specifically, the information processing device 100 sets "45, 23, 2..." as a first partial query QE1-1, "127, 34, 5..." as a second partial query QE1-2, and sets "20, 98, 110...'' is the third partial query QE1-3, and ``12, 45, 4...'' is the fourth partial query QE1-4.

Then, the information processing device 100 calculates the distance between each subvector of the search query QE1 and the vector of the codebook corresponding to that subvector. The information processing device 100 calculates the distance (difference) between each subvector of the search query QE1 and the vector of each codebook, as shown in codebook information TB1 to TB4 in FIG. 2. FIG. 2 is a diagram showing an example of data according to the first embodiment.

The codebook information TB1 indicates first codebook information corresponding to the first subvector, and the information processing apparatus 100 stores each vector of the codebooks CD11 to CD19 corresponding to the first subvector and the first partial query QE1. -1. In FIG. 2, the information processing device 100 calculates the distance between the codebook CD11 and the first partial query QE1-1 as a distance DS11, as shown in codebook information TB1. Similarly, the information processing device 100 calculates the distances between each of the codebooks CD12 to CD14, etc. and the first partial query QE1-1 as distances DS12 to DS14, etc.

Similarly, each of codebook information TB2 to TB4 indicates second to fourth codebook information corresponding to each of the second to fourth subvectors. The information processing device 100 calculates the distances between each of the codebooks CD21 to CD24, etc. and the second partial query QE1-2 as distances DS21 to DS24, etc., as shown in the codebook information TB2. The information processing device 100 calculates the distances between each of the codebooks CD31 to CD34, etc. and the third partial query QE1-3 as distances DS31 to DS34, etc., as shown in the codebook information TB3. The information processing device 100 calculates the distances between each of the codebooks CD41 to CD44, etc. and the fourth partial query QE1-4 as distances DS41 to DS44, etc., as shown in codebook information TB4.

Although the distances DS11 to DS44 and the like are shown abstractly in FIG. 2 for the sake of explanation, it is assumed that the distances DS11 to DS44, etc. are concrete values. The distance is a value expressed as a floating point number, but scale-offset-compression (https://www.unidata.ucar.edu/blogs/ developer/entry/compression_by_scaling_and_offfset" etc.) may be used to compress it into a 1-byte integer type. The information processing apparatus 100 may calculate the distance between each codebook and the query at the timing when the search query QE1 is acquired, or at the timing when the information becomes necessary. In the search process, the information processing device 100 uses the codebook information TB1 to TB4 as a lookup table to calculate the distance between each node and the search query QE1, that is, the approximate distance (also referred to as "first distance"). do. In this way, when searching a graph in the search process, the information processing device 100 uses approximate distances (also referred to as "second distances") rather than true distances (also referred to as "second distances") between each node and the search query. 1 distance) and perform the processing, it is possible to perform efficient search processing.

The information processing device 100 executes a search process targeting the search query QE1 (step S12). The information processing device 100 obtains a search result for the search query QE1 by performing a search process as shown in FIG. 11 using the graph GR1 for the search query QE1. Details of the search process shown in FIG. 11 will be described later. The information processing device 100 performs a search process on the search query QE1, thereby extracting the searched nodes as nodes in the vicinity of the search query QE1.

In the search process targeting the search query QE1, the information processing device 100 selects a predetermined node from among the nodes as a node (hereinafter also referred to as "starting point node") that becomes the starting point (starting point) of the search for the graph GR1. do. For example, the information processing apparatus 100 selects the starting node using a predetermined index such as a tree-structured index. In the example of FIG. 1, it is assumed that the information processing apparatus 100 selects node N7 as the starting point node. Note that although FIG. 1 shows a case where only node N7 is selected as the starting point node using a predetermined index to simplify the explanation, the information processing apparatus 100 can also select multiple nodes as the starting point node. Alternatively, the starting node may be selected by various methods such as random.

Here, in the search process targeting the search query QE1, the information processing device 100 parallelizes the distance between the search query and a node to which edges from one node are connected (hereinafter also referred to as "connection node"). Calculate (step S13). In FIG. 1, the information processing device 100 uses the search query QE1 for nodes N9, N12, N54, N85, etc., which are nodes (connection nodes) to which edges from the node N7 are connected, as shown in the batch processing information LT1. Calculate the distance between the two in parallel.

For example, in node N9, as shown in node information INF1, the first subvector is codebook CD12, the second subvector is codebook CD23, the third subvector is codebook CD35, and the fourth subvector is codebook CD47. Indicates that it is associated with. Note that the size of the variable CD is determined by the size of the codebook, so if the codebook size is 16, for example, the variable CD only needs to be 4 bits, making it possible to significantly compress data. Therefore, the information processing device 100 uses the distance DS12 of the codebook CD12, the distance DS23 of the codebook CD23, the distance DS35 of the codebook CD35, and the distance DS47 of the codebook CD47 to determine the distance between the node N9 and the search query QE1. Calculate. For example, the information processing device 100 calculates the sum of the distance DS12 of the codebook CD12, the distance DS23 of the codebook CD23, the distance DS35 of the codebook CD35, and the distance DS47 of the codebook CD47 as the distance between the node N9 and the search query QE1. calculate.

For example, in the node N12, as shown in the node information INF2, the first subvector is codebook CD14, the second subvector is codebook CD29, the third subvector is codebook CD31, and the fourth subvector is codebook CD45. Indicates that it is associated with. For example, the information processing device 100 calculates the sum of the distance DS14 of the codebook CD14, the distance DS29 of the codebook CD29, the distance DS31 of the codebook CD31, and the distance DS45 of the codebook CD45 as the distance between the node N12 and the search query QE1. calculate. Similarly, the information processing device 100 uses the node information INF3 of the node N54 to calculate the distance between the node N54 and the search query QE1, and uses the node information INF4 of the node N85 to calculate the distance between the node N85 and the search query QE1. Calculate distance.

For example, the information processing device 100 calculates the distance between each of the nodes N9, N12, N54, and N85 and the search query QE1 at once by parallelizing SIMD calculations. Thereby, the information processing device 100 can speed up the distance calculation by parallelizing the distance calculation, and can perform efficient search processing.

Note that, in order to simplify the explanation, FIG. 1 shows a case where the distance is computed in parallel for four nodes, but the number to be computed in parallel is determined based on the specifications of the information processing device 100. . For example, if the number that the information processing device 100 can process at once using SIMD (also referred to as “unit that can be processed at once”) is “4”, the processing will be similar to that shown in FIG. 1, but it will be possible to process at once using SIMD. When the number (unit that can be collectively processed) is "16", the information processing apparatus 100 calculates distances in parallel for 16 nodes. Further, when the batch processable unit is "32", the information processing apparatus 100 calculates distances in parallel for 32 nodes. In this way, the information processing device 100 enables efficient search processing by collectively calculating the distances of the number of nodes corresponding to the number that can be processed at once by SIMD (batch processable unit). be able to.

The information processing device 100 calculates the number of searches by performing a search process as shown in FIG. The node is extracted as a node in the vicinity of the search query QE1.

As described above, the information processing device 100 calculates the approximate distance (first distance) between the vector quantized vector of each node and the search query, and performs the search process using the first distance. This enables efficient search processing. Furthermore, the information processing apparatus 100 can perform efficient search processing by performing distance calculations for a number of nodes that can be parallelized at once. As described above, the information processing device 100 repeatedly uses a limited memory area (lookup table) (memory cache) by collectively calculating approximate distances between nodes of the graph and multiple connected nodes. speed).

In addition, in the example of FIG. 1, the information processing device 100 performs search processing using vectors divided by direct product quantization, thereby achieving more efficient search processing than when performing search processing without dividing vectors. Search processing can be enabled. For example, the information processing device 100 calculates distances between short vectors by direct product quantization, and can reduce the size of vectors that are targets of distance calculation. Further, the information processing apparatus 100 can limit the memory space accessed by the lookup table used during distance calculation to the lookup table of divided subvectors. The information processing apparatus 100 can reduce the vector size while suppressing a decrease in search accuracy by using Cartesian product quantization. In addition, in FIG. 1, the processing in the case where Cartesian product quantization is performed has been described, but the case where Cartesian product quantization is performed is only an example, and the information processing device 100 can process vectors that have not been subjected to Cartesian product quantization. The search process may be performed using .

[1-1-1. others〕
The above-described processing is merely an example, and the information processing apparatus 100 may perform search processing using various information and methods for efficient search processing. Regarding this point, each item will be explained in detail.

(Storage mode of edges (connection nodes))
Regarding the information of edges (connection nodes) connected to each node, there are two types of storage modes (storage methods) for edges (connection nodes) in nodes: the following first storage mode and second storage mode. Therefore, the storage mode may be selected depending on the usage mode.

For example, as a first storage mode, each node may be stored in association with the ID of the connected node (object), and information on objects subjected to product quantization may be stored in another table. For example, the data storage format shown in FIGS. 7 and 8 corresponds to the first storage format. In the case of the first storage mode, memory usage can be reduced.

Furthermore, for example, as a second storage mode, each node has a quantized object directly. In the case of the second storage mode, information obtained by quantizing connection nodes (objects) of a certain node is stored in association with each node. For example, a data storage mode in which quantized information (node information INF1 to INF4 in FIG. 2) of nodes N9, N12, N54, and N85, which are connected nodes of node N7, is stored in association with node N7. corresponds to the second storage mode. In the case of the second storage mode, since objects are accessed sequentially during a search, a decrease in speed can be suppressed.

(lookup table)
In the above example, each divided subvector is individually clustered and a lookup table (codebook information) is generated individually, but the lookup table is not limited to this case. There may be.

For example, all subvectors may be clustered and one lookup table may be generated. In the above example, for example, one lookup table may be generated by clustering all four subvectors: the first subvector, the second subvector, the third subvector, and the fourth subvector. .

Further, lookup tables may be generated for each of a plurality of classes by partially merging individual subvectors. For example, subvectors with similar variances may be merged to form multiple classes, and a lookup table may be generated for each of the multiple classes. In the above example, for example, if the variances of two subvectors, the first subvector and the third subvector, are similar, and the variances of the two subvectors, the second subvector and the fourth subvector, are similar, then the first The sub-vector and the third sub-vector may be merged to form the first class, and the second sub-vector and the fourth sub-vector may be merged to form the second class. In this case, two lookup tables, a first class lookup table (codebook information) and a second class lookup table (codebook information), may be generated.

(transposed)
The data retention described above is merely an example, and the information processing apparatus 100 may retain data in various ways so as to enable efficient data reference. For example, the information processing device 100 may hold transposed data. This point will be explained using FIG. 3. FIG. 3 is a diagram showing an example of data according to the first embodiment.

For example, if data is held as shown in node information INF1 to INF4 and node information INF1 to INF4 are sequentially processed, codebook information TB1 to TB4, which is a lookup table, is repeatedly processed as shown in step S14 in FIG. Please refer to it.

Therefore, as shown in step S15, transposed data TR1 to TR4 are generated by transposing the node information INF1 to INF4. For example, the information processing device 100 generates transposed data TR1 to TR4 by transposing node information INF1 to INF4.

In FIG. 3, first data TR1 is generated which is a list of codebooks corresponding to the first subvectors of nodes N9, N12, N54, and N85 among the node information INF1 to INF4. Specifically, codebook CD12 corresponding to the first subvector of node N9, codebook CD14 corresponding to the first subvector of node N12, codebook CD13 corresponding to the first subvector of node N54, and node N85. First data TR1, which is a list of codebook CD18 corresponding to the first subvector of , is generated.

Similarly, second data TR2 is generated, which is a list of codebooks corresponding to the second subvectors of nodes N9, N12, N54, and N85 among the node information INF1 to INF4. Also, third data TR3 is generated, which is a list of codebooks corresponding to the third subvectors of nodes N9, N12, N54, and N85 among the node information INF1 to INF4. Fourth data TR4 is generated, which is a list of codebooks corresponding to the fourth subvectors of nodes N9, N12, N54, and N85 among the node information INF1 to INF4.

Note that correspondence information is generated that indicates the correspondence between which data in the list of each transposed data TR1 to TR4 corresponds to which node. In the example of FIG. 3, among the list of transposed data TR1 to TR4, the first data corresponds to node N9, the second data corresponds to node N12, and the third data corresponds to node N54. Correspondingly, association information indicating that the fourth (last) data corresponds to node N85 is generated. By referring to the correspondence information, the information processing device 100 can specify which node each data in the list of transposed data TR1 to TR4 corresponds to. For example, the information processing device 100 generates association information.

The information processing device 100 performs processing using transposed data TR1 to TR4 obtained by transposing node information INF1 to INF4. For example, when the information processing device 100 refers to the lookup table using the transposed data TR1, it refers only to the codebook information TB1, and refers to only one codebook information TB1. Similarly, when the information processing device 100 refers to the lookup table using the transposed data TR2, it refers only to the codebook information TB2, and refers to only one codebook information TB2. Similarly, only one codebook information is referred to for the transposed data TR3 and TR4. As a result, the information processing apparatus 100 can efficiently refer to data, thereby enabling efficient search processing.

In this way, in order to refer to the lookup table for each subclass (subvector, etc.) during batch distance calculation, the data of the neighboring objects (connected nodes) of the node is transposed, and the data is sent to the node in the order in which it is compiled for each subclass. By having both, the locality of reference is further increased and speeding up is possible.

Note that, in order to simplify the explanation, FIG. 3 shows a case where transposed data is generated for four nodes, but the unit (number of nodes) in which transposed data is generated is based on the specifications of the information processing device 100. Determined by For example, if the batch processable unit of the information processing device 100 is "4", the process is similar to that shown in FIG. 3, but if the batch processable unit is "16", 16 nodes are treated as one unit. Transposed data is generated. Further, when the batch processable unit is "32", transposed data is generated with 32 nodes as one unit. In this way, by using the transposed data that is generated according to the number that the information processing apparatus 100 can process at once using SIMD (units that can be processed at once), the information processing apparatus 100 can refer to data efficiently. This makes it possible to perform efficient search processing.

For example, much of the time during the above-mentioned search process is spent on distance calculation, but distance calculation can be sped up by parallel calculation using SIMD as described above. When distance calculation is parallelized in this way, the time required to fetch object data occupies the search process. If this fetch time can be reduced, further speeding up can be achieved. Note that there is a method of prefetching to reduce the time required to fetch data, but there is a limit.

On the other hand, the information processing device 100 can suppress data fetching and achieve further speedup by increasing the locality of reference and making it possible to efficiently refer to data as described above. .

(search results)
Note that when returning search results based on the second distance (true distance) described above, the following two methods, the first method and the second method, can be considered.

For example, the first method is to search for more than the specified number of searches, and at the end of the search, calculate the true distance, sort by distance, and select the specified number of objects as search results. good. In this case, the information processing device 100 extracts nodes whose number (also referred to as "extended search number") is greater than the specified number of searches (also referred to as "first number"). By setting the number of searches (also referred to as ``number 2'') as the search number and performing the search process as shown in Figure 11, nodes with the extended search number, that is, nodes whose number is greater than the specified number of searches, are extracted as nearby candidate nodes. do. Then, the information processing device 100 calculates a second distance (true distance) for the neighboring candidate nodes, and selects the first number of nodes from among the neighboring candidate nodes, starting from the one with the shortest second distance, in the vicinity of the search query. Extract as a node.

For example, as a second method, only when a node (object) is within the search range or within the search range based on the first distance (approximate distance), the second distance (true The search result based on the true distance may be returned by calculating the approximate distance (distance) and replacing the approximate distance with the true distance.

The search range here is the range defined by "r" in FIG. 11, and the search range is defined by "r(1+ε)" using the search range coefficient "ε" in FIG. range. For example, when calculating the second distance (true distance) when the node enters the search range, the information processing device 100 calculates whether the node A second distance (true distance) is calculated. Further, when calculating the second distance (true distance) when the node enters the search range, the information processing device 100 calculates the second distance (true distance) when the first distance (approximate distance) of the node is equal to or less than “r(1+ε)”. , calculate the second distance (true distance) of that node.

For example, to improve accuracy, the condition may be set to be within the search range. Furthermore, when speeding up the processing is desired, the condition may be set to be within the search range. Note that the above is just an example, and which condition is to be used may be set as appropriate depending on the value of the search range coefficient "ε", the purpose of processing, etc.

(vector quantization)
Note that the information processing device 100 may perform two-stage vector quantization as shown in Patent Document 3. Then, the information processing device 100 may calculate the distance between the search query and each node using the following equation (1).

Here, the value on the left side of the above equation (1) indicates, for example, the square distance between the search query and the node. Further, for example, "x" in the above formula (1) corresponds to a query. Further, for example, "y" in the above formula (1) corresponds to a node. Further, for example, "q _c (y)" on the right side of the above equation (1) indicates a representative vector (centroid) of "y". For example, when the information processing apparatus 100 does not have node vector data for "y" in the above equation (1), the information processing apparatus 100 may use the numerical value of the centroid of the partial area to which each node belongs. Further, for example, "yq _c (y)" indicates a residual vector. Further, for example, “q _p ” on the right side of the above equation (1) indicates a predetermined quantizer (function).

Further, for example, "j" on the right side of the above equation (1) may be the number of divided spaces. For example, in the example of FIG. 1, "j" on the right side of the above equation (1) may be the number of divided spaces "4". Further, for example, "u _j ()" on the right side of the above equation (1) indicates a partial residual vector between the vectors in parentheses. For example, the information processing device 100 may calculate the distance between the query and the node by calculating the squared distance between the query and the node in each subspace using the above equation (1), and summing the squared distance between the query and the node. good.

[1-2. Information processing system configuration]
As shown in FIG. 4, the information processing system 1 includes a terminal device 10, an information providing device 50, and an information processing device 100. The terminal device 10, the information providing device 50, and the information processing device 100 are connected via a predetermined network N so that they can communicate by wire or wirelessly. FIG. 4 is a diagram illustrating a configuration example of an information processing system according to the first embodiment. Note that the information processing system 1 shown in FIG. 4 may include multiple terminal devices 10, multiple information providing devices 50, and multiple information processing devices 100.

The terminal device 10 is an information processing device used by a user. The terminal device 10 accepts various operations by the user. Note that below, the terminal device 10 may be referred to as a user. That is, in the following, the user can also be read as the terminal device 10. Note that the above-described terminal device 10 is realized by, for example, a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like.

The information providing device 50 is an information processing device that stores information for providing various information to users and the like. For example, the information providing device 50 stores object IDs based on character information etc. collected from various external devices such as web servers. For example, the information providing device 50 is an information processing device that provides image search services to users and the like. For example, the information providing device 50 stores various pieces of information for providing an image search service. For example, the information providing device 50 provides the information processing device 100 with vector information corresponding to an image that is a target of an image search service. Furthermore, by transmitting a query to the information processing device 100, the information providing device 50 receives an object ID and the like indicating an image corresponding to the query from the information processing device 100.

The information processing device 100 is a computer that provides a search service. The information processing apparatus 100 provides a specified number of search objects corresponding to the search query as search results. The information processing apparatus 100 uses a graph in which search target nodes (objects) are connected by edges to provide nodes (objects) near a search query as search results. The information processing device 100 uses a graph in which a group of nodes corresponding to each of a plurality of objects are connected by edges to search for nodes in the vicinity of a search query. The distance between the search query and the search query is calculated using vector information of a plurality of nodes subjected to vector quantization.

For example, upon receiving a query (search query) from the terminal device 10, the information processing device 100 searches for a target (object) similar to the search query, and provides the search result to the terminal device. Further, for example, the data that the information processing device 100 provides to the terminal device may be data itself such as image information, or information for referring to corresponding data such as a URL (Uniform Resource Locator). Good too. Further, the search query or search target (object) may be any type of data such as image, audio, text data, etc.

[1-3. Configuration of information processing device]
Next, the configuration of the information processing apparatus 100 according to the first embodiment will be described using FIG. 5. FIG. 5 is a diagram illustrating a configuration example of the information processing device 100 according to the first embodiment. As shown in FIG. 5, the information processing device 100 includes a communication section 110, a storage section 120, and a control section 130. Note that the information processing device 100 includes an input unit (for example, a keyboard, a mouse, etc.) that accepts various operations from an administrator of the information processing device 100, and a display unit (for example, a liquid crystal display, etc.) for displaying various information. May have.

(Communication Department 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card). The communication unit 110 is connected to a network (for example, network N in FIG. 4) by wire or wirelessly, and transmits and receives information to and from the terminal device 10 and the information providing device 50.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 5, the storage unit 120 according to the first embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123, and a codebook information storage unit 124. .

(Object information storage unit 121)
The object information storage unit 121 according to the first embodiment stores various information regarding objects. For example, the object information storage unit 121 stores object IDs and vector data. FIG. 6 is a diagram illustrating an example of an object information storage unit according to the first embodiment. The object information storage unit 121 shown in FIG. 6 includes items such as "object ID" and "vector information."

"Object ID" indicates identification information for identifying an object. Moreover, "vector information" indicates vector information corresponding to the object identified by the object ID. That is, in the example of FIG. 6, vector data (vector information) corresponding to the object is registered in association with an object ID that identifies the object.

For example, the example in FIG. 6 indicates that the object (target) identified by the object ID "OB1" is associated with multidimensional vector information of "10, 24, 51, 2...".

Note that the object information storage unit 121 is not limited to the above, and may store various types of information depending on the purpose.

(Graph information storage unit 122)
The graph information storage unit 122 according to the first embodiment stores various information regarding graphs. For example, the graph information storage unit 122 stores the generated graph. FIG. 7 is a diagram illustrating an example of a graph information storage unit according to the first embodiment. The graph information storage unit 122 shown in FIG. 7 has items such as "node ID", "object ID", and "connected node information".

"Node ID" indicates identification information for identifying each node (object) in the graph. Moreover, "object ID" indicates identification information for identifying an object. Note that if the node ID and object ID are the same, the object ID is stored in the "node ID" and the graph information storage unit 122 does not need to include the item "object ID." For example, when used as an object ID and a node ID, the object ID may be stored in the "node ID" and the graph information storage unit 122 may not include the "object ID" item.

Further, "connected node information" indicates information regarding a node (reference destination node) that can be traced from the corresponding node. For example, "connected node information" includes information such as "reference destination." “Reference destination” indicates information for identifying a reference destination (node) that is connected by an edge and can be traced from that node. That is, in the example of FIG. 7, a reference destination (node) that can be traced from the node by an edge is registered in association with a node ID (object ID) that identifies the node. Note that the "connected node information" may include information (edge ID) for identifying an edge connected to a reference destination.

The example of FIG. 7 indicates that the node (node N1) identified by the node ID "N1" corresponds to the object (target) identified by the object ID "OB1". Further, an edge is connected from the node N1 to the node (node N4) identified by the node ID "N4", indicating that it is possible to trace from the node N1 to the node N4.

Further, the node (node N2) identified by the node ID "N2" corresponds to the object (target) identified by the object ID "OB2". Further, an edge is connected from the node N2 to the node (node N6) identified by the node ID "N6", indicating that it is possible to trace from the node N2 to the node N6.

Further, the node (node N7) identified by the node ID "N7" corresponds to the object (target) identified by the object ID "OB7". Furthermore, edges are connected from node N7 to nodes N9, N12, N54, and N85, indicating that each of nodes N9, N12, N54, and N85 can be traced from node N2.

Note that the graph information storage unit 122 is not limited to the above, and may store various information depending on the purpose. For example, the graph information storage unit 122 may store the length of an edge connecting each node (vector). That is, the graph information storage unit 122 may store information indicating the distance between each node (vector). Note that the graph information storage unit 122 is not limited to the above, and may store graph information using various data structures.

The graph may also include a program module that receives a query as input, searches for nodes by tracing edges in the graph, and extracts and outputs nodes similar to the query. That is, the graph may be assumed to be used as a program module that performs search processing using the graph. For example, the graph GR1 may be a program that, when vector data is input as a query, extracts nodes corresponding to vector data similar to the vector data from the graph and outputs them. For example, the graph GR1 may be data used as a program module that searches for similar images corresponding to a query image. For example, the graph GR1 causes a computer to function based on an input query to extract and output nodes similar to the query in the graph.

(Quantization information storage unit 123)
The quantization information storage unit 123 according to the first embodiment stores various information regarding allocation processing. FIG. 8 is a diagram illustrating an example of a quantization information storage unit according to the first embodiment. In the example of FIG. 8, the quantization information storage unit 123 has items such as "node ID", "object ID", and "quantization information".

"Node ID" indicates identification information for identifying each node (object) in the graph. Moreover, "object ID" indicates identification information for identifying an object. Note that when the node ID and the object ID are common, the object ID is stored in the "node ID", and the quantization information storage unit 123 does not need to include the item "object ID".

Moreover, "quantization information" indicates information on quantized vectors of each node (object). For example, "quantization information" includes information such as "element" and "codebook ID." "Element" indicates the arrangement of the corresponding object in the vector. In the example of FIG. 8, the "element" includes "#1", "#2", "#3", and "#4". In this case, the vector of each node (object) is divided into four parts, and each divided partial vector is quantized by the codebook. Note that the number of divisions is not limited to 4. For example, if the number of divisions is 6, "element" may include "#1", "#2", "#3", "#4", "#5", " #6” is included. "Codebook ID" indicates information for identifying the codebook corresponding to each element (partial vector).

In the example of FIG. 8, the vector of node N9 (object OB9) is quantized by the codebook (codebook CD12) whose first partial vector among the four divided partial vectors is identified by the codebook ID "CD12". Indicates that the In addition, in the vector of node N9 (object OB9), among the four divided partial vectors, the second partial vector from the beginning is quantized by the codebook (codebook CD23) identified by the codebook ID "CD23". to show that

In addition, in the vector of node N9 (object OB9), among the four divided partial vectors, the third partial vector from the beginning is quantized by the codebook (codebook CD35) identified by the codebook ID "CD35". to show that In addition, the vector of node N9 (object OB9) has a codebook (codebook CD47) in which the fourth (i.e., the last) partial vector from the beginning among the four divided partial vectors is identified by the codebook ID "CD47". ) indicates that it is quantized.

Note that the quantization information storage unit 123 is not limited to the above, and may store various information depending on the purpose.

(Codebook information storage unit 124)
The codebook information storage unit 124 according to the first embodiment stores various information regarding the codebook. For example, the codebook information storage unit 124 stores codebook IDs and vector information of each codebook. FIG. 9 is a diagram illustrating an example of a codebook information storage unit according to the first embodiment. In the example of FIG. 9, the codebook information storage unit 124 stores a lookup table that indicates the association between each codebook and a vector.

The codebook information storage unit 124 stores codebook information TB1 used for quantizing the first partial vector among the four divided partial vectors, and the second partial vector from the beginning among the four divided partial vectors. Codebook information TB2 used for quantization is stored. The codebook information storage unit 124 also stores codebook information TB3 used for quantizing the third partial vector from the beginning among the partial vectors divided into four; Codebook information TB4 and the like used for quantizing the th (ie, last) partial vector is stored.

In the example of FIG. 9, the codebook information TB1 includes the codebook identified by the codebook ID "CD11" (codebook CD11), the codebook identified by the codebook ID "CD12" (codebook CD12), etc. Store codebook information. For example, codebook CD11 indicates that multidimensional vector information of "5, 13..." is associated. Further, codebook CD12 indicates that multidimensional vector information of "27, 51..." is associated.

Note that the codebook information storage unit 124 is not limited to the above, and may store various information depending on the purpose. For example, the codebook information storage unit 124 may store information indicating the difference (distance) between each codebook and the search query.

(Control unit 130)
Returning to the explanation of FIG. 5, the control unit 130 is a controller, and for example, controls the internal processing of the information processing apparatus 100 by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), etc. This is realized by executing various programs (corresponding to an example of an information processing program) stored in the storage device of , using the RAM as a work area. Further, the control unit 130 is a controller, and is realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 5, the control unit 130 includes an acquisition unit 131, a generation unit 132, a search processing unit 133, and a provision unit 134, and realizes or executes information processing functions and operations described below. do. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 5, and may be any other configuration as long as it performs information processing to be described later.

(Acquisition unit 131)
The acquisition unit 131 acquires various information. For example, the acquisition unit 131 acquires various information from the storage unit 120. For example, the acquisition unit 131 acquires various information from the object information storage unit 121, the graph information storage unit 122, the quantization information storage unit 123, the codebook information storage unit 124, and the like. Further, the acquisition unit 131 acquires various information from an external information processing device. The acquisition unit 131 acquires various information from the terminal device 10 and the information providing device 50.

The acquisition unit 131 acquires a graph. For example, the information processing device 100 may acquire spatial information SP1 in FIG. For example, the information processing device 100 may acquire a graph from an external device such as the information providing device 50.

The acquisition unit 131 acquires search queries for a plurality of objects that are data search targets. For example, the acquisition unit 131 acquires information regarding the search query QE1. For example, the acquisition unit 131 acquires a search query related to image search. For example, the acquisition unit 131 acquires a query from the terminal device 10 to be used. For example, the acquisition unit 131 acquires a query from the information providing device 50 that has received the query from the terminal device 10 to be used.

(Generation unit 132)
The generation unit 132 generates various information. For example, the generation unit 132 generates various information (data) from the information (data) stored in the storage unit 120. For example, the generation unit 132 generates various information from information (data) stored in the object information storage unit 121, the graph information storage unit 122, the quantization information storage unit 123, the codebook information storage unit 124, etc. .

The generation unit 132 may generate a graph as shown in the graph information storage unit 122. For example, the generation unit 132 generates spatial information SP1. Further, the generation unit 132 may generate information related to vector quantization as shown in the quantization information storage unit 123. For example, the generation unit 132 generates information in which each object such as the node N1 (object OB1) is vector quantized. Further, the generation unit 132 may generate information regarding the codebook as shown in the codebook information storage unit 124. The generation unit 132 may generate a codebook lookup table. For example, the generation unit 132 generates information regarding codebooks, such as codebook information TB1 to TB4. Note that when the information processing device 100 acquires the information shown in the graph information storage unit 122, quantization information storage unit 123, and codebook information storage unit 124 from an external device such as the information providing device 50, the information processing device 100 The generation unit 132 may not be included.

(Search processing unit 133)
The search processing unit 133 provides search services regarding objects. The search processing unit 133 searches for various information. The search processing unit 133 searches various information. For example, the search processing unit 133 searches for objects by searching a graph. For example, when the query acquired by the acquisition unit 131 is acquired, the search processing unit 133 searches for an object similar to the query by searching a graph. For example, the search processing unit 133 extracts objects similar to the query by searching the graph. For example, the search processing unit 133 extracts objects similar to the query by searching the graph based on the processing procedure shown in FIG. Note that the information processing device 100 does not need to include the search processing unit 133 when not providing a search service.

The search processing unit 133 selects various types of information in the search process. The search processing unit 133 extracts various information in the search process. The search processing unit 133 determines various information in the search process. The search processing unit 133 determines various information in the search process. The search processing unit 133 changes various information in the search process. The search processing unit 133 updates various information during search processing.

The search processing unit 133 uses a graph in which a group of nodes corresponding to each of a plurality of objects are connected by edges to search for nodes in the vicinity of a search query. The distance between the search query and the search query is calculated using vector information of a plurality of nodes subjected to vector quantization. The search processing unit 133 parallelizes and collectively calculates distances between a plurality of nodes and a search query. The search processing unit 133 performs parallel processing to calculate distances between a search query and a plurality of nodes whose number is determined based on the specifications of the information processing device 100.

The search processing unit 133 calculates distances between the plurality of nodes and the search query using a codebook in which each of the plurality of nodes is associated with a corresponding representative vector. The search processing unit 133 calculates the distance between the search query and a plurality of nodes in which edges from one node are connected. The search processing unit 133 searches the plurality of nodes and the search query using node information stored in association with one node and reference information indicating a representative vector with which each of the plurality of nodes is associated. Calculate the distance to.

The search processing unit 133 calculates distances between a plurality of nodes and a search query using vector information of a plurality of nodes, each of which is divided into a plurality of partial vectors by Cartesian product quantization. The search processing unit 133 uses a plurality of codebooks that are associated with representative vectors corresponding to vectors for each division position of a plurality of partial vectors into which each of the plurality of nodes is divided, to combine the plurality of nodes with the search query. Calculate the distance. The search processing unit 133 uses the node information stored in such a manner that the plurality of nodes and reference information indicating representative vectors associated with each of the plurality of partial vectors of each of the plurality of nodes are associated with one node. , calculate the distance between multiple nodes and the search query.

The search processing unit 133 uses reference information in which each of the plurality of nodes is associated with a list of representative vectors to which each of the plurality of partial vectors of each node is associated, and the search processing unit 133 associates the plurality of nodes with the search query. Calculate the distance. The search processing unit 133 includes transposed information in which representative vectors to which each of a plurality of partial vectors of each of a plurality of nodes corresponds are arranged in a list for each division position, and a correspondence indicating a position to which each of a plurality of nodes corresponds in a list. The distance between the plurality of nodes and the search query is calculated using the reference information including the attached information. The search processing unit 133 calculates distances between the plurality of nodes and the search query using one codebook in which each of the plurality of partial vectors of each of the plurality of nodes is associated with a corresponding representative vector.

The search processing unit 133 targets a predetermined node among the nodes to be processed in the search process. Calculate distance. The search processing unit 133 extracts a second number of nodes, which is a larger number than the first number of nodes to be extracted as neighboring nodes of the search query, as neighboring candidate nodes in the search process, and extracts a second number of nodes as neighboring candidate nodes. Calculate distance. The search processing unit 133 extracts a first number of nodes from among the nearby candidate nodes having the shorter second distance as nodes in the vicinity of the search query.

The search processing unit 133 calculates a second distance for nodes whose first distance is within a predetermined threshold. The search processing unit 133 calculates a second distance for nodes within the search range indicating the target range to be extracted as nearby nodes. The search processing unit 133 calculates a second distance for nodes within a search range indicating a target range of the search process.

(Providing unit 134)
The providing unit 134 provides various information. For example, the providing unit 134 transmits various information to the terminal device 10 and the information providing device 50. For example, the providing unit 134 provides an object ID corresponding to a search query as a search result. The providing unit 134 transmits the search results to the terminal device 10. The providing unit 134 provides the object ID searched by the search processing unit 133 to the terminal device 10 as a search result corresponding to the search query.

Further, the providing unit 134 may provide the object ID searched by the search processing unit 133 to the information providing device 50. For example, the providing unit 134 provides the information providing device 50 with the object ID extracted by the search processing unit 133 through the search. The providing unit 134 provides the object ID extracted by the search processing unit 133 to the information providing device 50 as information indicating a vector corresponding to the query.

[1-4. Information processing flow]
Next, the procedure of information processing by the information processing system 1 according to the first embodiment will be described using FIG. 10. FIG. 10 is a flowchart illustrating an example of information processing according to the first embodiment.

As shown in FIG. 10, the information processing apparatus 100 obtains search queries for a plurality of objects that are data search targets (step S101). In the example of FIG. 1, the information processing device 100 obtains the search query QE1.

The information processing apparatus 100 uses a graph in which a group of nodes corresponding to each of a plurality of objects are connected by edges to search for nodes near a search query. The distance between the node and the search query is calculated using vector information of a plurality of nodes subjected to vector quantization (step S102). In the example of FIG. 1, the information processing device 100 calculates the distances between the plurality of nodes N9, N12, N54, N85 and the search query QE1 by vector quantizing the distances between the plurality of nodes N9, N12, N54, N85. Calculate using information.

[1-5. Search processing example]
Here, an example of the search process according to the first embodiment will be described using FIG. 11 as an example. FIG. 11 is a flowchart illustrating an example of search processing according to the first embodiment. The search processing described below is performed by the search processing unit 133 of the information processing device 100. Further, the object referred to below may be read as a node. Note that in the following, the information processing device 100 (search processing unit 133) performs search processing. Note that if the search service is not provided, the information processing device 100 does not need to include the search processing unit 133. The search query for the process described below may be an additional node, a target node, an object specified by the user, or the like.

Here, the neighboring object set N(G, y) is a set of neighboring objects that are related by edges attached to node y. "G" may be predetermined graph data (for example, graph GR1 shown in spatial information SP1). For example, the information processing device 100 executes k-nearest neighbor search processing.

For example, the information processing device 100 sets the radius r of the hypersphere to ∞ (infinity) (step S300), and extracts a subset S from an existing object set (step S301). For example, the information processing apparatus 100 may extract the object (node) selected as the root node as the subset S. Further, for example, a hypersphere is a virtual sphere that indicates a search range. Note that the objects included in the object set S extracted in step S301 are also included in the initial set of the object set R as a search result.

Next, among the objects included in the object set S, the information processing apparatus 100 extracts the object having the shortest distance from the object y, assuming that the search query object is y, and sets it as the object s (step S302). For example, when the only object (node) selected as the root node is an element of S, the information processing apparatus 100 extracts the root node as the object s. Next, the information processing device 100 excludes the object s from the object set S (step S303).

Next, the information processing apparatus 100 determines whether the distance d(s, y) between the object s and the object y exceeds r(1+ε) (step S304). Here, ε is an expansion element, and r(1+ε) is a value indicating the radius of the search range (only nodes within this range are searched. Accuracy can be improved by making it larger than the search range). be. If the distance d(s, y) between object s and object y exceeds r(1+ε) (step S304: Yes), the information processing device 100 outputs the object set R as a neighboring object set of object y (step S305), the process ends.

If the distance d(s, y) between the object s and the search query object y does not exceed r(1+ε) (step S304: No), the information processing device 100 determines the neighboring object set N(G, s) of the object s. One object that is not included in object set C is selected from among the objects that are elements of , and the selected object u is stored in object set C (step S306). The object set C is provided for convenience in order to avoid duplicate searches, and is set to be an empty set at the start of processing.

Next, the information processing device 100 determines whether the distance d(u, y) between the object u and the object y is less than or equal to r(1+ε) (step S307). If the distance d(u,y) between the object u and the object y is less than or equal to r(1+ε) (step S307: Yes), the information processing device 100 adds the object u to the object set S (step S308). Further, if the distance d(u,y) between the object u and the object y is not less than or equal to r(1+ε) (step S307: No), the information processing apparatus 100 performs the determination (process) of step S309.

Next, the information processing device 100 determines whether the distance d(u, y) between the object u and the object y is less than or equal to r (step S309). If the distance d(u, y) between object u and object y exceeds r (step S309: No), the information processing apparatus 100 performs the determination (process) of step S315. That is, if the distance d(u, y) between object u and object y is not less than or equal to r, the information processing apparatus 100 performs the determination (process) of step S315.

If the distance d(u,y) between the object u and the object y is less than or equal to r (step S309: Yes), the information processing device 100 adds the object u to the object set R (step S310). Then, the information processing device 100 determines whether the number of objects included in the object set R exceeds ks (step S311). The predetermined number ks is an arbitrarily determined natural number. For example, ks may be the number of searches or the number of extraction targets. Further, for example, when there is no upper limit on the number of objects to be extracted in a range search or the like, ks may be set to infinity. For example, ks may be 4, etc. If the number of objects included in the object set R does not exceed ks (step S311: No), the information processing device 100 performs the determination (process) of step S313.

If the number of objects included in the object set R exceeds ks (step S311: Yes), the information processing device 100 selects the object that is the longest (farthest) from the object y among the objects included in the object set R. Exclude from object set R (step S312).

Next, the information processing device 100 determines whether the number of objects included in the object set R matches ks (step S313). If the number of objects included in the object set R does not match ks (step S313: No), the information processing device 100 performs the determination (process) of step S315. Further, when the number of objects included in the object set R matches ks (step S313: Yes), the information processing device 100 determines that the distance to the object y is the longest (farthest) among the objects included in the object set R. The distance between the object and object y is set to a new r (step S314).

Then, the information processing device 100 determines whether all objects that are elements of the neighboring object set N(G, s) of the object s have been selected and stored in the object set C (step S315). . If all objects that are elements of the neighboring object set N(G, s) of the object s have not been selected and stored in the object set C (step S315: No), the information processing device 100 selects the objects that are elements of the neighboring object set N(G, s) of the object s and stores them in the object set C (step S315: No). Return to and repeat the process.

When all objects that are elements of the object set N(G, s) in the vicinity of the object s have been selected and stored in the object set C (step S315: Yes), the information processing device 100 It is determined whether or not is an empty set (step S316). If the object set S is not an empty set (step S316: No), the information processing device 100 returns to step S302 and repeats the process. Furthermore, if the object set S is an empty set (step S316: Yes), the information processing device 100 outputs the object set R and ends the process (step S317). For example, the information processing apparatus 100 may select an object (node) included in the object set R as a neighboring node corresponding to the additional node (input object y). For example, the information processing apparatus 100 may extract (select) objects (nodes) included in the object set R as neighboring nodes corresponding to the target node (input object y). Further, for example, the information processing device 100 may provide the objects (nodes) included in the object set R as the search results corresponding to the search query (input object y) to the terminal device or the like that performed the search.

[2. Second embodiment]
The second embodiment will now be described. In the second embodiment, neighboring objects are grouped by clustering or the like, and each group (hereinafter also referred to as a "blob") is subjected to distance calculation at once. That is, in the second embodiment, batch distance calculation is performed in units of blobs. Note that descriptions of points similar to those in the first embodiment will be omitted as appropriate. In the second embodiment, the information processing system 1 includes an information processing device 100A instead of the information processing device 100.

[2-1. Information processing〕
First, an overview of information processing according to the second embodiment will be explained using FIG. 12. FIG. 12 is a diagram illustrating an example of information processing according to the second embodiment.

In the example of FIG. 12, it is assumed that the information processing device 100A has already acquired a graph GR21 as shown in the spatial information SP21. For example, the spatial information SP21 in FIG. 1 may be a Euclidean space. For example, it is assumed that the spatial information SP21 corresponds to the number of dimensions of the vector of the object, and is a multidimensional space such as 100 dimensions or 1000 dimensions. Note that the example of FIG. 12 will be explained based on a diagram in which direct product quantization is not performed to simplify the explanation, but vector (space) may be subjected to direct product quantization as in FIG. 1. .

First, each piece of information shown in FIG. 12 will be explained. Dotted lines connecting white circles (◯) that are nodes in the spatial information SP21 in FIG. 12 indicate edges connecting the nodes. In FIG. 12, an undirected edge is shown as an example as in FIG. 1, but the edges of the graph GR21 are not limited to undirected edges and may be directed edges. Note that notes and edges are the same as those in FIG. 1, so detailed explanations will be omitted.

In the spatial information SP21 of FIG. 12, an area surrounded by straight lines is a region in which nearby objects are grouped together (grouped) by clustering or the like, and the area is called a blob. In the example shown below, blobs are the processing unit for batch distance calculation. FIG. 12 shows a case where clustering is performed into ten regions (blobs) of blobs BL1 to BL10. For example, blob BL1 is a blob to which multiple nodes such as nodes N7, N9, N85, and N126 belong. Note that the black dots in the blobs BL1 to BL10 indicate the centroid (representative vector) of each blob.

Note that the blobs BL1 to BL10 are generated using various methods related to clustering and the like as appropriate. For example, clustering for classifying into blobs may be k-means clustering. Also, for example, clustering for classification into blobs does not use the centroid obtained at the center coordinates (average) of each cluster in one iteration (assignment) with k-means for the next iteration. Alternatively, the centroids may be replaced with the object closest to each centroid before the next iteration. In this case, the centroid of the final cluster obtained by k-means is an existing object, that is, a node. This increases the tendency for clusters (blobs) and edges (neighboring nodes) of each node to match, making it possible to further improve search performance.

In the example of FIG. 12, the information processing device 100A performs a search process using information on a plurality of classified blobs BL1 to BL10 by clustering a plurality of nodes (objects), as shown in the spatial information SP21. .

From here, a search process targeting the search query QE2 will be explained. First, the information processing device 100A obtains the search query QE2 (step S21). For example, the information processing device 100A obtains the search query QE2 from the terminal device 10 (see FIG. 4) used by the user.

Then, the information processing device 100A calculates the distance between the vector of the search query QE2 and the vector of the codebook. Although illustration is omitted, when cross product quantization is not performed, the information processing device 100A calculates the distance (difference) between the vector of the search query QE2 and the vector of the codebook for quantizing the entire vector. . The information processing device 100A then stores the distance (difference) between the vector of the search query QE2 and the vector of each codebook as codebook information TB21 in association with information for identifying each codebook. Note that when performing Cartesian product quantization, the information processing device 100A stores each subvector of the search query QE2 and the vector of each codebook, as shown in codebook information TB1 to TB4 of FIG. 2, as in FIG. Calculate the distance (difference) between Note that the calculation of the distance between the vector of the search query and the vector of the codebook is the same as in FIGS. 1, 2, etc., and therefore detailed explanation will be omitted.

The information processing device 100A executes a search process targeting the search query QE2 (step S22). The information processing device 100A obtains a search result for the search query QE2 by performing a search process as shown in FIG. 16 using the graph GR21 for the search query QE2. Details of the search process shown in FIG. 16 will be described later. The information processing device 100A performs a search process on the search query QE2, thereby extracting the searched nodes as nodes in the vicinity of the search query QE2.

In the search process targeting the search query QE2, the information processing device 100A selects a predetermined node from among the nodes as the node (starting point node) that becomes the starting point for the search of the graph GR21. In the example of FIG. 12, it is assumed that the information processing apparatus 100A selects node N9 as the starting point node. In the example of FIG. 12, the information processing device 100A starts the search process with node N9 as the processing target, for example.

Here, in the search processing targeting the search query QE2, the information processing device 100A parallelizes and calculates distances between the search query and a plurality of nodes belonging to the blob to which the processing target node belongs (step S23). In FIG. 12, as shown in the batch processing information LT2, the information processing device 100A uses the search query QE2 for nodes N7, N9, N85, N126, etc. that belong to the blob BL1 to which the processing target node N9 belongs. Calculate the distance in parallel.

The information processing device 100A uses the distance between each codebook indicated by the codebook information TB21 and the search query QE2 to calculate the distance between each of the nodes N7, N9, N85, and N126 and the search query QE2. For example, the information processing device 100A refers to the codebook information TB21 and calculates the distance of the codebook corresponding to the vector of the node N7 as the distance between the node N7 and the search query QE2. Note that distance calculation when cross-product quantization is performed is the same as in FIGS. 1, 2, etc., and detailed description thereof will be omitted.

Note that, as in the case of the information processing apparatus 100, the units that can be collectively processed are determined based on the specifications of the information processing apparatus 100A. For example, if the number that the information processing device 100A can process at once using SIMD (unit that can be processed at once) is "4", the distances are computed in parallel for four nodes, as in FIG. 1. For example, the information processing device 100A collectively calculates the distance between each of the nodes N7, N9, N85, and N126 and the search query QE2 by parallelizing SIMD calculations. Thereby, the information processing device 100A can speed up the distance calculation by parallelizing the distance calculation, and can perform efficient search processing. In addition, the information processing device 100A uses the graph GR21 to trace nodes (for example, nodes N12, N64) to which edges from the node N7 are connected, and target blobs (for example, blobs BL2, BL3) to which these nodes belong. The search process is performed by performing batch distance calculation. Note that in the search process, the information processing device 100A suppresses the same blob from being repeatedly subjected to batch distance calculation, but details will be described later.

Note that, in order to simplify the explanation, FIG. 12 shows a case where distances are computed in parallel for four nodes, but the number of parallelized nodes is determined based on the specifications of the information processing device 100A. . This point is also similar to FIGS. 1, 2, etc., so detailed explanation will be omitted. Further, the information processing device 100A may use transposed data similarly to the example shown in FIG.

As described above, the information processing device 100A calculates the approximate distance (second distance) between the vector quantized vector of each node and the search query, and performs the search process using the second distance. This enables efficient search processing. Further, the information processing device 100A can perform efficient search processing by collectively performing distance calculations for a number of nodes that can be parallelized. As described above, the information processing apparatus 100A can perform efficient search processing by collectively calculating blobs as a unit of batch processing.

In the search process, the information processing device 100A traces the graph GR21 and performs the above-described process on the nodes to be processed, thereby extracting the searched nodes as nodes in the vicinity of the search query QE2. For example, like the information processing device 100, the information processing device 100A obtains search results using either the first method or the second method described above.

The above-described processing is merely an example, and the information processing device 100A may perform search processing using various information and methods for efficient search processing. Regarding this point, each item will be explained in detail. For example, the information processing device 100A performs clustering and generates blobs so that the number of nodes belonging to each blob becomes the number that the information processing device 100 can process at once using SIMD (batch processable unit). Good too.

Further, the information processing device 100A may use information regarding blobs to suppress redundant processing. The information processing device 100A may have a blob distance calculation flag (hereinafter also simply referred to as a "flag") that indicates whether distance calculation has been performed for each blob, and a table that indicates which blob each object belongs to. good. In this case, the information processing device 100A may manage whether each blob has been processed using a flag. For example, immediately before processing neighboring nodes one by one, the information processing device 100A identifies the blob to which the node belongs based on a table, determines whether the blob has already been subjected to batch distance calculation using a flag, and if the distance has not yet been calculated, performs batch calculation. You may also perform processing on individual nodes. This point will also be explained with reference to FIGS. 15 and 16. Thereby, the information processing device 100A can prevent duplicate processing from being performed.

[2-2. Configuration of information processing device]
Next, the configuration of the information processing apparatus 100A according to the second embodiment will be described using FIG. 13. FIG. 13 is a diagram illustrating a configuration example of an information processing apparatus according to the second embodiment. As shown in FIG. 13, the information processing device 100A includes a communication section 110, a storage section 120A, and a control section 130A. Note that in the information processing device 100A, descriptions of the same points as the information processing device 100 will be omitted as appropriate.

(Storage unit 120A)
The storage unit 120A is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 13, the storage unit 120A according to the second embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123A, a codebook information storage unit 124, and a blob information storage unit 121. It has an information storage section 125.

(Quantization information storage unit 123A)
The quantization information storage unit 123A according to the second embodiment stores various information regarding allocation processing. FIG. 14 is a diagram illustrating an example of a quantization information storage unit according to the second embodiment. In the example of FIG. 14, the quantization information storage unit 123A has items such as "node ID", "object ID", "blob ID", and "quantization information".

"Node ID" indicates identification information for identifying each node (object) in the graph. Moreover, "object ID" indicates identification information for identifying an object. Note that when the node ID and object ID are common, the object ID is stored in the "node ID", and the quantization information storage unit 123A does not need to include the item "object ID".

“Blob ID” indicates information for identifying a blob to which a node (object) belongs.

Moreover, "quantization information" indicates information on quantized vectors of each node (object). For example, "quantization information" includes information such as "element" and "codebook ID." "Element" indicates the arrangement of the corresponding object in the vector. In the example of FIG. 14, the "element" includes "#1", "#2", "#3", and "#4". In this case, the vector of each node (object) is divided into four parts, and each divided partial vector is quantized by the codebook.

The example in FIG. 14 indicates that the node N1 (object OB1) belongs to the blob (blob BL9) identified by the blob ID "BL9". For example, when a vector is divided into four by Cartesian product quantization, the quantization information of node N1 stores the same information as the four codebooks indicating the quantization information of node N1 shown in FIG. Further, for example, when cross-product quantization is not performed, the quantization information of the node N1 stores information indicating one codebook for quantizing the vector of the node N1.

Note that the quantization information storage section 123A is not limited to the above, and may store various information depending on the purpose.

(Blob information storage unit 125)
The blob information storage unit 125 according to the second embodiment stores information regarding blobs. For example, the blob information storage unit 125 stores various information that identifies objects associated with each blob. FIG. 15 is a diagram illustrating an example of a blob information storage unit according to the second embodiment. In the example of FIG. 15, the blob information storage unit 125 includes items such as "blob ID", "node ID", and "vector information".

“Blob ID” indicates identification information for identifying a blob. Further, "node ID" indicates a node (object) associated with the blob identified by the blob ID. "Vector information" indicates vector information of a blob. For example, "vector information" indicates the vector corresponding to the centroid of the blob.

The example shown in FIG. 15 indicates that the nodes (objects) associated with the blob (blob BL1) identified by the blob ID "BL1" are nodes N7, N9, N85, N126, etc. Blob BL1 indicates that multidimensional vector information of "51, 4, 102, 33..." is associated with it.

Further, the nodes (objects) associated with the blob (blob BL9) identified by the blob ID "BL9" are nodes N1, N4, N5, etc. Blob BL9 indicates that multidimensional vector information of "12, 55, 12, 6..." is associated with it.

Note that the blob information storage unit 125 is not limited to the above, and may store various types of information depending on the purpose. The blob information storage unit 125 may store a flag indicating whether each blob is a target of distance calculation processing. For example, the blob information storage unit 125 stores a value indicating that the blob has not been processed as a target for distance calculation (also referred to as "unprocessed flag value") and a value indicating that it has been processed as a target for distance calculation ("processed flag value"). (also referred to as "flag value") as the value of each blob's flag. For example, at the start of the search process, the blob information storage unit 125 sets the value of the flag of each blob to a value (for example, 0) indicating that the distance calculation for that blob has not been processed, and The value of the flag of the blob that has become , is changed to a value (for example, 1) that indicates that distance calculation has been completed.

For example, the information processing device 100A refers to the value of the flag of the blob that is the processing target and determines whether to perform the batch distance calculation process for that blob. The information processing device 100A refers to the flag value of each blob, and if the blob's flag value is an unprocessed flag value, the information processing device 100A determines that the blob has not yet been processed in the batch distance calculation, and processes the blob. Perform bulk distance calculation of nodes belonging to . On the other hand, if the value of the flag of the blob is the processed flag value, the information processing device 100A determines that the blob has been subjected to the batch distance calculation, and performs the batch distance calculation of the nodes belonging to the blob. Not performed.

(Control unit 130A)
Returning to the explanation of FIG. 13, the control unit 130A is a controller, and includes various programs (information processing programs) stored in a storage device inside the information processing device 100A, for example, by a CPU, MPU, GPU, etc. (corresponding to one example) is realized by being executed using RAM as a work area. Further, the control unit 130A is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

As shown in FIG. 13, the control unit 130A includes an acquisition unit 131, a generation unit 132A, a search processing unit 133A, and a provision unit 134, and realizes or executes information processing functions and operations described below. do. Note that the internal configuration of the control unit 130A is not limited to the configuration shown in FIG. 13, and may be any other configuration as long as it performs information processing to be described later.

(Generation unit 132A)
The generation unit 132A generates various information similarly to the generation unit 132.

The generation unit 132A may generate information related to vector quantization as shown in the quantization information storage unit 123. For example, the generation unit 132 generates information indicating that the node N1 (object OB1) belongs to the blob BL9. Further, the generation unit 132 may generate information regarding blobs as shown in the blob information storage unit 125. For example, the generation unit 132 generates information indicating that nodes N7, N9, N85, N126, etc. belong to blob BL1. Note that when the information processing device 100 acquires the information shown in the graph information storage unit 122, quantization information storage unit 123A, codebook information storage unit 124, and blob information storage unit 125 from an external device such as the information providing device 50, The information processing device 100 does not need to include the generation unit 132A.

(Search processing unit 133A)
Similar to the search processing unit 133, the search processing unit 133A performs various processes related to search processing.

The search processing unit 133A uses information on a plurality of blobs in which a plurality of objects are classified in a search process for searching for nodes near a search query in a group of nodes corresponding to each of a plurality of objects. Distances between a plurality of nodes belonging to a blob and a search query are calculated using vector quantized vector information of a plurality of nodes. The search processing unit 133A parallelizes and collectively calculates distances between a plurality of nodes and a search query. The search processing unit 133A performs parallel processing to calculate the distance between the search query and a plurality of nodes whose number is determined based on the specifications of the information processing device 100A.

The search processing unit 133A calculates the distance between the search query and a plurality of nodes belonging to one blob adjacent to the blob to which the search query applies. The search processing unit 133A calculates distances between a plurality of nodes and a search query in a search process that searches for nodes in the vicinity of a search query using a graph in which a plurality of nodes are connected by edges. The search processing unit 133A calculates the distance between the search query and a plurality of nodes belonging to one blob to which a connection node to which edges from the target node to be processed belong in the search process.

The search processing unit 133A calculates the distance between the search query and a plurality of nodes belonging to a blob other than the blob to which the target node belongs. In a search process in which the search processing unit 133A searches for nodes near the search query using a blob graph connected from each of the plurality of nodes to other blobs other than the blob to which each of the plurality of nodes belongs, Calculate the distance between multiple nodes and a search query. In the graph before conversion in which a plurality of nodes are connected by edges, the search processing unit 133A searches for a blob to which a node to which an edge is connected from one of the plurality of nodes belongs, other than the blob to which the one node belongs. In a search process that searches for nodes near a search query using a blob graph, which is a converted graph generated by connecting edges from one node to a blob, multiple nodes and a search query are Calculate the distance.

The search processing unit 133A calculates the distance between the search query and a plurality of nodes belonging to one blob, which is a blob in which edges from the target node to be processed are connected in the search process. The search processing unit 133A uses information indicating a processed blob, which is a blob that has already been a processing target, to determine whether one blob is to be processed. If the one blob is a processed blob, the search processing unit 133A determines that the one blob is not to be processed.

The search processing unit 133A targets a predetermined node among the nodes to be processed in the search process. Calculate distance. In the search process, the search processing unit 133A extracts a second number of nodes, which is a larger number than the first number of nodes to be extracted as nodes in the vicinity of the search query, as neighboring candidate nodes, and selects a second number of nodes as neighboring candidate nodes. Calculate distance. The search processing unit 133A extracts a first number of nodes from among the nearby candidate nodes having the shorter second distance as nodes in the vicinity of the search query.

The search processing unit 133A calculates a second distance for nodes whose first distance is within a predetermined threshold. The search processing unit 133A calculates a second distance for nodes within the search range indicating the target range to be extracted as nearby nodes. The search processing unit 133A calculates a second distance for nodes within a search range indicating a target range of the search process.

[2-3. Search processing example]
Here, an example of the search process according to the second embodiment will be described using FIG. 16 as an example. FIG. 16 is a flowchart illustrating an example of search processing according to the second embodiment. The search processing described below is performed by the search processing unit 133A of the information processing device 100A. Note that descriptions of points similar to those of the first embodiment, such as FIG. 11, will be omitted as appropriate. For example, in FIG. 16, the same steps as those in FIG. 11 are given the same step numbers, and the description thereof will be omitted as appropriate.

Here, the neighboring object set N(G, y) is a set of neighboring objects that are related by edges attached to node y. "G" may be predetermined graph data (for example, graph GR21 shown in spatial information SP21, etc.). For example, the information processing device 100A executes k-nearest neighbor search processing.

For example, the search process shown in FIG. 16 differs from the search process shown in FIG. 11 in that the process shown in step S304a is performed after step S304. In this way, the search process shown in FIG. 16 is a normal graph search, and when a blob that has not been accessed yet is reached, the distance between the object (node) in that blob and the query is calculated all at once. , performs processing on each node.

Specifically, in the search process of FIG. 16, if the distance of the blob to which the object s belongs has not yet been calculated, the information processing device 100A performs all of the following (“-” in step S304a in FIG. 16). The following four processes (hereinafter referred to as "first process" to "fourth process") are executed (step S304a). In step S304a, the information processing device 100A executes a process (first process) of calculating the distances of objects within the blob at once. Furthermore, in step S304a, the information processing device 100A executes a process (second process) of setting a blob distance calculation flag. Furthermore, in step S304a, the information processing apparatus 100A executes a process (third process) of storing all objects in the blob for which the batch distance calculation has been performed in C. Furthermore, in step S304a, the information processing apparatus 100A executes a process (fourth process) in which all objects of the blob are set as u and processes from step S307 to step S314 are performed one by one. After that, the information processing device 100A performs the processing from step S306 onwards.

[2-4. Modified example]
From here, modified examples will be explained. In a modification of the second embodiment, a graph including the concept of blob may be used. For example, in a modification of the second embodiment, a graph may be used in which a blob is referenced by an edge from a node. Note that descriptions of points similar to those in the first embodiment and the second embodiment will be omitted as appropriate. The information processing device 100A according to the modification includes a graph information storage section 122A instead of the graph information storage section 122.

[2-4-1. Information processing〕
First, an overview of information processing according to the modified example will be explained using FIG. 17. FIG. 17 is a diagram illustrating an example of information processing according to a modification.

The information processing device 100A converts the graph GR21 in which nodes are connected by edges into a graph GR31 in which a blob is referenced by an edge from a node (step S31). That is, as shown in FIG. 17, the information processing device 100A generates a graph GR31, which is a graph in which the concept of a blob is introduced (blob graph), from a graph GR21, which is a normal graph.

In the example of FIG. 17, the information processing device 100A generates a graph GR31 including directed edges from nodes to blobs, as shown in spatial information SP31. Note that in the graph GR31, edges between nodes are deleted, and the graph GR31 includes directed edges from nodes to blobs, but does not include edges between nodes. The arrow line shown in FIG. 17 indicates a directed edge from the node at the base of the arrow to the blob at the base of the arrow. That is, the arrow line shown in FIG. 17 indicates a directed edge with the node at the base of the arrow as the reference source and the blob at the tip of the arrow as the reference destination. For example, FIG. 17 shows that edges from node N1 to two blobs, blob BL8 and blob BL10, are connected.

For example, the information processing device 100A converts the edge of each node from an edge to a connection node (nearby node) to an edge to a blob to which the connection node belongs. Note that the information processing device 100A does not generate edges to the blob to which each node itself belongs. In the example of FIG. 17, since the node N9 and the node N126 belong to the same blob BL1, the information processing device 100A does not generate an edge from the node N9 to the blob BL1 and an edge from the node N126 to the blob BL1. On the other hand, in the example of FIG. 17, since the node N9 and the node N18 belong to different blobs BL1 and BL4, the information processing device 100A generates an edge from the node N9 to the blob BL4 and an edge from the node N18 to the blob BL1. .

In the example of FIG. 17, the information processing device 100A performs a search process using a graph GR31 including directed edges from nodes (objects) to blobs, as shown in spatial information SP31. For example, the information processing device 100A obtains the search result of the search query QE2 by performing a search process as shown in FIG. 19 using the graph GR21 for the search query QE2. Details of the search process shown in FIG. 19 will be described later. Note that the search process in FIG. 17 is similar to FIG. 12 in that the search process is performed using the concept of a blob, and is the same as the search process shown in FIG. 12 except for the process related to edges.

[2-4-2. graph〕
Next, an outline of a graph according to a modified example will be explained using FIG. 18. FIG. 18 is a diagram illustrating an example of a graph information storage unit according to a modification. For example, the graph information storage unit 122A shown in FIG. 18 stores a graph (blob graph) in which the concept of blob is introduced. The graph information storage unit 122A shown in FIG. 18 has items such as "node ID", "object ID", and "blob information". In this way, the graph information storage unit 122A according to the modified example differs from the graph information storage unit 122 of FIG. 7 in that it includes “blob information” instead of “connection node information”. Note that descriptions of the same points as those of the graph information storage unit 122 in FIG. 7 will be omitted as appropriate.

Moreover, "blob information" indicates information regarding a blob (a referenced blob) that can be traced from a corresponding node. For example, "blob information" includes information such as "reference destination." “Reference destination” indicates information for identifying a reference destination (blob) that is connected by an edge and can be traced from that node. That is, in the example of FIG. 18, a reference destination (blob) that can be traced from the node by an edge is registered in association with a node ID (object ID) that identifies the node. Note that the "blob information" may include information (edge ID) for identifying an edge connected to a reference destination.

The example in FIG. 18 indicates that the node (node N1) identified by the node ID "N1" corresponds to the object (target) identified by the object ID "OB1". Furthermore, an edge is connected from the node N1 to the blob (blob BL8) identified by the blob ID "BL8", indicating that it is possible to trace from the node N1 to the blob BL8. Further, an edge is connected from the node N1 to the blob (blob BL10) identified by the blob ID "BL10", indicating that the blob BL10 can be traced from the node N1.

Further, the node (node N2) identified by the node ID "N2" corresponds to the object (target) identified by the object ID "OB2". Furthermore, an edge is connected from the node N2 to the blob (blob BL5) identified by the blob ID "BL5", indicating that the blob BL5 can be traced from the node N1.

Note that the graph information storage unit 122A is not limited to the above, and may store various information depending on the purpose.

The graph may also include a program module that receives a query as input, searches for nodes by tracing edges in the graph, and extracts and outputs nodes similar to the query. That is, the graph may be assumed to be used as a program module that performs search processing using the graph. For example, the graph GR31 may be a program that, when vector data is input as a query, extracts nodes corresponding to vector data similar to the vector data from the graph and outputs them. For example, the graph GR31 may be data used as a program module that searches for similar images corresponding to a query image. For example, based on an input query, the graph GR31 causes the computer to extract and output nodes similar to the query in the graph.

[2-4-3. Search processing example]
An example of the search process according to the modified example will now be described using FIG. 19 as an example. FIG. 19 is a flowchart illustrating an example of a search process according to a modification. The search processing described below is performed by the search processing unit 133A of the information processing device 100A. Note that descriptions of points similar to those of the first embodiment or the second embodiment, such as FIGS. 11 and 16, will be omitted as appropriate. For example, in FIG. 19, the same steps as those in FIG. 11 are given the same step numbers, and the description thereof will be omitted as appropriate.

In FIG. 19, the set N(G, y) is different from the search processing in FIGS. 11 and 16 in that it is a set of blobs related by edges attached to node y. In FIG. 19, N(G, s) and C are blob sets. That is, "nearby object set N" in FIG. 11 is replaced with "blob set N" in FIG. 19, and "object set C" in FIG. 11 is replaced with "blob set C" in FIG. . Furthermore, the word "object" in the process related to the change to a blob can be read as "blob" as appropriate. "G" may be graph (blob graph) data (for example, graph GR31 shown in spatial information SP31) into which the concept of blob has been introduced. For example, the information processing device 100A executes k-nearest neighbor search processing.

For example, in the search process shown in FIG. 19, the process in step S306 from FIG. 11 is changed as follows. In the search process in FIG. 19, the information processing device 100A selects one blob from N(G, s) that is not included in the blob set C, stores the selected blob in the blob set C, and stores the selected blob in the blob set C. The objects within the blob are collectively calculated, and the set thereof is set as B (also referred to as "object set B within the target blob") (step S306).

Furthermore, the search process shown in FIG. 19 differs from the search process shown in FIG. 11 in that the process shown in step S306a is performed after step S306. Specifically, in the search process of FIG. 19, the information processing device 100A selects one object u from the target blob object set B (step S306a). After that, the information processing device 100A performs the processing from step S307 onwards.

The search process shown in FIG. 19 is different from the search process shown in FIG. 11 in that the process shown in step S314a is performed before step S315. Specifically, in the search process of FIG. 19, the information processing device 100A determines whether all objects have been selected from object set B in the target blob (step S314a).

If all objects have been selected from object set B within the target blob (step S314a: Yes), the information processing device 100A performs the process of step S315. On the other hand, if all objects have not been selected from object set B within the target blob (step S314a: No), the information processing device 100A returns to step S306a and repeats the process.

[2-4-4. Other examples of generation processing]
In the above example, a case was shown in which a blob graph is generated by converting the edge of each node from an edge to a connected node to an edge to a blob to which the connected node belongs, but the information processing device 100A converts various information. It may be used as appropriate to generate a blob graph. For example, the information processing device 100A may generate a blob graph using a neighborhood search index (also simply referred to as an "index") that searches for multiple objects. Note that descriptions of points similar to those described above will be omitted as appropriate.

An example of generating a blob graph will be described using FIG. 20. FIG. 20 is a diagram illustrating another example of information processing according to the modification. In addition, although the case where a graph is used as an example of an index is demonstrated below as an example, the index may be any type of index as long as it searches for a plurality of objects. For example, the index may be an index using a hash-related description such as a hash table (hash index), an index having a tree structure (tree index), or the like.

Further, information indicating an object or a node corresponding to the object (also referred to as an "object node") may be described as first information, and information used to classify a plurality of objects may be described as second information. Further, information indicating a blob may be described as third information, and a graph in which object nodes are connected by edges may be described as fourth information. For example, the graph GR21 in FIG. 20 has node N1, which is an object node, connected by edges, and corresponds to the fourth information. Also, unlike the fourth information, graph GR31 in FIG. 17 and graph GR32 in FIG. 20 are blob graphs (also referred to as "group graphs") in which object nodes and blobs are connected at edges. In other words, the group graph is a graph in which edges are connected from object nodes to blobs, the object nodes are reference sources, and the blobs (groups) are reference destinations.

The information processing device 100A uses the graph GR21 in which object nodes are connected by edges as an index (fourth information) to generate a graph GR32 that is a group graph in which blobs are referenced by edges from object nodes (step S41). In FIG. 20, the information processing device 100A generates a graph GR32 including directed edges from nodes to blobs, as shown in spatial information SP32.

For example, the information processing device 100A selects one object from a plurality of objects, searches for neighboring objects of the one object using the graph GR21, and determines which neighboring objects belong from one object node corresponding to the one object. A graph GR32 is generated by a connection process that connects edges to other groups. For example, the information processing device 100A randomly selects one object from a plurality of objects, performs a concatenation process on the selected object, and generates the graph GR32.

For example, the information processing device 100A performs the following processes (1-1) to (1-4) to generate the graph GR32. Note that the object may be read as a node corresponding to the object.

(1-1): Obtain object NX randomly (selection)
(1-2): Search for a constant number of k neighboring objects of object NX (1-3): Obtain the blob to which each neighboring object belongs (1-4): Generate an edge to the blob obtained from object NX

For example, in process (1-1), the information processing device 100A randomly acquires one object NX from a plurality of objects stored in the object information storage unit 121.

For example, in process (1-2), the information processing device 100A performs a search process as shown in FIG. is extracted as a neighboring object of object NX.

For example, in process (1-3), the information processing device 100A refers to blob information stored in the blob information storage unit 125 and obtains blobs associated with neighboring objects (nodes) of object NX.

For example, in process (1-4), the information processing device 100A generates an edge to the blob acquired from the object NX. For example, the information processing device 100A generates the graph GR32 by associating information indicating the acquired blob with the object NX in the quantization information storage unit 123A.

For example, the information processing device 100A repeatedly performs the above processes (1-1) to (1-4) until there are no more objects to be processed, and generates the graph GR32. Note that when the blob shown in the spatial information SP21 is generated by a search, the graph GR32 may be generated by using the search results when the blob is generated. Note that an example of generating blobs by search will be described later.

For example, the information processing device 100A performs a search process using a graph GR32 including directed edges from nodes (objects) to blobs, as shown in spatial information SP32. For example, the information processing device 100A obtains the search result of the search query QE2 by performing a search process as shown in FIG. 19 using the graph GR32 for the search query QE2.

Further, in the above example, a case was shown in which blobs were generated by clustering, but blobs are not limited to clustering, and may be generated by various methods. For example, the information processing device 100A may generate information indicating a blob (third information) using second information such as an index.

The information processing device 100A uses the graph GR21 as an index as shown in FIG. 21 to generate third information indicating a blob. FIG. 21 is a diagram illustrating an example of blob information generation processing. Spatial information SP20 in FIG. 21 shows a state before the blob information in spatial information SP21 is generated, and graph GR21 in FIG. 21 is a graph similar to graph GR21 in FIG. 20. Note that the process shown in FIG. 21 shows a part of the first method described later, and the details will be described later.

For example, the information processing device 100A generates third information that classifies a plurality of objects into a plurality of blobs by searching using the graph GR21. For example, the information processing device 100A selects one node (also referred to as a "processing target node") from a plurality of nodes in the graph GR21, and selects at least one of the neighboring nodes that are nodes connected to the processing target node by an edge. The third information is generated by a classification process of classifying a group of objects corresponding to each of the classification target node, which is a node of the section, and the processing target node into one group. For example, the information processing device 100A selects a processing target node from a plurality of nodes by a predetermined process, performs a classification process on the selected processing target node, and generates third information.

For example, the information processing device 100A may select a processing target node based on the number of neighboring nodes (connection nodes) connected to the node by edges. For example, the information processing apparatus 100A may select processing target nodes in descending order of the number of neighboring nodes. A method of selecting processing target nodes in order of increasing number of neighboring nodes and generating third information indicating a blob is also referred to as a "first method." Further, for example, the information processing apparatus 100A may select processing target nodes in descending order of the number of neighboring nodes (connected nodes). A method of selecting processing target nodes in order of decreasing number of neighboring nodes and generating blobs is also referred to as a "second method." Further, for example, the information processing apparatus 100A may randomly select a processing target node. A method of randomly selecting processing target nodes and generating third information indicating a blob is also referred to as a "third method." Below, the outline of each processing procedure of the first method, second method, and third method will be explained.

[2-4-4-1. 1st method]
First, the first method will be explained. For example, in the case of the first method, the information processing device 100A performs the following processes (2-1) to (2-3) to generate third information indicating a blob.

(2-1): Obtain the node ND from the one with the largest number of neighboring nodes (number of edges) for each node (2-2): Set the neighboring nodes of the node ND and the node ND as one blob (2-3): Delete blob nodes and edges of those nodes

For example, the information processing device 100A performs processes (2-1) to (2-3) on all nodes.

For example, in process (2-1), the information processing device 100A refers to the graph GR21 and sequentially acquires one node ND. The information processing device 100A may obtain one node ND using list information of nodes arranged in descending order of the number of neighboring nodes.

For example, in process (2-2), the information processing device 100A refers to the graph to be processed (graph GR21, etc.) and generates information regarding the neighboring nodes of the node ND and the node ND as one blob. Note that the information processing device 100A may set neighboring nodes that are less than or equal to a constant K as blobs instead of setting all neighboring nodes as blobs. For example, instead of making all the neighboring nodes of the node ND into a blob together with the node ND, the information processing device 100A may make K neighboring nodes and the node ND among the neighboring nodes into one blob.

For example, in process (2-3), the information processing device 100A deletes the node that was made into a blob in process (2-2) and the edges of that node so that nodes are not selected redundantly. For example, the information processing device 100A deletes the blob (edge and) node if the node is a neighboring node of another node. Note that the above method is only an example, and the process (2-3) is not limited to the method of deleting edges and the like, but may be any method as long as it is possible to generate blobs. For example, the information processing device 100A may generate a blob by maintaining a list of nodes that already belong to the blob and managing nodes to be excluded. For example, the information processing device 100A may manage nodes that already belong to a blob by setting a predetermined flag (such as a deletion flag) on the node that is a blob. In this way, when using flags, the information processing apparatus 100A refers to the flags of each node during processing and determines whether that node is to be processed.

A specific example of the first method described above will be explained using FIG. 21. In FIG. 21, the information processing device 100A acquires the node N7, which has the maximum number of neighboring nodes of four, from the graph GR21 (step S51). In FIG. 21, the node N1 in the graph GR21 also has four neighboring nodes, but the information processing device 100A acquires the node N7. Note that when there are multiple nodes with the same number of neighboring nodes, the information processing device 100A may randomly acquire nodes from the multiple nodes.

Then, the information processing device 100A sets four nodes N9, N12, N54, and N126, which are neighboring nodes of the node N7, and five nodes including the node N7, as one blob BL11 (step S52). The information processing device 100A generates third information BLT1 indicating that five nodes N7, N9, N12, N54, and N126 belong to one blob BL11. The information processing device 100A then deletes the five nodes N7, N9, N12, N54, and N126 and the edges of each node from the graph GR21. As a result, the graph GR21 is updated to the graph GR21-1, as shown in the spatial information SP20-1.

Then, the information processing device 100A obtains the node N1 having four neighboring nodes from the graph GR21 (step S53). Then, the information processing device 100A sets four nodes N4, N5, N88, and N99, which are neighboring nodes of the node N1, and five nodes including the node N1, as one blob BL12 (step S54). The information processing device 100A generates third information BLT2 indicating that five nodes N1, N4, N5, N88, and N99 belong to one blob BL12. Then, the information processing device 100A deletes the five nodes N1, N4, N5, N88, and N99 and the edges of each node from the graph GR21-1.

The information processing device 100A repeats the above-described processing until there are no more nodes to be processed, and generates third information indicating a plurality of blobs.

[2-4-4-2. Second method]
Next, the second method will be explained. For example, in the case of the second method, the information processing device 100A performs the following processes (3-1) to (3-3) to generate third information indicating a blob.

(3-1): Obtain nodes ND from the smallest number of neighboring nodes (number of edges) for each node (3-2): Set neighboring nodes of node ND and node ND as one blob (3-3): Delete blob nodes and edges of those nodes

For example, the information processing device 100A performs processes (3-1) to (3-3) on all nodes.

For example, in process (3-1), the information processing device 100A refers to the graph GR21 and sequentially acquires one node ND. The information processing device 100A may obtain one node ND using list information of nodes arranged in descending order of the number of neighboring nodes.

For example, in the process (3-2), the information processing device 100A refers to the graph to be processed (graph GR21, etc.) and generates information regarding the neighboring nodes of the node ND and the node ND as one blob. Note that the information processing device 100A may set neighboring nodes that are less than or equal to a constant K as blobs instead of setting all neighboring nodes as blobs. For example, instead of making all the neighboring nodes of the node ND into a blob together with the node ND, the information processing device 100A may make K neighboring nodes and the node ND among the neighboring nodes into one blob.

For example, in process (3-3), the information processing apparatus 100A deletes the node that was made into a blob in process (3-2) and the edges of that node so that nodes are not selected twice. For example, the information processing device 100A deletes the blob (edge and) node if the node is a neighboring node of another node. Note that the second method is the same as the first method except that the order of node acquisition is from the smallest number of neighboring nodes (number of edges), so a description of specific examples and the like will be omitted. For example, like the first method, the process (3-3) is not limited to the method of deleting edges and the like, but may be any method as long as it is possible to generate blobs. For example, similar to the first method, the information processing device 100A may generate a blob by maintaining a list of nodes that already belong to the blob and managing nodes to be excluded.

[2-4-4-3. Third method]
Next, a third method will be explained. For example, in the case of the third method, the information processing device 100A performs the following processes (4-1) to (4-3) to generate third information indicating a blob.

(4-1): Randomly acquire nodes ND in sequence (4-2): Make neighboring nodes of node ND and node ND one blob (4-3): Delete the blob node and the edges of that node

For example, the information processing device 100A performs processes (4-1) to (4-3) on all nodes. Note that the third method is similar to the first method and the second method, except that the order of node acquisition is random, and detailed description thereof will be omitted. For example, the information processing device 100A may generate the third information in random order (third method) if the number of edges is the same.

[2-4-4-4. Fourth method]
Note that the first to third methods described above are merely examples, and the information processing apparatus 100A may generate blobs by various processes. For example, the information processing device 100A may generate a blob through a search using a neighborhood search index that searches for multiple objects. The method of generating blobs by searching using an index is also referred to as the "fourth method."

The fourth method will be explained below. Note that descriptions of points similar to those described above will be omitted as appropriate. In the following, a case will be described in which a graph is used as an example of the index, but as described above, the index may be of any type as long as it searches for a plurality of objects. For example, the index may be any index such as a hash index or a tree index.

From here, a case will be described in which the information processing device 100A generates third information indicating a blob by searching using the graph GR21 as an index as shown in FIG. 21. For example, the information processing device 100A performs the following processes (5-1) to (5-4) to generate the graph GR32.

(5-1): Obtain object NX randomly (selection)
(5-2): Search for a constant number of k neighboring objects of object NX (5-3): Make neighboring objects of object NX and object NX one blob (5-4): Execute processing to avoid duplicate processing

For example, in process (5-1), the information processing device 100A randomly acquires one object NX from the graph GR21.

For example, in process (5-2), the information processing device 100A performs a search process as shown in FIG. 11 using the graph GR21 with the object NX as the target (query). is extracted as a neighboring object of object NX. Note that the information processing device 100A may scan (search) all the objects and extract neighboring objects of the object NX.

For example, in process (5-3), the information processing device 100A generates information that makes the neighboring nodes of the neighboring objects of the object NX and the object NX one blob.

For example, in process (5-4), the information processing device 100A executes a process for avoiding duplicate processing so that one object does not belong to multiple blobs due to duplicate searches or the like. For example, the information processing device 100A deletes the blob object from the index. For example, the information processing device 100A deletes the blob object from the graph GR21.

Note that the information processing apparatus 100A is not limited to the above method, and may perform processing for avoiding duplicate processing using any method as long as one object does not belong to multiple blobs. For example, the information processing device 100A may set a predetermined flag (such as a deletion flag) on an object that has been made into a blob, thereby preventing objects that already belong to the blob from becoming processing targets. In this way, when using flags, the information processing apparatus 100A refers to the flag of each object during processing and determines whether the object is to be processed. For example, when the information processing device 100A uses the search results to generate the group graph described above, the information processing device 100A may perform processing to avoid duplicate processing using a method using a flag.

[2-4-5. Information processing device according to modification]
In the information processing device 100A according to the modification of the second embodiment, the acquisition unit 131 also performs the following processing. The acquisition unit 131 acquires first information indicating a plurality of objects to be searched for data, and second information used to classify the plurality of objects. The acquisition unit 131 acquires second information that is an index that searches for a plurality of objects. The acquisition unit 131 acquires second information that is a graph in which a plurality of nodes corresponding to each of a plurality of objects are connected by edges. The acquisition unit 131 acquires fourth information that is an index that searches for multiple objects. The acquisition unit 131 acquires fourth information that is a graph in which a plurality of object nodes corresponding to each of a plurality of objects are connected by edges.

Furthermore, in the information processing apparatus 100A according to the modification of the second embodiment, the generation unit 132A also performs the following processing. The generation unit 132A uses the second information acquired by the acquisition unit 131 to classify the plurality of objects indicated by the first information, and classifies the plurality of objects used for batch processing in the search process targeting the plurality of objects. Third information indicating the group is generated. The generation unit 132A uses the second information to generate third information that classifies the plurality of objects into a plurality of groups. The generation unit 132A generates third information for classifying a plurality of objects into a plurality of groups by searching using the second information.

The generation unit 132A selects one node from a plurality of nodes in the graph, and generates classification target nodes that are at least some nodes among neighboring nodes that are nodes connected to the one node by edges, and the one node. The third information is generated by a classification process that classifies the object group corresponding to each of the objects into one group. The generation unit 132A generates the third information through a classification process of classifying into one group a group of objects corresponding to each of the classification target nodes, which are a predetermined number of nodes among the nodes in the vicinity of one node, and one node. generate. The generation unit 132A generates the third information through a classification process of classifying the classification target node, which is all of the neighboring nodes of one node, and the object group corresponding to each of the one node into one group.

The generation unit 132A selects one node from the plurality of nodes based on the number of neighboring nodes that are nodes connected to each of the plurality of nodes by edges, and selects one node from the plurality of nodes, and selects a classification target node of the one node and a node of the one node. The third information is generated by a classification process of classifying the corresponding object groups into one group. The generation unit 132A selects one node in descending order of the number of neighboring nodes, and performs a classification process of classifying the classification target node of one node and the object group corresponding to each of the one node into one group. Generate third information. The generation unit 132A selects one node in descending order of the number of neighboring nodes, and performs a classification process of classifying the classification target node of one node and the object group corresponding to each of the one node into one group. Generate third information.

The generation unit 132A randomly selects one node from the plurality of nodes, and generates third information by performing a classification process of classifying the classification target node of the one node and the object group corresponding to each of the one node into one group. generate. The generation unit 132A excludes the classification target node of the one node classified into the one group and the processed node group that is the one node from the graph, and performs classification using the graph after excluding the processed node group. Third information is generated by repeating the process. The generation unit 132A generates third information by repeating the classification process using the graph after excluding the processed node group until a plurality of objects are classified into any group. The generation unit 132A generates third information indicating a plurality of groups to which objects each belong to are mutually exclusive.

Using the fourth information, the generation unit 132A generates an edge from an object node, which is a node corresponding to at least some of the plurality of objects, to a group other than the group to which the object node belongs among the plurality of groups. Generates a connected group graph. The generation unit 132A selects one object from the plurality of objects, searches for neighboring objects of the one object among the plurality of objects using the fourth information, and selects one object from the one object node corresponding to the one object. , a group graph is generated by a connection process that connects edges to other groups to which neighboring objects belong. The generation unit 132A randomly selects one object from a plurality of objects, searches for neighboring objects of the one object, and creates an edge from one object node corresponding to the one object to another group to which the neighboring object belongs. A group graph is generated by the concatenation process. The generation unit 132A generates a group graph using the fourth information.

[2-4-6. Information processing flow]
Next, an information processing procedure according to a modified example will be described using FIG. 22. FIG. 22 is a flowchart illustrating an example of information processing according to a modification.

As shown in FIG. 22, the information processing device 100A acquires first information indicating a plurality of objects that are data search targets (step S401). For example, the information processing device 100A acquires first information indicating a plurality of objects from the object information storage unit 121.

The information processing device 100A also acquires second information used to classify a plurality of objects (step S402). For example, the information processing device 100A obtains a graph as the second information from the graph information storage unit 122A.

The information processing device 100A uses the second information to classify the plurality of objects indicated by the first information, and indicates a plurality of groups used for performing batch processing in a search process targeting a plurality of objects. Third information is generated (step S403). For example, the information processing device 100A generates third information indicating a plurality of blobs in which objects belonging to each blob are mutually exclusive.

[3. Third embodiment]
In the above example, there are two graphs: a graph (also called an "object graph") in which nodes corresponding to objects (object nodes) are connected by edges, and a graph (blob graph) in which object nodes and groups (blobs) are connected by edges. Although the description uses two types of graphs, the information processing system 1 may use various types of graphs. For example, unlike object graphs and blob graphs, the information processing system 1 generates a graph that connects groups (blobs) (hereinafter also referred to as a "group connected graph" or "blob connected graph"), and generates a blob connected graph. may be used for the search.

This point will be described below as a third embodiment. In the third embodiment, the information processing system 1 includes an information processing device 100B instead of the information processing device 100 or the information processing device 100A. Note that descriptions of points similar to those described above in the first embodiment, second embodiment, etc. will be omitted as appropriate.

The information processing device 100B uses the index (index information) to connect edges from a first group (first blob) to which one object belongs to a second group (second blob) to which neighboring objects of one object belong. generate a group connected graph. In the following, a case will be described in which a graph is used as an example of the index, but as described above, the index may be of any type as long as it searches for a plurality of objects. For example, the index may be any index such as a hash index or a tree index.

[3-1. Information processing〕
First, an overview of information processing according to the third embodiment will be explained using FIG. 23. FIG. 23 is a diagram illustrating an example of information processing according to the third embodiment.

In the example of FIG. 23, it is assumed that the information processing apparatus 100B has already acquired a graph GR21 as shown in the spatial information SP21. Note that the information processing device 100B may generate the graph GR21 using various techniques related to graph generation as appropriate. Since the graph GR21 is similar to the graph GR21 shown in FIG. 12 etc., the explanation will be omitted.

The information processing device 100B generates information indicating the blobs BL1 to BL10, etc. using an arbitrary method. The information processing device 100B may classify each node into one of the blobs BL1 to BL10 using arbitrary clustering such as k-means. Note that the information processing device 100B may use any method to generate blobs as long as it is capable of generating blobs. For example, the information processing device 100B may generate information indicating the blobs BL1 to BL10, etc. using any of the first to fourth methods described above.

The information processing device 100B generates a blob connection graph using the graph GR21 as shown in the spatial information SP21 and the blob information (step S61). In the example of FIG. 23, the information processing device 100B uses a graph GR41, which is a blob connection graph in which a blob (first blob) and another blob (second blob) are connected by edges, as shown in spatial information SP41. generate. The arrow line shown in FIG. 23 indicates a directed edge from the blob at the base of the arrow (first blob) to the blob at the tip (second blob). That is, the arrow line shown in FIG. 23 indicates a directed edge that uses the blob at the beginning of the arrow as a reference source and the blob at the tip of the arrow as a reference destination. For example, FIG. 23 shows that edges from blob BL1 to five blobs, blob BL2, blob BL3, blob BL4, blob BL5, and blob BL8, are connected.

For example, the information processing device 100B searches for graph GR21 and generates graph GR41 based on the search results. For example, the information processing device 100B performs the following processes (6-1) to (6-4) to generate a graph GR41 that is a blob connection graph. Note that the following objects may be replaced with nodes corresponding to the objects.

(6-1): Obtain object NX randomly (selection)
(6-2): Search constant k neighboring objects of object NX (6-3): Obtain the blob to which each neighboring object belongs (6-4): Generate an edge from the blob to which object NX belongs to the obtained blob

For example, in process (6-1), the information processing device 100B randomly acquires one object NX from a plurality of objects stored in the object information storage unit 121.

For example, in process (6-2), the information processing device 100B performs a search process as shown in FIG. is extracted as a neighboring object of object NX.

For example, in process (6-3), the information processing device 100B refers to the blob information stored in the blob information storage unit 125 and obtains a blob to which a neighboring object (node) of the object NX is associated.

For example, in process (6-4), the information processing device 100B generates an edge from the blob to which the object NX belongs to the acquired blob. The information processing device 100B generates edges from the blob to which the object NX belongs to the blobs to which all neighboring objects belong. For example, the information processing device 100B associates the information (blob ID) indicating the blob acquired as a reference destination of the blob to which the object NX belongs with the information (blob ID) indicating the blob to which the object NX belongs, and generates blob connection graph information. By registering in the storage unit 126, a graph GR41 is generated.

For example, the information processing device 100B repeatedly performs the above processes (6-1) to (6-4) until there are no more objects to be processed, and generates the graph GR41. Note that if blob Y has already been registered as a reference destination of blob The same blob may be prevented from being registered redundantly in the reference destination of each blob.

For example, the information processing device 100B performs a search process using a graph GR41 that connects blobs with edges, as shown in spatial information SP41. For example, the information processing device 100B obtains the search results for the search query QE2 by performing search processing as shown in FIGS. 27 and 28 using the graph GR41, which is a blob connection graph, for the search query QE2.

The above-described method for generating a blob connected graph is merely an example, and the information processing apparatus 100B may generate a blob connected graph using various methods. For example, the information processing device 100B may generate a blob connection graph without searching for a graph. When the blob shown in the spatial information SP21 is generated by a search, a blob connection graph such as the graph GR41 may be generated by using the search results at the time of blob generation.

For example, the information processing device 100B may generate a blob connection graph by converting the connection relationship of object nodes in the graph GR21 into the connection relationship of blobs to which the object nodes belong. For example, an edge is connected from a node N1 belonging to blob BL9 to a node N88 belonging to blob BL8 and a node belonging to blob BL10, and an edge is connected from node N4 belonging to blob BL9 to a node belonging to blob BL7. . Therefore, the information processing device 100B generates a blob connection graph in which edges from blob BL9 to three blobs, blob BL7, blob BL8, and blob BL10, are connected. In this way, the information processing apparatus 100B may generate a blob connection graph that connects blobs with edges, based on edges that connect object nodes.

[3-2. Configuration of information processing device]
Next, the configuration of the information processing apparatus 100B according to the third embodiment will be described using FIG. 24. FIG. 24 is a diagram illustrating a configuration example of an information processing apparatus according to the third embodiment. As shown in FIG. 24, the information processing device 100B includes a communication section 110, a storage section 120B, and a control section 130B. Note that in the information processing device 100B, descriptions of the same points as the information processing device 100 or the information processing device 100A will be omitted as appropriate.

(Storage unit 120B)
The storage unit 120B is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 24, the storage unit 120B according to the third embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123, a codebook information storage unit 124, and a blob information storage unit 121. It has an information storage section 125 and a blob connection graph information storage section 126.

(Blob connection graph information storage unit 126)
The blob connection graph information storage unit 126 according to the third embodiment stores various information regarding the blob connection graph. For example, the blob connection graph information storage unit 126 stores the generated blob connection graph. FIG. 25 is a diagram illustrating an example of a blob connection graph information storage unit according to the third embodiment. The blob connection graph information storage unit 126 shown in FIG. 25 has items such as "blob ID" and "connection blob information."

“Blob ID” indicates identification information for identifying each blob (group) in the graph. “Connection blob information” indicates information regarding a blob (reference destination blob) that can be traced from a corresponding blob. For example, "connection blob information" includes information such as "reference destination." “Reference destination” indicates information for identifying a reference destination (blob) that is connected by an edge and can be traced from the blob. That is, in the example of FIG. 25, a reference destination (blob) that can be traced from the blob by an edge is registered in association with a blob ID that identifies the blob. Note that the "connection blob information" may include information (edge ID) for identifying an edge connected to a reference destination.

In the example of FIG. 25, the blob (blob BL1) identified by the blob ID "BL1" is identified by each of the blob IDs "BL2", "BL3", "BL4", "BL5", and "BL8". Shows that edges are connected to five blobs. That is, it shows that each of the five blobs BL2, BL3, BL4, BL5, and BL8 can be traced from the blob BL1.

Note that the blob connection graph information storage unit 126 is not limited to the above, and may store various information depending on the purpose. The blob connection graph may also include a program module that receives a query as input, searches for objects (object nodes) by tracing edges in the blob connection graph, and extracts and outputs objects similar to the query. That is, the blob connection graph may be assumed to be used as a program module that performs search processing using the blob connection graph. For example, the graph GR41, which is a blob connection graph, may be a program that, when vector data is input as a query, extracts and outputs objects corresponding to vector data similar to the vector data using a blob connection graph. . For example, the graph GR41 may be data used as a program module that searches for similar images corresponding to a query image. For example, based on an input query, the graph GR41 causes the computer to extract and output objects similar to the query in the blob connection graph.

(Control unit 130B)
Returning to the explanation of FIG. 24, the control unit 130B is a controller, and includes various programs (information processing programs) stored in a storage device inside the information processing device 100B by, for example, a CPU, MPU, GPU, etc. (corresponding to one example) is realized by being executed using RAM as a work area. Further, the control unit 130B is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

As shown in FIG. 24, the control unit 130B includes an acquisition unit 131, a generation unit 132B, a search processing unit 133B, and a provision unit 134, and realizes or executes information processing functions and operations described below. do. Note that the internal configuration of the control unit 130B is not limited to the configuration shown in FIG. 24, and may be any other configuration as long as it performs information processing to be described later.

The acquisition unit 131 according to the third embodiment acquires blob information indicating a plurality of blobs in which a plurality of objects to be searched are classified, and index information indicating an index for searching a plurality of objects. do. The acquisition unit 131 acquires index information indicating a graph in which a plurality of nodes corresponding to each of a plurality of objects are connected by edges. The acquisition unit 131 acquires blob information indicating a plurality of blobs in which a plurality of objects are classified by clustering processing. The acquisition unit 131 acquires blob information indicating a plurality of blobs, each of which has mutually exclusive objects. The acquisition unit 131 acquires a query.

(Generation unit 132B)
The generation unit 132B generates various types of information in the same way as the generation unit 132 or the generation unit 132A.

The generation unit 132B uses the index information acquired by the acquisition unit 131 to generate an edge from a first blob to which one of the plurality of objects belongs to a second blob to which a neighboring object of the first object belongs. Generate a connected blob connectivity graph. The generation unit 132B selects one object from the plurality of objects, searches for the previous neighboring objects of the one object among the plurality of objects using the index information, and selects the neighboring objects from the first blob to which the one object belongs. A blob connectivity graph is generated by connecting edges to the second blob to which the object belongs.

The generation unit 132B randomly selects one object from a plurality of objects, searches for neighboring objects of the one object, and connects edges from the first blob to which the one object belongs to the second blob to which the neighboring object belongs. By doing this, a blob connected graph is generated. The generation unit 132B uses the graph to generate a blob connection graph in which edges are connected from the first blob to the second blob.

The generation unit 132B uses the graph to search for neighboring objects of one object, and generates a blob connection graph by connecting edges from the first blob to which the one object belongs to the second blob to which the neighboring object belongs. do. The generation unit 132B connects an edge from the first blob to a second blob to which a neighboring object corresponding to a neighboring node, which is a node connected by an edge to one node corresponding to one object in the graph, belongs. Generate a blob connected graph.

The generation unit 132B generates a blob connection graph by connecting, from the first blob, a first representative point that is a representative point of the first blob and a second representative point that is a representative point of a second blob. generate. The generation unit 132B generates a blob connection graph by connecting the first representative point, which is the centroid of the first blob, and the second representative point, which is the centroid of the second blob, with an edge. The generation unit 132B generates an image between a first representative point, which is a center point calculated based on an object belonging to the first blob, and a second representative point, which is a center point calculated based on an object belonging to a second blob. A blob-connected graph is generated by connecting the blobs with edges.

(Search processing unit 133B)
The search processing unit 133B performs various processes related to search processing in the same way as the search processing unit 133 or the search processing unit 133A.

The search processing unit 133B performs a search process using the blob connection graph generated by the generation unit 132B. The search processing unit 133B performs a search process to search for nearby objects of the search query using the blob connection graph. The search processing unit 133B performs a search process to search for nearby objects of the search query by tracing edges connecting blobs in the blob connection graph.

In the search process, the search processing unit 133B uses blob information indicating a plurality of blobs to calculate the distance between the target object, which is an object belonging to one blob, and the search query, and the vector information of the target object that has been vector quantized. Calculate using The search processing unit 133B parallelizes and collectively calculates the distance between the target object and the search query. The search processing unit 133B performs parallel processing to calculate the distance between the search query and the target object of the batch processing number determined based on the specifications of the information processing device.

[3-3. Information processing flow]
Next, the information processing procedure according to the third embodiment will be described using FIG. 26. FIG. 26 is a flowchart illustrating an example of information processing according to the third embodiment.

As shown in FIG. 26, the information processing apparatus 100B acquires group information indicating a plurality of groups into which a plurality of objects to be data searched are classified (step S501). For example, the information processing device 100B acquires blob information indicating a plurality of blobs in which nodes (object nodes) corresponding to a plurality of objects to be data searched are classified.

The information processing apparatus 100B acquires index information indicating an index that searches for a plurality of objects (step S502). For example, the information processing device 100B obtains, as an index, a graph in which nodes (object nodes) corresponding to a plurality of objects are connected by edges.

The information processing device 100B uses index information to create a group connection graph in which edges are connected from a first group to which one of the plurality of objects belongs to a second group to which a neighboring object of the one object belongs. is generated (step S503). For example, the information processing device 100B uses index information to create a blob whose edges are connected from a first blob to which one of the plurality of objects belongs to a second blob which is a blob to which a neighboring object of the one object belongs. Generate a connected graph.

[3-4. Search processing example]
Here, an example of the search process according to the third embodiment will be described using FIGS. 27 and 28 as examples. 27 and 28 are flowcharts illustrating an example of search processing according to the third embodiment. The search processing described below is performed by the search processing unit 133B of the information processing device 100B. Note that descriptions of points similar to the search processing described in the first embodiment and the second embodiment will be omitted as appropriate.

In FIGS. 27 and 28, N(G, s) and C are a set of blobs (blob set). Further, “G” may be graph data (blob connected graph) in which blobs are connected by edges (for example, graph GR41 shown in spatial information SP41). For example, the information processing device 100B executes k-nearest neighbor search processing.

First, the process (main process) shown in FIG. 27 will be described. For example, the information processing device 100B sets the radius r of the hypersphere to ∞ (infinity) and sets the blob set C to the empty set (Φ) (step S601). Then, the information processing device 100B extracts a partial blob set B from the existing blob set (all blobs) (step S602). For example, the information processing device 100B may extract, as the partial blob set B, the blob to which the object (node) selected as the root node belongs. Further, for example, the information processing device 100B may randomly extract blobs as the partial blob set B.

The information processing device 100B then executes determination processing (step S603). The information processing device 100B executes the determination processing of steps S701 to S717 shown in FIG. 28. Details of the determination process shown in FIG. 28 will be described later.

Then, the information processing device 100B sets partial blob set B in blob set S (step S604).

The information processing device 100B selects the blob s with the minimum query distance d among the blobs included in the blob set S (step S605). Note that the "query distance" here is the shortest distance between all objects in the blob and the query object (query). For example, when the object serving as a query (search query) is y, the information processing device 100B selects the blob s having the shortest query distance to the object y from the blob set S. Note that the query distance is not limited to the above, and may be, for example, the distance between the query and a representative point (centroid, etc.) of the blob. The information processing device 100B then excludes the blob s from the blob set S (step S606).

Then, the information processing device 100B determines whether the query distance d of the blob s exceeds r(1+ε) (step S607). If the query distance d of the blob s exceeds r(1+ε) (step S607: Yes), the information processing device 100B outputs the object set R as a neighboring object set of the object y (step S608), and ends the process.

If the query distance d of the blob s does not exceed r(1+ε) (step S607: No), the information processing device 100B determines whether the blob s is included in the blob set C (step S609).

If the blob s is included in the blob set C (step S609: Yes), the information processing device 100B returns to step S605 and repeats the process.

If the blob s is not included in the blob set C (step S609: No), the information processing device 100B adds the blob s to the blob set C (step S610).

Then, the information processing device 100B sets a neighboring blob set N(G, s) of the blob s in the partial blob set B (step S611). The neighboring blob set N(G, s) is, for example, a set of blobs (nearby blobs) associated with blob s. For example, the neighboring blob set N(G, s) is a set of blobs (nearby blobs) to which edges from blob s are connected. The information processing device 100B then executes determination processing (step S612). Although details will be described later, for example, the information processing device 100B executes the determination processing of steps S701 to S717 shown in FIG. 28, similar to step S603 described above.

Then, the information processing device 100B determines whether the blob set S is an empty set (Φ) (step S613). If the blob set S is not an empty set (step S613: No), the information processing device 100B returns to step S605 and repeats the process. Furthermore, if the blob set S is an empty set (step S613: Yes), the information processing device 100B outputs the object set R and ends the process (step S614). For example, the information processing device 100B may select an object (node) included in the object set R as a neighboring node corresponding to the additional node (input object y). For example, the information processing device 100B may extract (select) objects (nodes) included in the object set R as neighboring nodes corresponding to the target node (input object y). Further, for example, the information processing device 100B may provide the objects (nodes) included in the object set R as the search results corresponding to the search query (input object y) to the terminal device or the like that performed the search.

From here, the process (determination process) shown in FIG. 28 will be explained. First, the information processing device 100B sets a partial blob set B in the blob set T (step S701).

Then, the information processing device 100B obtains one blob b from the blob set T, and deletes blob b from the blob set T (step S702). Then, the information processing device 100B sets min, which is a variable used in the determination process, to ∞ (infinity) (step S703).

Then, the information processing device 100B acquires one object u from blob b, and deletes object u from blob b (step S704).

Then, the information processing device 100B determines whether the distance d(u, y) between the object u and the object y is less than min (step S705).

If the distance d(u,y) between the object u and the object y is less than min (step S705: Yes), the information processing device 100B sets (updates) the value of min to the distance d(u,y). (Step S706), and the process of Step S707 is performed.

If the distance d(u, y) between object u and object y is not less than min (step S705: No), the information processing device 100B does not perform the process of step S706, but performs the process of step S707.

The information processing device 100B determines whether the distance d(u, y) between the object u and the object y is less than or equal to r(1+ε) (step S707).

If the distance d(u,y) between the object u and the object y is less than or equal to r(1+ε) (step S707: Yes), the information processing device 100B adds the blob b to the blob set S (step S708), The process of step S709 is performed.

If the distance d(u, y) between object u and object y is not less than or equal to r(1+ε) (step S707: No), the information processing device 100B performs the process of step S709 without performing the process of step S708. .

The information processing device 100B determines whether the distance d(u, y) between the object u and the object y is less than or equal to r (step S709).

If the distance d(u,y) between the object u and the object y is less than or equal to r (step S709: Yes), the information processing device 100B adds the object u to the object set R (step S710) and performs the steps in step S711. Perform processing.

If the distance d(u, y) between the object u and the object y is not less than or equal to r (step S709: No), the information processing device 100B performs the process of step S711 without performing the process of step S710.

The information processing device 100B determines whether the number of objects included in the object set R exceeds ks (step S711). The predetermined number ks is an arbitrarily determined natural number. For example, ks may be the number of searches or the number of extraction targets. Further, for example, when there is no upper limit on the number of objects to be extracted in a range search or the like, ks may be set to infinity. For example, ks may be 4, etc.

If the number of objects included in the object set R exceeds ks (step S711: Yes), the information processing device 100B excludes from the object set R the object that is farthest from the object y among the objects included in the object set R. (Step S712), and performs the process of Step S713.

If the number of objects included in the object set R does not exceed ks (step S711: No), the information processing device 100B performs the process of step S713 without performing the process of step S712.

The information processing device 100B determines whether the number of objects included in the object set R matches ks (step S713).

If the number of objects included in the object set R matches ks (step S713: Yes), the information processing device 100B determines the distance between the object y and the object that is farthest from the object y among the objects included in the object set R. is set to r (step S714), and the process of step S715 is performed.

If the number of objects included in the object set R does not match ks (step S713: No), the information processing device 100B performs the process of step S715 without performing the process of step S714.

The information processing device 100B determines whether blob b is an empty set (Φ) (step S715). If blob b is not an empty set (step S715: No), the information processing device 100B returns to step S704 and repeats the process.

Further, when blob b is an empty set (step S715: Yes), the information processing device 100B sets min as the blob query distance of blob b in partial blob set B (step S716). For example, the information processing device 100B sets the shortest distance among the distances between all objects of blob b and the query to min as the query distance.

The information processing device 100B determines whether the blob set T is an empty set (Φ) (step S717). If the blob set T is not an empty set (step S717: No), the information processing device 100B returns to step S702 and repeats the process.

Further, if the blob set T is an empty set (step S717: Yes), the determination process is ended.

[3-5. Modified example]
The blob connection graph is not limited to the example described above, and may be configured in various ways. For example, the blob connected graph may be a graph in which representative points of each group (blob) are connected by edges. This point will be explained below as a modification of the third embodiment. Note that descriptions of points similar to the first embodiment, second embodiment, and third embodiment will be omitted as appropriate. The information processing device 100B according to the modification of the third embodiment includes a blob information storage section 125A and a blob connection graph information storage section 126A instead of the blob information storage section 125 and the blob connection graph information storage section 126.

[3-5-1. Information processing〕
First, an overview of information processing according to a modified example will be explained using FIG. 29. FIG. 29 is a diagram illustrating an example of blob connection graph information according to a modification. The example in FIG. 29 shows a case where blobs are generated by clustering.

In the example of FIG. 29, the information processing device 100B generates a blob connection graph that connects the centroids of each blob. The information processing device 100B generates a graph GR42, which is a blob connection graph in which centroids C1 to C10 corresponding to each of the blobs BL1 to BL10 are connected by edges, as shown in the spatial information SP41. The information processing device 100B generates the graph GR42 using various conventional techniques related to graph generation.

For example, in the information processing device 100B, the centroid C1 of the blob BL1 is the centroid C2 of the blob BL2, the centroid C3 of the blob BL3, the centroid C4 of the blob BL4, the centroid C7 of the blob BL7, and the centroid C8 of the blob BL8. A graph GR42 is generated which is connected to the five centroids of by edges. For example, the information processing device 100B connects the centroid C2 of the blob BL2 to three centroids, the centroid C1 of the blob BL1, the centroid C3 of the blob BL3, and the centroid C4 of the blob BL4, by edges. Generate graph GR42.

Note that the information processing device 100B may generate the graph GR42 using any method. The information processing device 100B may generate the graph GR42 using various indexes. For example, the information processing device 100B may generate the graph GR42 using the graph GR21 in which object nodes are connected. In this case, the information processing apparatus 100B can generate the graph GR42 by the same process as the graph GR41 in FIG. 23, so detailed explanation will be omitted. For example, the information processing device 100B may generate the graph GR42 by a process similar to the example shown in FIG. 23. Although the graph GR42 shows an undirected (bidirectional) edge as an example, it may be a directed edge.

In this way, the information processing device 100B according to the modification generates a centroid graph as a blob connection graph. Then, the information processing device 100B performs a normal search using the centroid using the generated blob connection graph, and performs distance calculation on a blob-by-blob basis. In this way, the information processing apparatus 100B can simplify processing by representing blobs with centroids. Note that if the blobs are not clustered, the information processing device 100B may calculate the center point based on the objects belonging to each blob, and use the calculated center point as the representative point.

In the example of FIG. 29, the information processing device 100B performs a search process using a graph GR42 in which centroids, which are representative points of blobs, are connected by edges, as shown in spatial information SP42. For example, the information processing device 100B obtains the search results for the search query QE2 by performing search processing as shown in FIGS. 27 and 28 using the graph GR21 for the search query QE2. This point is similar to the example described above, and detailed explanation will be omitted.

[3-5-2. information〕
Next, an outline of the information related to the modified example will be explained using FIGS. 30 and 31. FIG. 30 is a diagram illustrating an example of a blob information storage unit according to a modification. FIG. 31 is a diagram illustrating an example of a blob connection graph information storage unit according to a modification.

First, a blob information storage unit 125A according to a modification shown in FIG. 30 will be described. The blob information storage unit 125A according to the modification shown in FIG. 30 includes items such as "blob ID", "node ID", "vector information", and "centroid ID". In this way, the blob information storage unit 125A according to the modified example is different from the blob information storage unit 125 of FIG. 15 in that it has a "centroid ID." Note that descriptions of the same points as those of the blob information storage unit 125 in FIG. 15 will be omitted as appropriate.

"Centroid ID" indicates identification information for identifying the centroid of each blob. "Vector information" indicates vector information of a centroid, which is a representative point of a blob.

The example shown in FIG. 30 indicates that the centroid of blob BL1 identified by blob ID "BL1" is the centroid (centroid C1) identified by centroid ID "C1". The centroid of blob BL9 identified by blob ID "BL9" indicates that it is the centroid (centroid C9) identified by centroid ID "C9".

Note that the blob information storage unit 125A is not limited to the above, and may store various information depending on the purpose.

Next, a blob connection graph information storage unit 126A according to a modification shown in FIG. 31 will be described. The blob connection graph information storage unit 126A according to the modification shown in FIG. 30 has items such as "centroid ID" and "connection centroid information." Note that descriptions of points similar to those of the blob connected graph information storage unit 126 in FIG. 25 will be omitted as appropriate.

"Centroid ID" indicates identification information for identifying a centroid that is a representative point of each blob in the graph. “Connection centroid information” indicates information regarding a centroid (reference destination centroid) that can be traced from a corresponding centroid. For example, "connection centroid information" includes information such as "reference destination." “Reference destination” indicates information for identifying a reference destination (centroid) that is connected by an edge and can be traced from the centroid. That is, in the example of FIG. 31, a reference destination (centroid) that can be traced from the centroid by an edge is registered in association with a centroid ID that identifies the centroid. Note that the "connection centroid information" may include information (edge ID) for identifying an edge connected to a reference destination.

In the example of FIG. 31, from the centroid (centroid C1) identified by the centroid ID "C1", each of the centroid IDs "C2", "C3", "C4", "C7", and "C8" The edge is connected to the five centroids identified by . In other words, it is shown that each of the five centroids C2, C3, C4, C7, and C8 can be traced from the centroid C1.

Note that the blob connection graph information storage unit 126A is not limited to the above, and may store various information depending on the purpose. The blob connection graph may also include a program module that receives a query as input, searches for objects (object nodes) by tracing edges in the blob connection graph, and extracts and outputs objects similar to the query. That is, the blob connection graph may be assumed to be used as a program module that performs search processing using the blob connection graph. For example, the graph GR42, which is a blob connection graph, may be a program that, when vector data is input as a query, extracts and outputs objects corresponding to vector data similar to the vector data using a blob connection graph. . For example, the graph GR42 may be data used as a program module that searches for similar images corresponding to a query image. For example, based on an input query, the graph GR 42 causes the computer to extract and output objects similar to the query in the blob connection graph.

As described above, in each embodiment and modification, the information processing system 1 executes the following processing. For example, the information processing system 1 generates blobs by clustering vectors. For example, the information processing system 1 generates a graph index using the centroid of a blob as a node.

For example, the information processing system 1 acquires a certain number of arbitrary vectors and performs clustering using Cartesian product quantization. That is, the information processing system 1 divides the vector into partial vectors, performs clustering for each partial vector, and generates a codebook.

For example, the information processing system 1 performs product quantization on objects belonging to blob units and generates transposed indexes. For example, the information processing system 1 generates a graph (blob graph) including edges from objects to blobs. For example, the information processing system 1 searches for nearby objects using a general product quantization method. The general method of direct product quantization mentioned here is, for example, after calculating the distances between objects and all centroids of blobs and obtaining a certain number of top blobs, the distances of objects within the blobs are subjected to direct product quantization. It may also be possible to obtain nearby objects by calculating the distance. Note that, as described above, when the information processing system 1 generates a centroid graph, it is possible to search for and obtain a certain number of blobs using the graph. Alternatively, the information processing system 1 may generate a graph in which blobs are connected by edges (blob connection graph) instead of the above-described graph including edges from objects to blobs (blob graph).

[4. Fourth embodiment]
In the above example, a case is shown in which blobs, which are an example of groups for classifying objects, are used, but the groups for classifying objects are not limited to blobs, and various graphs for classifying objects may be used. For example, in addition to blobs (the first group), clusters, which are the second group that serve as units for quantizing objects, may be used.

This point will be described below as a fourth embodiment. In the fourth embodiment, the information processing system 1 includes an information processing device 100C instead of the information processing device 100, the information processing device 100A, or the information processing device 100B. Note that descriptions of points similar to those described above in the first embodiment, second embodiment, third embodiment, etc. will be omitted as appropriate.

The information processing device 100C stores blob information (first group information) indicating a plurality of blobs that classify a plurality of objects, and cluster information (second group information) indicating a cluster, which is a group classified differently from the blobs. generate.

[4-1. Information processing〕
First, an overview of information processing according to the fourth embodiment will be explained using FIG. 32. FIG. 32 is a diagram illustrating an example of information processing according to the fourth embodiment. Note that although FIG. 32 shows a case where the information processing device 100C classifies objects into a plurality of blobs and classifies them into clusters based on the blobs, it is also possible to classify objects into blobs after classifying them into clusters. This will be discussed later.

First, the information processing device 100C generates blob information that classifies a plurality of objects into a plurality of blobs (step S71). In the example of FIG. 32, the information processing device 100C generates blob information that classifies a plurality of objects into blobs BL1 to BL10, etc., using an arbitrary method. The information processing device 100C may classify each node into one of the blobs BL1 to BL10 using arbitrary clustering such as k-means. Note that the information processing device 100C may use any method to generate blobs as long as it is capable of generating blobs. For example, the information processing device 100C may generate a blob using an index such as a graph. For example, the information processing device 100C may generate information indicating the blobs BL1 to BL10, etc. using any of the first to fourth methods described above.

Note that in FIG. 32, the information processing device 100C generates the graph GR41, which is a blob connection graph, as shown in the spatial information SP41, but it is not necessary to generate the graph GR41. For example, when generating the graph GR41, the information processing device 100C generates the graph GR41 by processing using the same information as in FIG. 23, but since it is the same as the explanation in FIG. 23, detailed explanation will be omitted.

The information processing device 100C then generates cluster information for classifying the plurality of blobs into a plurality of clusters (step S72). For example, the information processing device 100C generates cluster information indicating a cluster for classifying a plurality of blobs such as blobs BL1 to BL10 using an arbitrary method. The information processing device 100C may classify the blobs into any one of a plurality of clusters such as blobs BL1 to BL10 using arbitrary clustering such as k-means. Note that the information processing device 100C may use any method to generate blobs as long as it is capable of generating blobs. For example, the information processing device 100C may generate a blob using an index such as a graph.

In the example of FIG. 32, the information processing device 100C generates cluster information that classifies blobs BL1 to BL10 and the like into clusters CL1 to CL4. For example, the information processing device 100C classifies blob BL1 and blob BL7 into cluster CL1. That is, the information processing device 100C classifies objects belonging to blob BL1 and objects belonging to blob BL7 into cluster CL1. Although the blob boundaries are shown to be on the cluster boundaries in FIGS. 32 and 34, they may not match depending on the dividing method, and depending on the dividing method, the space may be divided by the boundary lines. Not exclusively.

Further, the information processing device 100C classifies the blob BL2 and the blob BL4 into a cluster CL2. That is, the information processing device 100C classifies objects belonging to blob BL2 and objects belonging to blob BL4 into cluster CL2.

Furthermore, the information processing device 100C classifies blob BL3, blob BL8, and blob BL10 into cluster CL3. That is, the information processing device 100C classifies objects belonging to blob BL3, objects belonging to blob BL8, and objects belonging to blob BL10 into cluster CL3.

Furthermore, the information processing device 100C classifies blob BL5, blob BL6, and blob BL9 into cluster CL4. That is, the information processing device 100C classifies objects belonging to blob BL5, objects belonging to blob BL6, and objects belonging to blob BL9 into cluster CL4.

For example, the information processing device 100C uses cluster information to perform processing related to quantization. The information processing device 100C performs processing related to quantization targeting a plurality of objects using cluster information. For example, the information processing device 100C performs quantization for each object belonging to each of the clusters CL1 to CL4, but this point will be described later.

As described above, the information processing device 100C stores blob information indicating a plurality of blobs that classify a plurality of objects that are data search targets, and cluster information indicating a plurality of clusters that are classified differently from the plurality of blobs. generate. Thereby, the information processing apparatus 100C can generate two types of groups for classifying a plurality of objects, and can generate groups for classifying objects.

[4-1-1. Other processing examples]
Note that the process shown in FIG. 32 is only an example, and the information processing apparatus 100C may generate blob information and cluster information using any type of process as long as it is capable of generating the blob information and cluster information.

For example, the information processing device 100C may generate cluster information for classifying into clusters, and then generate blob information for classifying into blobs. In this case, the information processing apparatus 100C may generate cluster information for classifying each object into one of the plurality of clusters by performing clustering in which the plurality of objects are classified into clusters. For example, the information processing device 100C may classify each object into one of clusters CL1 to CL4 using arbitrary clustering such as k-means.

The information processing device 100C may generate blob information indicating the blob by dividing at least one of the clusters CL1 to CL4 and generating two or more blobs. For example, the information processing device 100C divides each cluster and generates blobs by clustering objects belonging to each cluster using arbitrary clustering such as k-means. For example, the information processing device 100C may generate blob BL1 and blob BL7 by dividing cluster CL1 into two blobs. For example, the information processing device 100C clusters objects belonging to the cluster CL1 using k-means with the number of clusters being 2, and divides the cluster CL1 to generate blob BL1 and blob BL7.

For example, the information processing device 100C may generate blob BL2 and blob BL4 by dividing cluster CL2 into two blobs. For example, the information processing device 100C may generate blob BL3, blob BL8, and blob BL10 by dividing cluster CL3 into three blobs. For example, the information processing device 100C may generate blob BL5, blob BL6, and blob BL9 by dividing cluster CL4 into three blobs.

[4-2. Configuration of information processing device]
Next, the configuration of an information processing apparatus 100C according to the fourth embodiment will be described using FIG. 33. FIG. 33 is a diagram illustrating a configuration example of an information processing apparatus according to the fourth embodiment. As shown in FIG. 33, the information processing device 100C includes a communication section 110, a storage section 120C, and a control section 130B. Note that in the information processing device 100C, descriptions of the same points as the information processing device 100, the information processing device 100A, or the information processing device 100B will be omitted as appropriate.

(Storage unit 120C)
The storage unit 120C is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 33, the storage unit 120C according to the fourth embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123, a codebook information storage unit 124, and a blob information storage unit 120. It has an information storage section 125, a blob connection graph information storage section 126, and a cluster information storage section 127.

The cluster information storage unit 127 according to the fourth embodiment stores information regarding clusters that are the second group. The cluster information storage unit 127 stores information regarding clusters, which are units of quantization. The cluster information storage unit 127 may store identification information for identifying each cluster (cluster ID, etc.) in association with information (blob ID, etc.) indicating a blob that is the first group belonging to the cluster. Further, the cluster information storage unit 127 may store identification information (such as a cluster ID) for identifying each cluster in association with information (such as an object ID) indicating an object belonging to the cluster. The cluster information storage unit 127 also associates identification information (cluster ID, etc.) for identifying each cluster with information (base point ID, etc.) for specifying a base point such as a centroid of that cluster, its vector, etc. You may memorize it.

(Control unit 130C)
The control unit 130C is a controller, and allows various programs (corresponding to an example of an information processing program) stored in a storage device inside the information processing device 100C to work on the RAM by, for example, a CPU, MPU, GPU, etc. This is achieved by executing it as a region. Further, the control unit 130C is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

As shown in FIG. 33, the control unit 130C includes an acquisition unit 131, a generation unit 132C, a search processing unit 133C, and a provision unit 134, and realizes or executes information processing functions and operations described below. do. Note that the internal configuration of the control unit 130C is not limited to the configuration shown in FIG. 33, and may be any other configuration as long as it performs information processing to be described later.

The acquisition unit 131 according to the fourth embodiment acquires object information indicating a plurality of objects that are data search targets. The acquisition unit 131 acquires index information indicating an index that searches for a plurality of objects.

The acquisition unit 131 includes a plurality of blobs that classify a plurality of objects that are data search targets, and a group that is classified differently from the plurality of blobs, and among the plurality of objects, the plurality of blobs belong to the same blob. Processing information indicating a plurality of clusters that classify objects into the same group is obtained. The acquisition unit 131 acquires processing information indicating a first number of blobs for classifying a plurality of objects and clusters for classifying a plurality of objects into a second number smaller than the first number. The acquisition unit 131 acquires processing information indicating a plurality of clusters in which object groups belonging to each of two or more blobs among the plurality of blobs are classified into one cluster.

The acquisition unit 131 acquires processing information indicating a plurality of blobs used to execute a first process which is a batch process in a search process targeting a plurality of objects. The acquisition unit 131 acquires processing information indicating a plurality of clusters used to perform second processing, which is processing related to quantization targeting a plurality of objects.

The acquisition unit 131 acquires processing information that associates a plurality of first identification information that identifies each of a plurality of blobs with a plurality of second identification information that identifies each of a plurality of clusters. The acquisition unit 131 acquires processing information in which one piece of second identification information is associated with two or more pieces of first identification information.

The acquisition unit 131 acquires processing information associated with first identification information of a blob to which each of the plurality of objects belongs. The acquisition unit 131 acquires processing information in which quantized object information obtained by quantizing each of a plurality of objects is associated with first identification information. The acquisition unit 131 associates the quantized object information generated by vector quantization, which quantizes each partial vector obtained by dividing the vector corresponding to each of the plurality of objects, with the first identification information by direct product quantization. Get the information for processing. The acquisition unit 131 allows the quantized object information generated based on each partial centroid corresponding to each partial region of each centroid corresponding to quantization to obtain processing information associated with the first identification information. get. The acquisition unit 131 generates a quantized object based on a residual vector generated by each centroid corresponding to quantization and each partial centroid corresponding in each partial region of each centroid. The information acquires processing information associated with the first identification information.

(Generation unit 132C)
The generation unit 132C generates various information in the same way as the generation unit 132, the generation unit 132A, or the generation unit 132B.

The generation unit 132C generates blob information indicating a plurality of blobs that classify a plurality of objects based on the object information acquired by the acquisition unit 131. The generating unit 132C generates cluster information indicating a plurality of clusters, which are groups that are classified differently from a plurality of blobs, and which are groups that classify objects belonging to the same blob among a plurality of blobs into the same group. generate.

The generating unit 132C generates blob information that classifies a plurality of objects into a first number of blobs, and cluster information that classifies a plurality of objects into a second number of clusters that are smaller than the first number. The generating unit 132C generates cluster information indicating a plurality of clusters in which a group of objects belonging to each of two or more blobs among the plurality of blobs is classified into one cluster.

The generation unit 132C generates blob information indicating a plurality of blobs used for performing batch processing in a search process targeting a plurality of objects. The generation unit 132C generates blob information indicating a plurality of blobs used to collectively calculate distances regarding a plurality of objects. The generation unit 132C generates blob information indicating a plurality of blobs used to calculate the distance between the search query and the object used in the search process in parallel.

The generation unit 132C generates cluster information indicating a plurality of clusters used to perform processing related to quantization targeting a plurality of objects. The generating unit 132C generates cluster information indicating a plurality of clusters that are units of quantization. The generating unit 132C generates cluster information indicating a plurality of clusters used to quantize vectors corresponding to each of a plurality of objects.

After classifying the plurality of blobs, the generation unit 132C performs classification of the plurality of clusters. The generation unit 132C generates blob information indicating a plurality of blobs by clustering a plurality of objects. The generation unit 132C uses the index information to generate blob information indicating a plurality of blobs. The generation unit 132C generates cluster information indicating a plurality of clusters using blob information indicating blobs. The generation unit 132C generates cluster information indicating a plurality of clusters by clustering a plurality of blobs.

After classifying the plurality of clusters, the generation unit 132C performs classification of the plurality of blobs. The generating unit 132C generates cluster information indicating a plurality of clusters by clustering a plurality of objects. The generating unit 132C generates blob information indicating a plurality of blobs using cluster information indicating a cluster. The generating unit 132C generates blob information indicating a plurality of blobs by dividing at least one cluster among the plurality of clusters and generating two or more blobs. The generation unit 132C generates first group information indicating a plurality of first groups by clustering objects belonging to each second group and dividing each second group into two or more first groups.

(Search processing unit 133C)
The search processing unit 133C functions as a processing unit that executes a first process that targets objects belonging to a blob and a second process that targets objects that belong to a cluster. The search processing unit 133C performs various processes related to search processing in the same way as the search processing unit 133, the search processing unit 133A, or the search processing unit 133B. The search processing unit 133C performs processing using the information generated by the generation unit 132C.

The search processing unit 133C uses the processing information acquired by the acquisition unit 131 to execute a first process targeting objects belonging to a blob and a second process targeting objects belonging to a cluster. The search processing unit 133C executes the second process on a group of objects belonging to one cluster. The search processing unit 133C performs a first process on a group of objects belonging to one blob.

The search processing unit 133C executes a first process of collectively calculating distances for a group of objects belonging to one blob. The search processing unit 133C executes a first process that is a parallel process of calculating distances between each object group belonging to one blob and a search query used in the search process.

The search processing unit 133C executes the second process on a group of objects belonging to one cluster. The search processing unit 133C executes second processing, which is quantization, on a group of objects belonging to one cluster. The search processing unit 133C executes a second process of quantizing the vectors corresponding to each of the object groups belonging to one cluster.

When two or more blobs belong to one cluster, the search processing unit 133C executes the second process on a group of objects that belong to the two or more blobs. When two or more blobs belong to one cluster, the search processing unit 133C performs quantization on all objects belonging to the two or more blobs.

The search processing unit 133C refers to the processing information and executes the first process and the second process. The search processing unit 133C refers to the processing information to identify the object to be processed. In a search process targeting a plurality of objects, the search processing unit 133C performs a first process on one blob of first identification information among two or more pieces of first identification information, and also performs a first process on one blob of first identification information. After generating the reference information to be referred to in the search process regarding the information cluster, the first identification information is generated for other blobs of the first identification information other than one of the two or more first identification information. No reference information is generated during processing.

The search processing unit 133C performs search processing using a blob connection graph. The search processing unit 133C performs a search process to search for nearby objects of the search query using the blob connection graph. The search processing unit 133C performs a search process to search for nearby objects of the search query by tracing edges connecting blobs in the blob connection graph.

In the search process, the search processing unit 133C uses blob information indicating a plurality of blobs to calculate the distance between the target object, which is an object belonging to one blob, and the search query, and the vector information of the target object that has been vector quantized. Calculate using The search processing unit 133C parallelizes and collectively calculates the distance between the target object and the search query. The search processing unit 133C performs parallel processing to calculate the distance between the search query and the target object of the batch processing number determined based on the specifications of the information processing device.

The search processing unit 133C uses the blob connection graph to perform search processing as shown in FIGS. 27 and 28. For example, the search processing unit 133C performs a search process using a graph GR41 that connects blobs with edges, as shown in the spatial information SP51. For example, the search processing unit 133C obtains the search results for the search query QE3 by performing search processing as shown in FIGS. 27 and 28 using the graph GR41, which is a blob connection graph, for the search query QE3. For example, the search processing unit 133C executes the first process on the blob that is the processing target by tracing the blob connection graph.

The search processing unit 133C generates reference information if reference information has not yet been generated for the cluster to which the processing target blob belongs. For example, the search processing unit 133C generates a distance lookup table (LUT) as reference information. For example, the search processing unit 133C generates a distance lookup table for each cluster in the search process. That is, in the search processing, the search processing unit 133C generates a lookup table of distances not in units of blobs but in units of clusters. Thereby, the information processing device 100C can suppress an increase in search time.

The search processing unit 133C uses the generated reference information to collectively calculate distances for objects belonging to the blob that is the processing target. For example, the search processing unit 133C does not generate reference information if reference information has already been generated for the cluster to which the blob targeted for processing belongs. Then, the search processing unit 133C uses the reference information that has already been generated to perform distance calculations all at once for objects belonging to the blob that is the processing target.

[4-3. Examples of processing and information]
From here, examples of processing, information used for processing, etc. in the fourth embodiment will be described.

First, the concept regarding quantization will be explained using FIG. 34. FIG. 34 is a conceptual diagram regarding quantization according to the fourth embodiment.

For example, base point CL1 indicates a base point used for quantization of cluster CL1. In this case, base point CL1 indicates the centroid of cluster CL1 including blob BL1 and blob BL7. Further, for example, base point CL2 indicates a base point used for quantization of cluster CL2. In this case, base point CL2 indicates the centroid of cluster CL2 including blob BL2 and blob BL4.

For example, base point CL3 indicates a base point used for quantization of cluster CL3. In this case, base point CL3 indicates the centroid of cluster CL3 including blob BL3, blob BL8, and blob BL10. Further, for example, base point CL4 indicates a base point used for quantization of cluster CL4. In this case, base point CL4 indicates the centroid of cluster CL4 including blob BL5, blob BL6, and blob BL9.

In this case, the information processing device 100C finds residual vectors in each of the four clusters CL1 to CL4. The information processing device 100C then performs quantization using the residual vectors found in each of the four clusters CL1 to CL4. For example, the information processing device 100C performs quantization on objects belonging to the cluster CL1 using the residual vector from the base point CL1. Although a detailed explanation of quantization using residual vectors will be omitted, the information processing device 100C performs quantization using residual vectors, for example, appropriately using the technique disclosed in Patent Document 3.

Note that although FIG. 34 shows a state in which the vector is not divided into subvectors for the sake of illustration, the information processing device 100C derives a base point for each subspace divided by direct product quantization, Quantization may be performed using a residual vector based on the derived base point. For example, when performing Cartesian product quantization by dividing a vector into four partial vectors, the information processing device 100C has a base point for each subspace corresponding to each partial vector, and uses the residual vector from the base point to Perform quantization.

Hereinafter, an example of object quantization and quantization processing by the information processing apparatus 100C will be described with reference to FIG. 34. In FIG. 34, a case where the query object (search query) is search query QE3 will be described as an example. Note that in the quantization of objects (vectors), direct product quantization, etc. described below, detailed explanations of points similar to those disclosed in Patent Document 3 will be omitted as appropriate.

As shown in the spatial information SP52 of FIG. 34, although the distances between the search query QE3 and each vector corresponding to each object belonging to the cluster CL1 such as the node N7 are different, the information processing apparatus 100C do not use vectors corresponding to each object belonging to . In this case, the distance between the search query QE3 and each vector is considered to be, for example, the distance between the search query QE3 and the vector corresponding to the base point CL1 of the cluster CL1. That is, the vectors corresponding to each object belonging to cluster CL1 are quantized into vectors corresponding to base point CL1. Specifically, the information processing device 100C obtains a residual vector that is the difference between each vector from the base point CL1, and performs direct product quantization (further divides into subspaces and quantizes) the residual vector.

For example, the information processing device 100C further divides each cluster into a plurality of partial regions, and generates partial centroids corresponding to the plurality of partial regions as a codebook. For example, the information processing device 100C may generate partial centroids corresponding to a plurality of partial regions into which each cluster is divided, based on residual vector information regarding each cluster.

For example, the information processing device 100C may determine the number of partial regions of each cluster based on the number of objects included in each cluster, or may determine the number of partial regions based on a predetermined setting value. Good too. For example, the information processing device 100C may determine the partial area using various conventional techniques related to clustering as appropriate. For example, the information processing device 100C calculates a residual vector from the object included in each partial region of the cluster CL1 and the base point CL1 of the cluster CL1, and generates a partial centroid of the partial region from the residual vector. For example, the information processing device 100C uses information on the residual vector between the base point of the cluster and the partial centroid of each partial region. For example, the information processing device 100C uses information on the residual vector between the base point CL1 of the cluster CL1 and the partial centroid of each partial region thereof. For example, the information processing device 100C stores, in association with each object, information on the partial centroid of the partial area to which the object belongs.

For example, by using the above-mentioned information, the information processing device 100C can quantize the position of each object to the position of the centroid of the partial area to which the vector belongs. Thereby, the information processing device 100C can refine the distance between the search query QE3 and each object to the distance between the search query QE3 and the vector corresponding to the partial centroid of the partial area to which each object belongs.

For example, the information processing device 100C uses information indicating the base point vector of each cluster and the residual vector regarding the partial centroid. Thereby, the information processing device 100C can calculate the distance to the partial centroid for, for example, the search query QE3. Therefore, the information processing device 100C can, for example, set the distance between the search query QE3 and each object as the distance to a partial centroid that is even closer than the base point of the cluster.

Note that the information processing device 100C may divide the vector of each object into a plurality of partial vectors for processing. That is, the information processing device 100C may perform processing using a technique related to so-called direct product quantization. Since this point is similar to the first embodiment and the like described above, overlapping explanation will be omitted as appropriate.

For example, in the case of FIG. 34, the information processing device 100C divides the vector of search query QE3 into a plurality of partial vectors (also referred to as "partial vectors of search query QE3"). In this case, the spatial information SP52 is also divided into a plurality of subspaces (also referred to as "subspaces of the spatial information SP52"). For example, the information processing device 100C may divide the vector of the search query QE3 into four as in FIG. In this case, the spatial information SP52 is also divided into four subspaces, and each cluster is divided into a plurality of subspaces.

For example, the information processing device 100C stores, in association with each object, information indicating a partial centroid (codebook) corresponding to a region to which each object belongs in the four partial spaces. For example, the information processing device 100C calculates the distance between each object and the search query QE3 by summing the distances between the four codebooks corresponding to each object and the search query QE3.

Next, an overview of the transposed index will be explained using FIGS. 35 and 36. FIG. 35 is a diagram illustrating an example of a transposed index according to the fourth embodiment. FIG. 36 is a diagram showing a specific example of a transposed index according to the fourth embodiment.

First, the transposed index IND1 shown in FIG. 35 will be explained. The transposed index IND1 shown in FIG. 35 includes "blob ID", "cluster ID", "quantization object number", "quantization object #1", "quantization object #2", "quantization object #3", It includes items such as "quantization object #4" and the like. In addition, in FIG. 35, four objects from "quantization object #1" to "quantization object #4" are illustrated, but "quantization object #5" and "quantization object #6" are shown in the transposed index IND1. ”, etc., the number of items corresponding to the number of objects belonging to each blob is included.

“Blob ID” indicates identification information for identifying blobs that are the first group. "Cluster ID" indicates identification information for identifying the cluster that is the second group. "Number of quantized objects" indicates the number of objects belonging to the blob. “Quantized object #1” to “quantized object #4” indicate quantized objects. For example, "quantization object #1" to "quantization object #4" store a list of information indicating the codebook corresponding to each partial vector after the object has been subjected to product quantization.

In the example of FIG. 35, the blob with the blob ID "1" belongs to the cluster with the cluster ID "2", indicating that the number of belonging objects is four. Furthermore, for the blob with blob ID "1", "4, 2, 8, 10" are stored in "quantization object #1", and the object is composed of four partial vectors, each of which has a codebook "4". ”, “2”, “8”, and “10”. For example, for the object corresponding to "quantization object #1" with blob ID "1", among the four subvectors divided, the first subvector is quantized with codebook "4", and the second subvector is quantized with codebook "4". It shows that the subvectors are quantized with codebook "2", the third subvector is quantized with codebook "8", and the last subvector is quantized with codebook "10".

Note that the transposed index IND1 is not limited to the above, and may store various information depending on the purpose. For example, the transposed index IND1 may store information indicating each quantization object (object ID, etc.) in association with each quantization object.

Next, the transposed index IND2 shown in FIG. 36 will be explained. FIG. 36 shows a specific example of a transposed index corresponding to the example shown in FIG. Note that descriptions of points similar to the transposed index IND1 will be omitted as appropriate. The transposed index IND2 shown in FIG. 36 includes items such as "blob ID", "cluster ID", "quantization object number", and "quantization object #1". Note that although only "quantized object #1" is illustrated in FIG. 36, it includes a number of items corresponding to the number of objects belonging to each blob, similar to the transposed index IND1.

The transposed index IND2 indicates that the blob BL1 identified by the blob ID "BL1" belongs to the cluster CL1 having the cluster ID "CL1", similar to the spatial information SP51 shown in FIG. 32. Further, the number of objects belonging to blob BL1 is “NM1”. Although it is indicated by an abstract code such as "NM1" in FIG. 36, the "quantization object number" stores a value indicating a specific number (for example, 10, 100, etc.) similarly to the transposed index IND1. .

In addition, in the blob BL1, "quantization object #1" stores "CD51, CD64, CD77, CD82", and the object is composed of four partial vectors, codebooks "CD51", "CD64", Indicates that the data is quantized as "CD77" or "CD82".

Further, the transposed index IND2 indicates that the blob BL2 identified by the blob ID "BL2" belongs to the cluster CL2 having the cluster ID "CL2", similar to the spatial information SP51 shown in FIG. 32. Further, the transposed index IND2 indicates that the blob BL7 identified by the blob ID "BL7" belongs to the cluster CL1 having the cluster ID "CL1", similarly to the spatial information SP51 shown in FIG. 32.

Note that the transposed index IND2 is not limited to the above, and may store various information depending on the purpose. For example, the transposed index IND2 may store information indicating each quantization object (object ID, etc.) in association with each quantization object. In this way, information for identifying clusters (cluster ID) is attached to the transposed index, and the information processing device 100C calculates the residual vector using the transposed index. For example, the residual vector is used during index generation and search, but since the process is similar to that in conventional techniques such as Patent Document 3, detailed explanation will be omitted as appropriate.

[4-4. Information processing flow]
Next, the information processing procedure according to the fourth embodiment will be described using FIGS. 37 and 38. 37 and 38 are flowcharts illustrating an example of information processing according to the fourth embodiment.

First, FIG. 37 will be explained. For example, FIG. 37 is a flowchart illustrating an example of group generation processing. As shown in FIG. 37, the information processing apparatus 100C acquires object information indicating a plurality of objects that are data search targets (step S601). For example, the information processing device 100C acquires object information indicating a plurality of objects from the object information storage unit 121.

The information processing device 100C generates first group information indicating a plurality of first groups into which a plurality of objects are classified based on the object information (step S602). For example, the information processing device 100C generates blob information indicating a plurality of blobs that classify a plurality of objects based on the object information.

The information processing device 100C is a group that has a different classification from the plurality of first groups, and is a group in which objects belonging to the same first group in the plurality of first groups among the plurality of objects are classified into the same group. Second group information indicating a plurality of second groups is generated (step S603). For example, the information processing device 100C generates cluster information indicating a plurality of clusters in which at least one cluster includes two or more blobs. Note that the information processing apparatus 100C may generate cluster information indicating clusters that correspond one-to-one to blobs.

Next, FIG. 38 will be explained. For example, FIG. 38 is a flowchart illustrating an example of processing using groups. As shown in FIG. 38, the information processing device 100C has a plurality of first groups that classify a plurality of objects that are data search targets, and a plurality of groups that are classified differently from the plurality of first groups. Among the objects, processing information indicating a plurality of second groups, which are groups in which objects belonging to the same first group are classified into the same group, is acquired (step S701). For example, the information processing device 100C obtains blob information stored in the blob information storage unit 125 and cluster information stored in the cluster information storage unit 127 as processing information. For example, the information processing device 100C obtains the transposed index IND1 or the transposed index IND2.

The information processing apparatus 100C uses the processing information to execute a first process targeting objects belonging to the first group and a second process targeting objects belonging to the second group (step S702). For example, the information processing device 100C performs batch processing on objects belonging to a blob, and performs quantization on objects belonging to a cluster.

[5. Fifth embodiment]
Note that when performing a search process using a blob connection graph (also referred to as a "blob graph") in which blobs are connected as described above, a search range from a plurality of viewpoints may be used. This point will be described below as a fifth embodiment. In the fifth embodiment, the information processing system 1 includes an information processing device 100D instead of the information processing device 100, the information processing device 100A, the information processing device 100B, or the information processing device 100C. Note that descriptions of points similar to those described above in the first embodiment, second embodiment, third embodiment, fourth embodiment, etc. will be omitted as appropriate. For example, the blob BL* (* is an arbitrary number) in the following description is a group of objects in the same way as the blob BL* (* is an arbitrary number) described above, so detailed explanation will be omitted. Further, since the centroid C* (* is an arbitrary number) in the following explanation is the same as the centroid C* (* is an arbitrary number) described above, detailed explanation will be omitted. Cluster CL* (* is an arbitrary number) in the following explanation is a cluster for classifying blobs in the same way as cluster CL* (* is an arbitrary number) described above, so detailed explanation will be omitted.

In addition, in FIGS. 39 and 40, the expressions are the same as in FIG. 32 except for the points where the clusters are distinguished by solid thick lines and the connections between blobs are shown by dotted lines connecting centroids C of blobs BL. It is. For example, among the solid lines, thick lines indicate correspondence between clusters and blobs, and thin lines indicate correspondence between divisions of blobs. For example, in FIGS. 39 and 40, thin solid lines indicate blob division correspondence within a cluster. Furthermore, in FIGS. 39 and 40, centroids CN11 to CN14 indicate the centroids of each of the clusters CL11 to CL14. The cluster CL11 includes three blobs BL, BL11, BL12, and BL13. Blob BL11 includes nodes N31, N205, and the like. The cluster CL12 includes two blobs BL, blobs BL14 and BL15. Cluster CL13 includes three blobs BL: blobs BL16, BL17, and BL18. The cluster CL14 includes two blobs BL, blobs BL19 and BL20. Note that the clusters CL, blobs BL, nodes N, etc. shown in FIGS. 39 and 40 are only a part, and the clusters CL, blobs BL, and nodes N to be processed include many clusters CL, A blob BL and a node N are included. Further, although the symbols of the blob BL and the cluster CL are illustrated only in FIG. 39 and omitted from illustration in FIG. 40, FIG. 39 and FIG. 40 indicate the same spatial information (spatial information SP61).

The information processing device 100D executes a search process using a blob graph, which is a group-connected graph in which a plurality of groups into which a plurality of objects to be data searched are classified are connected by edges. The information processing device 100D also has a range corresponding to a search for a blob as a group (hereinafter also referred to as a "first search range") and a range corresponding to a search for a plurality of objects (hereinafter also referred to as a "second search range"). ) to execute the search process. Thereby, the information processing device 100D extracts a nearby object corresponding to the search query from the plurality of objects.

[5-1. Information processing〕
First, an overview of information processing according to the fifth embodiment will be explained using FIGS. 39 and 40. 39 and 40 are diagrams illustrating an example of information processing according to the fifth embodiment. Specifically, FIG. 39 shows an overall outline of the search process executed by the information processing device 100D. Further, FIG. 40 shows changes in the first search range and the second search range in the search process executed by the information processing device 100D. In this way, FIGS. 39 and 40 will explain the changes in the first search range and the second search range and the outline of the search process, and the specific flow of the search process will be shown in FIG. 43.

Graph GR61 shown in spatial information SP61 in FIG. 39 is a blob graph in which blobs are connected, similar to graph GR41 shown in spatial information SP51 in FIG. For example, a dotted line (edge) connecting centroids C of blobs BL in graph GR61 is synonymous with an arrow line (edge) connecting blobs BL with each other in graph GR41. Graph GR61 in FIG. 39 illustrates the blobs BL11 to BL20. For example, the information processing device 100D may generate information indicating the blobs BL11 to BL10, etc. using the method described above, or may acquire information indicating the blobs BL11 to BL10, etc. from another device. Note that the spatial information SP61-1 to SP61-4 in FIG. 40 are given different symbols for convenience of explanation of changes in the first search range and the second search range, but they are the same as the spatial information SP61 in FIG. 39 and cannot be distinguished. If the information is not explained, it may be described as spatial information SP61.

Note that the graph GR61 may be in a state where each of the blobs BL11 to BL20 of the graph GR41 is illustrated. In this case, spatial information SP51 and spatial information SP61 target the same plurality of objects, and graph GR61 and graph GR41 illustrate different parts of the same graph. For example, graph GR41 in FIG. 32 may illustrate a portion of blobs BL1 to BL10, and graph GR61 may illustrate a portion of blobs BL11 to BL20. Further, the graph GR61 may be a graph in which the centroids of each blob are connected like the graph GR42 in FIG. 29. That is, the graph GR61 may be a graph in which blobs are connected in any manner as long as the search process described below can be executed.

FIG. 39 shows, as an example, a process in which the query is search query QE11 and the number of objects extracted as nearby objects (also referred to as "number of extracted nearby objects") is "5". Among the five circles centered on the search query QE11, the circle with a dash-dot line indicates the blob search range SB, which is an example of the first search range. In addition, in FIG. 39, only the circle with the largest radius among the three dot-dashed circles indicating the blob search range SB is given the symbol "SB", but the three dot-dashed circles indicate blob searches that change during the search process. Range SB is shown.

For example, the blob search range SB is a first search range defined by a radius (blob search radius) centered on the search query QE11. As shown by the dashed-dotted arrow line in FIG. 39, the range (radius) of the blob search range SB becomes wider as the search process progresses, but this point will be described later using FIG. 40. Note that blob search ranges SB1, SB2, and SB3 shown in FIG. 40 correspond to each of the three blob search ranges SB in FIG. 39. When describing the blob search ranges SB1, SB2, and SB3 without distinguishing them, they may be referred to as blob search range SB.

Furthermore, in FIG. 39, among the five circles centered on the search query QE11, the circle indicated by the two-dot chain line indicates the object search range SO, which is an example of the first search range. In addition, in FIG. 39, only the outer circle of the two dashed-dotted circles indicating the object search range SO is marked with the symbol "SO"; Indicates range SO.

For example, the object search range SO is a second search range defined by a radius (object search radius) centered on the search query QE11. As shown by the two-dot chain arrow line in FIG. 39, the range (radius) of the object search range SO becomes smaller as the search process progresses, but this point will be described later using FIG. 40. Note that object search ranges SO1 and SO2 shown in FIG. 40 correspond to each of the two object search ranges SO in FIG. 39. When describing the object search ranges SO1 and SO2 without distinguishing them, they may be referred to as object search range SO.

First, the information processing device 100D starts the search process from the state shown in the spatial information SP61-1. Then, the information processing device 100D determines the blob search range SB to be the blob search range SB1, and the object search range SO to infinity "∞", as shown in the spatial information SP61-2. That is, in FIG. 40, the information processing apparatus 100D determines the blob search range SB to be the blob search range SB1 defined by the radius based on the distance between the nearest blob BL20 and the search query QE11. In addition, in FIG. 40, the information processing device 100D extracts three nodes (objects corresponding to) nodes N2, N6, and N8 to which the blob BL20 belongs as candidates for nearby objects corresponding to the search query QE11, Add to nearby object candidates. Note that since there are three nearby object candidates and the number of extracted nearby objects is "5" or less, the object search range SO is maintained at infinity "∞".

Then, the information processing device 100D determines the blob search range SB to be the blob search range SB2 and the object search range SO to the object search range SO1, as shown in the spatial information SP61-3. That is, in FIG. 40, the information processing device 100D determines the blob search range SB to be the blob search range SB2 defined by the radius based on the distance between the next nearest blob BL11 of the blob BL20 and the search query QE11. . In addition, in FIG. 40, the information processing device 100D extracts three nodes (objects corresponding to) nodes N31, N205, and N312 to which the blob BL11 belongs as candidates for nearby objects corresponding to the search query QE11, Add to nearby object candidates. As a result, the neighboring object candidates include six nodes N2, N6, N8, N31, N205, and N312, which exceeds the number of extracted neighboring objects "5". Therefore, the information processing device 100D leaves five of the nearby object candidates that are closest to the search query QE11, and excludes the rest. In this case, the information processing device 100D excludes the node N312 which is the farthest from the search query QE11 among the six nodes, and sets the number of nearby object candidates to five. That is, the information processing device 100D sets five nearby object candidates: nodes N2, N6, N8, N31, and N205. Since the number of neighboring object candidates has reached the number of neighboring object extractions "5", the information processing device 100D sets the object search range SO to the area between the node N205, which is the farthest from the search query QE11 among the neighboring object candidates, and the search query QE11. The object search range SO1 defined by the radius based on the distance is determined.

Then, the information processing device 100D determines the blob search range SB to be the blob search range SB3 and the object search range SO to the object search range SO2, as shown in the spatial information SP61-4. That is, in FIG. 40, the information processing device 100D determines the blob search range SB to be the blob search range SB3 defined by the radius based on the distance between the next nearest blob BL19 of the blob BL11 and the search query QE11. . In addition, in FIG. 40, the information processing device 100D extracts three nodes (objects corresponding to) nodes N3, N11, and N114 to which the blob BL19 belongs as candidates for nearby objects corresponding to the search query QE11, Add to nearby object candidates. As a result, the neighboring object candidates include eight nodes N2, N6, N8, N31, N205, N3, N11, and N114, which exceeds the number of extracted neighboring objects "5". Therefore, the information processing device 100D leaves five of the nearby object candidates that are closest to the search query QE11, and excludes the rest. In this case, the information processing device 100D excludes the three nodes N205, N3, and N114 from the farthest distance from the search query QE11, and sets the number of nearby object candidates to five. That is, the information processing device 100D sets five nearby object candidates: nodes N2, N6, N8, N11, and N31. Then, the information processing device 100D determines the object search range SO to be an object search range SO2 defined by a radius based on the distance between the search query QE11 and the node N11 which is farthest from the search query QE11 among the neighboring object candidates. .

The information processing device 100D repeats the above-described processing until termination criteria #1 to #3, which will be described later, are met. For example, if termination criteria #1 to #3 are met, the information processing device 100D determines the objects included in the nearby object candidates at that time to be the objects near the search query QE11. Then, the information processing device 100D provides information indicating the object determined as an object in the vicinity of the search query QE11 to the requestor of the search that specified the search query QE11.

As described above, the information processing device 100D performs a search process using search ranges from multiple viewpoints, such as the first search range and the second search range, and changes each search range according to the situation of the search process. This makes it possible to perform flexible search processing using graphs.

Here, for example, in a blob graph, search speed may decrease due to unnecessary blob references in a large-scale blob graph. In addition, conventional product quantization (PQ) involves searching a graph, identifying nearby blobs, and then calculating approximate distances for the identified blobs, but in this case too, the search is performed using unnecessary blob references. Speed decreases.

On the other hand, the information processing device 100D searches the blob graph, calculates approximate distances of objects within the blobs for each blob whose rank has been determined in order of proximity, and terminates the search when a predetermined termination criterion is met. This can reduce unnecessary object searches.

[5-2. Configuration of information processing device]
Next, the configuration of the information processing device 100D according to the fifth embodiment will be described using FIG. 41. FIG. 41 is a diagram illustrating a configuration example of an information processing apparatus according to the fifth embodiment. As shown in FIG. 41, the information processing device 100D includes a communication section 110, a storage section 120C, and a control section 130B. Note that in the information processing device 100D, descriptions of the same points as the information processing device 100, the information processing device 100A, the information processing device 100B, or the information processing device 100C will be omitted as appropriate.

(Storage unit 120C)
The storage unit 120C is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 41, the storage unit 120C according to the fifth embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123, a codebook information storage unit 124, and a blob information storage unit 121. It has an information storage section 125, a blob connection graph information storage section 126, and a cluster information storage section 127.

(Control unit 130D)
The control unit 130D is a controller, and allows various programs (corresponding to an example of an information processing program) stored in a storage device inside the information processing device 100D to work on the RAM by, for example, a CPU, MPU, GPU, etc. This is achieved by executing it as a region. Further, the control unit 130D is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

As shown in FIG. 41, the control unit 130D includes an acquisition unit 131, a generation unit 132C, a search processing unit 133D, and a provision unit 134, and realizes or executes information processing functions and operations described below. do. Note that the internal configuration of the control unit 130D is not limited to the configuration shown in FIG. 41, and may be any other configuration as long as it performs information processing to be described later.

The acquisition unit 131 according to the fifth embodiment acquires various information from the storage unit 120C. For example, the acquisition unit 131 according to the fifth embodiment acquires various information used by the generation unit 132C, similarly to the acquisition unit 131 according to the fourth embodiment.

The acquisition unit 131 acquires a group connection graph in which a plurality of groups into which a plurality of objects to be data searched are classified are connected by edges. The acquisition unit 131 acquires search queries for multiple objects.

(Search processing unit 133D)
The search processing unit 133D executes the search process shown in FIG. 43. For example, the search processing unit 133D performs various processes related to search processing in the same way as the search processing unit 133, the search processing unit 133A, the search processing unit 133B, or the search processing unit 133BC. The search processing unit 133D performs processing using the information stored in the storage unit 120C. The search processing unit 133D performs processing using the information generated by the generation unit 132C.

The search processing unit 133D searches the group connection graph using a first search range indicating a range corresponding to a search of a plurality of groups and a second search range indicating a range corresponding to a search of a plurality of objects. A search process is executed to extract nearby objects corresponding to the search query from a plurality of objects. The search processing unit 133D executes a search process using a first search range and a second search range that is wider than the first search range.

The search processing unit 133D executes the search process using a first search range that expands as the search process progresses. The search processing unit 133D executes the search process using a first search range determined based on one group targeted for the search process among the plurality of groups. The search processing unit 133D executes the search process using the first search range determined based on the one group being searched. The search processing unit 133D performs search processing using a first search range based on the distance between the search query and the object closest to the search query among the objects included in one group, or the centroid of one group. Execute.

The search processing unit 133D executes the search process using a second search range that becomes narrower as the search process progresses. The search processing unit 133D executes the search process using a second search range determined based on one object targeted for the search process among the plurality of objects. During the search process, the search processing unit 133D determines a second search range based on one object that is the farthest object from the search query among the objects extracted as candidates for nearby objects corresponding to the search query. Execute the search process using . The search processing unit 133D executes a search process using a second search range based on the distance between the search query and one object.

The search processing unit 133D ends the search process when a predetermined end condition is met. The search processing unit 133D ends the search process when the number of groups targeted for the search process satisfies the condition. The search processing unit 133D ends the search process when the number of groups targeted for the search process exceeds a predetermined threshold.

The search processing unit 133D ends the search process when the distance related to the predetermined group exceeds the second search range. The search processing unit 133D ends the search process when the distance related to the predetermined group being searched exceeds the second search range. The search processing unit 133D ends the search process when the distance between the object closest to the search query among the objects included in the predetermined group and the search query exceeds the second search range.

The search processing unit 133D ends the search process when the size relationship between the first search range and the second search range satisfies the condition. The search processing unit 133D ends the search process when the first search range exceeds the second search range.

[5-3. Information processing flow]
Next, the information processing procedure according to the fifth embodiment will be described using FIG. 42. FIG. 42 is a flowchart illustrating an example of information processing according to the fifth embodiment.

First, FIG. 42 will be explained. For example, FIG. 42 is a flowchart illustrating an example of group generation processing. As shown in FIG. 42, the information processing apparatus 100D obtains a group connection graph in which a plurality of groups into which a plurality of objects to be data searched are classified are connected by edges (step S801). For example, the information processing device 100D obtains a blob connection graph from the blob connection graph information storage unit 126 as a group connection graph.

The information processing device 100D obtains search queries for a plurality of objects (step S802). For example, the information processing device 100D obtains a search query from the terminal device 10 (see FIG. 4) used by the user.

The information processing device 100D searches for a group connection graph using a first search range indicating a range corresponding to a search of a plurality of groups and a second search range indicating a range corresponding to a search of a plurality of objects. Accordingly, a search process is executed to extract a nearby object corresponding to the search query from a plurality of objects (step S803). For example, the information processing device 100D executes the search process shown in FIG. 43 using a blob search range that is a first search range and an object search range that is a second search range.

[5-4. Search processing example]
Here, an example of the search process according to the fifth embodiment will be described using FIG. 43 as an example. FIG. 43 is a flowchart illustrating an example of search processing according to the fifth embodiment. The search processing described below is performed by the search processing unit 133D of the information processing device 100D. Note that descriptions of points similar to the search processing described in the first embodiment, second embodiment, third embodiment, and fourth embodiment will be omitted as appropriate.

The information processing device 100D obtains a search blob starting point set and calculates the distance between each object and the query (step S901). For example, the information processing device 100D obtains a search blob starting point set indicating blobs that are search starting points, and calculates the distance between each object included in the blobs in the search blob starting point set and the query. For example, the information processing device 100D may select a search blob origin set at random, or may select a search blob origin set using a predetermined index, such as an index of a tree structure in which blobs are leaves. good. For example, the information processing device 100D calculates the approximate distance between each object included in the blob of the search blob origin set and the query by calculating the approximate distance described above.

The information processing device 100D sets the nearest neighbor blob of the query of the search blob origin set as the current nearest neighbor blob B (hereinafter also simply referred to as "blob B") (step S902). For example, the information processing device 100D determines the blob closest to the query as the current nearest neighbor blob B from the search blob origin set.

The information processing device 100D sets various parameters (step S903). In FIG. 43, the information processing apparatus 100D sets the number of blob searches to "1". Further, the information processing device 100D sets the object search radius R indicating the second search range (object search range) to “∞ (infinity)”. For example, the object search radius R is a radius centered on the query q that defines the second search range. Further, the information processing device 100D sets the object search radius E, which is an example of the second search range, to “∞ (infinity)”.

Further, the information processing device 100D sets a blob search radius Rb, which is an example of a first search range (blob search range), to “d(q,B)”. For example, the blob search radius Rb is a radius centered on the query q that defines the first search range. For example, the information processing device 100D sets the blob search radius Rb to the distance d(q,B) between the query q and the current nearest neighbor blob B. For example, the information processing device 100D sets the blob search radius Rb to the distance d(q,B) between the query q and the object closest to the query q among the objects included in the current nearest neighbor blob B. Note that the blob search radius Rb may be the distance between the query q and the centroid of the current nearest neighbor blob B. Further, the information processing device 100D sets the blob search radius Eb indicating the first search range (blob search range) to “Rb×(1+eb)”. Here, the parameter eb is a blob search radius expansion rate (also referred to as a "first search range coefficient"), and is set to an arbitrary value such as 0.1, for example. For example, the information processing device 100D sets the blob search radius Eb to a value obtained by multiplying the blob search radius Rb by 1+eb.

The information processing device 100D adds search blob origin sets other than blob B to the excluded blob set (step S904). For example, the information processing device 100D adds blobs excluding the current nearest neighbor blob B from the search blob origin set to the excluded blob set.

The information processing device 100D adds the search blob origin set to the unsearched blob set and the distance calculated set (step S905). For example, the information processing device 100D adds all blobs including the current nearest neighbor blob B from the search blob origin set to an unsearched blob set indicating that they have not yet been searched. For example, the information processing device 100D adds all blobs including the current nearest neighbor blob B from the search blob origin set to a distance calculation completed set indicating that distance calculation processing has been completed for objects within the blobs.

The information processing device 100D determines whether the unsearched blob set is empty (step S906). If the unsearched blob set is not empty (step S906: No), the information processing device 100D acquires the blob T closest to the query from the unsearched blob set (step S907). As a result, the unsearched blob set is removed from the unsearched blob set. For example, the information processing device 100D obtains, as the blob T, the blob with the shortest distance to the query q from among the set of unsearched blobs. For example, the distance between query q is the distance between query q and the object closest to query q among the objects included in the blob. Note that the distance between the query q and the blob may be the distance between the query q and the centroid of the blob.

The information processing device 100D determines whether the blob T is within the blob search range (step S908). For example, the information processing device 100D determines whether the distance between the blob T and the query q is less than or equal to the blob search radius Eb.

If the blob T is within the blob search range (step S908: Yes), the information processing device 100D determines whether all blobs connected by the edges of the blob T have been processed (step S909). For example, the information processing device 100D uses information indicating whether or not to process a blob (also referred to as a "connection blob") that is connected to the blob T by an edge ("processing/non-processing information"). Determine whether all connection blobs have been processed. For example, the information processing device 100D may manage whether or not a connected blob of the blob T is processed using information such as a flag. For example, the information processing device 100D updates the processing presence/absence information stored in the storage unit 120C according to the processing status, and refers to the processing presence/absence information in the storage unit 120C to determine whether all connected blobs of the blob T have been processed. You may decide whether

If the information processing device 100D has not processed all the blobs connected by the edges of the blob T (step S909: No), the information processing device 100D acquires the blob J connected by the edges from the blob T (step S910). For example, the information processing device 100D selects a blob that is unprocessed as a blob that is connected to the blob T by an edge (an "unprocessed connected blob") among blobs that are connected to the blob T by an edge. ) is obtained as blob J. For example, the information processing device 100D refers to the processing presence/absence information, acquires an unprocessed connected blob as a blob J among the blobs connected to the blob T by an edge, and processes the blob acquired as the blob J. Update the presence/absence information to processed.

The information processing device 100D determines whether blob J exists in the distance calculation completed set (step S911). If the blob J exists in the distance-calculated set (step S911: Yes), the information processing device 100D returns to step S909 and performs the process.

Further, if blob J does not exist in the distance calculated set (step S911: No), the information processing device 100D adds blob J to the distance calculated set (step S912), and adds blob J to the unsearched blob set. (Step S913). The information processing device 100D then calculates the distance De between the centroid of the blob J and the query (step S914).

The information processing device 100D determines whether the distance De is smaller than the blob search radius Rb (step S915). If the distance De is not smaller than the blob search radius Rb (step S915: No), the information processing device 100D adds blob J to the excluded blob set (step S916), and returns to step S909 for processing.

Further, if the distance De is smaller than the blob search radius Rb (step S915: Yes), the information processing device 100D adds the current nearest neighbor blob B to the excluded blob set and sets the blob J as the current nearest neighbor blob B ( Step S917). In this way, the information processing device 100D updates the current nearest neighbor blob B to the blob J when the distance De is smaller than the blob search radius Rb.

Then, the information processing device 100D updates parameters related to the blob based on the current nearest neighbor blob B that has been updated to the blob J (step S918). In FIG. 43, the information processing device 100D updates the blob search radius Rb to “d(q,B)” based on the current nearest neighbor blob B updated to the blob J. Furthermore, the information processing device 100D updates the blob search radius Eb to “Rb×(1+eb)” based on the updated blob search radius Rb. The information processing apparatus 100D then returns to step S909 and performs the process.

When the information processing device 100D has processed all the blobs connected by the edges of the blob T (step S909: Yes), the information processing device 100D returns to step S906 and performs the processing. If the unsearched blob set is empty (step S906: Yes), the information processing device 100D performs the process of step S920. Further, if the blob T is not within the blob search range (step S908: No), the information processing device 100D returns the blob T to an unsearched blob (step S919), and performs the process of step S920.

In step S920, the information processing device 100D adds 1 to the number of blob searches (step S920). Then, the information processing device 100D calculates the approximate distances of all objects in the blob B, and sets the shortest distance as the blob distance of the blob B (step S921). For example, the information processing device 100D calculates approximate distances between all objects in the blob B and the query q, and calculates the distance between the query q and the object closest to the query q among the objects included in the current nearest neighbor blob B. Set the distance between them to the blob distance of blob B. Note that the blob distance of blob B is used in the end determination process in step S924.

The information processing device 100D adds all the objects in the blob B to the search result set, sorts them, and sets only the designated top search result set as the search result set (step S922). For example, the information processing device 100D adds all objects in the blob B to a search result set indicating object candidates near the query q. The information processing device 100D then updates the search result set by sorting the objects in the search result set after adding the objects in blob B in descending order of distance from the query q, and updates the search result set with the updated search results. Objects lower than a specified number of searches (for example, 5, 10, etc.) in the set are excluded from the search result set. For example, the information processing device 100D selects objects in the search result set after adding the objects in blob B, starting from the one with the shortest distance from the query q, up to the number of searches (for example, 5, 10, etc.) (i.e., Keep the number of searched objects) and exclude the others.

Then, the information processing device 100D updates parameters related to the object based on the updated search result set (step S923). In FIG. 43, the information processing device 100D updates the object search radius R to the maximum distance of objects in the search result set based on the updated search result set. For example, the information processing device 100D updates the object search radius R to the distance of the object furthest from the query q in the search result set. Furthermore, the information processing device 100D sets the object search radius E to “R×(1+eo)” based on the updated search result set. Here, the parameter eo is an object search radius expansion rate (also referred to as a "second search range coefficient"), and is set to an arbitrary value such as 0.05, for example. For example, the parameter eo corresponds to the above-mentioned search range coefficient "ε". For example, the information processing device 100D sets the object search radius E to a value obtained by multiplying the object search radius R by 1+eo. Parameter eo and parameter eb may have the same value.

Then, the information processing device 100D determines whether termination criteria #1 to #3 are met (step S924). The information processing device 100D determines whether the processing status meets any of termination criteria #1 to #3 based on each termination condition. Termination criterion #1 is that the search is terminated when the upper limit number of blobs is exceeded. For example, termination criterion #1 is a criterion that requires that the number of blob searches, which is a parameter, exceeds the upper limit number of searches (for example, an arbitrary value such as 5, 10, etc.).

Termination criterion #2 is to terminate when the blob distance exceeds the search radius of the object. For example, termination criterion #2 is a criterion that requires the blob distance to exceed the object search radius E. Termination criterion #3 is that the search is terminated when the blob search radius exceeds the object search radius. For example, termination criterion #3 is a criterion that requires the blob search radius Eb to exceed the object search radius E. Note that the above termination criteria #1 to #3 are merely examples, and various termination criteria may be used. Further, a plurality of termination criteria may be used in combination.

If the termination criteria #1 to #3 are not applicable (step S924: No), the information processing device 100D obtains the blob with the shortest distance from the excluded blob set and sets it as the current nearest neighbor blob B (step S925). For example, the information processing device 100D obtains the blob with the shortest distance from the query q as the current nearest neighbor blob B from the set of excluded blobs. As a result, the blob with the shortest distance at that time is removed from the excluded blob set. For example, the information processing device 100D updates the current nearest neighbor blob B to the blob with the shortest distance to the query q among the set of excluded blobs.

Then, the information processing device 100D updates parameters regarding the blob based on the current nearest neighbor blob B that has been updated to the blob with the shortest distance (step S926). In FIG. 43, the information processing device 100D updates the blob search radius Rb to "d(q,B)" based on the current nearest neighbor blob B that has been updated to the blob with the shortest distance. Furthermore, the information processing device 100D updates the blob search radius Eb to “Rb×(1+eb)” based on the updated blob search radius Rb. The information processing apparatus 100D then returns to step S906 and performs the process.

Further, if the termination criteria #1 to #3 are met (step S924: Yes), the information processing device 100D terminates the process. For example, if the termination criteria #1 to #3 apply, the information processing device 100D determines the objects included in the search result set at that time to be objects in the vicinity of the query q. Then, the information processing device 100D provides information indicating the objects determined as objects in the vicinity of the query q to the requestor of the search that specified the query q.

[6. Sixth embodiment]
In the example described above, a case is shown in which an object is clustered (classified) into a plurality of blobs, and each blob is further clustered (classified) into a plurality of quantized clusters (also simply referred to as "clusters"). For example, in the fourth embodiment and the fifth embodiment, the number (first number) of the first group (blob) is the number (second number) of the second group (cluster) that is the unit for quantizing objects. Although the case where the number of clusters is larger is shown, the number of second groups (clusters) (second number) may be larger than the number of first groups (blobs) (first number). For example, the information processing apparatus 100E according to the sixth embodiment performs hierarchical clustering (also referred to as "hierarchical clustering") in which the classification of the second group (cluster) is further classified into the first group (blob). and a second group may be generated. That is, the second group (cluster) may be in a higher hierarchy than the first group (blob). Note that, as shown in the fourth embodiment and the fifth embodiment, the first group (blob) may be in a higher hierarchy than the second group (cluster).

This point will be described below as a sixth embodiment. In the sixth embodiment, the information processing system 1 includes an information processing device 100E instead of the information processing device 100, the information processing device 100A, the information processing device 100B, the information processing device 100C, or the information processing device 100D. Note that descriptions of points similar to those described above in the first embodiment, second embodiment, third embodiment, fourth embodiment, fifth embodiment, etc. will be omitted as appropriate.

The information processing device 100E stores blob information (first group information) indicating a plurality of blobs that classify a plurality of objects, and cluster information (second group information) indicating a cluster, which is a group classified differently from the blobs. generate. In the following, a classification process for classifying multiple objects into a first group (blob) will be referred to as a first classification process, and a classification process for classifying multiple objects into a second group (cluster) will be referred to as a second classification process. There are cases.

[6-1. Information processing〕
First, an overview of information processing according to the sixth embodiment will be explained using FIG. 44. FIG. 44 is a diagram illustrating an example of information processing according to the sixth embodiment. Specifically, FIG. 44 is a diagram showing a first generation example of the first group and the second group in the sixth embodiment. Note that the following describes a case where the information processing device 100E executes the first classification process after executing the second classification process, but if the desired first group and second group can be generated, the first classification After performing the process, a second classification process may be performed.

Prior to explaining information processing by the information processing device 100E, the illustrated aspect of FIG. 44 will be briefly explained. In FIG. 44, the distinction between blobs is shown by solid thick lines, and the connections between blobs are shown by dotted lines connecting centroids C of blobs BL. In this way, when explaining the centroid of each blob BL without distinguishing it, it will be described as "centroid C." For example, among the solid lines, thick lines indicate divisions corresponding to blobs, and thin lines indicate divisions corresponding to clusters. For example, in FIG. 44, thin solid lines indicate divisions corresponding to clusters within a blob.

In FIG. 44, centroids CN21 to CN30 indicate the centroids of each cluster CL21 to CL30. In addition, when explaining the centroid of each cluster CL without distinguishing it, it is written as "centroid CN." Further, in FIG. 44, centroids C21 to C24 indicate the centroids of each of the blobs BL21 to BL24. Blob BL21 includes three clusters CL21, CL22, and CL23. Cluster CL21 includes nodes N31, N205, etc. The blob BL22 includes two clusters CL24 and CL25. Blob BL23 includes three clusters CL26, CL27, and CL28. The blob BL24 includes two clusters CL29 and CL30. Note that the blob BL, cluster CL, node N, etc. shown in FIG. Node N is included. Note that the flow of a specific search process using the graph GR71 shown in the spatial information SP71 is shown in FIG.

An example of the first group and second group generation processing performed by the information processing device 100E will be described. First, the information processing device 100E executes a second classification process to generate blob information for classifying a plurality of objects into a plurality of blobs. For example, the information processing device 100E generates blob information indicating a blob that classifies a plurality of objects using an arbitrary method. The information processing device 100E executes a second classification process of classifying a plurality of objects into a plurality of blobs using arbitrary clustering such as k-means.

In the example of FIG. 44, the information processing device 100E classifies a plurality of objects into a plurality of blobs such as blobs BL21 to BL24. Note that the information processing apparatus 100E may use any method to generate clusters as long as it can generate a plurality of objects. For example, the information processing device 100E may generate blob information using an index of a graph (for example, graph GR71).

Further, the information processing device 100E generates cluster information for classifying a plurality of objects into a plurality of clusters (step S71). The information processing device 100E generates cluster information for classifying a plurality of objects into clusters CL21 to CL30, etc., using an arbitrary method. For example, the information processing device 100E uses information generated by executing the second classification process to generate cluster information for classifying a plurality of objects into a plurality of clusters. In the example of FIG. 44, the information processing device 100E classifies a plurality of objects into a plurality of clusters by performing a first classification process of classifying each of a plurality of blobs, such as blobs BL21 to BL24, into a plurality of clusters. Generate cluster information. The information processing device 100E generates cluster information for classifying a plurality of objects into a plurality of clusters by classifying a plurality of objects included in each of a plurality of blobs such as blobs BL21 to BL24 into a plurality of clusters.

Note that the above is just an example, and the information processing apparatus 100E may use any method to generate clusters as long as it can generate clusters. The information processing device 100E may classify each node (object) into one of the clusters CL21 to CL30 using arbitrary clustering such as k-means. For example, the information processing apparatus 100E may generate clusters using an index such as a graph.

In the example of FIG. 44, the information processing device 100E generates cluster information such as clusters CL21 to CL30, which are obtained by further dividing blobs BL21 to BL24, etc. For example, the information processing device 100E may classify each object into one of the clusters CL21 to CL30 by clustering the objects belonging to each blob BL21 to BL24.

In the example of FIG. 44, the information processing device 100E divides the blob BL21 into a cluster CL21, a cluster CL22, and a cluster CL23. That is, the information processing device 100E classifies objects belonging to the blob BL21 into objects belonging to the cluster CL21, objects belonging to the cluster CL22, and objects belonging to the cluster CL23. Note that although the cluster boundaries are shown to be on the blob boundaries in FIG. 44, they may not match depending on the dividing method, and the space may not always be divided by the boundary lines depending on the dividing method. Further, FIG. 44 conceptually shows objects, centroids, etc. in order to explain the generation process, and the positions of centroids, etc. do not indicate accurate positions.

Furthermore, the information processing device 100E divides the blob BL22 into a cluster CL24 and a cluster CL25. For example, the information processing device 100E divides the blob BL23 into clusters CL26, CL27, and cluster CL28. The information processing device 100E divides the blob BL24 into a cluster CL29 and a cluster CL30. In this way, the information processing device 100E uses the first group of information to generate the second group of information. In this way, the information processing device 100E performs the second classification process using the first group information generated by the first classification process.

Note that in FIG. 44, the information processing device 100E generates the graph GR71, which is a blob connection graph, as shown in the spatial information SP71, but it is not necessary to generate the graph GR71. For example, when generating the graph GR71, the information processing device 100E generates the graph GR71 by processing using information similar to that in FIGS. 23, 29, etc. Therefore, detailed explanation will be omitted.

For example, the information processing device 100E executes processing related to quantization using blob information. The information processing device 100E performs processing related to quantization targeting a plurality of objects using blob information. For example, the information processing device 100E performs quantization for each object belonging to each of the clusters CL21 to CL30, but this point will be described later.

As described above, the information processing device 100E stores cluster information indicating a plurality of clusters that classify a plurality of objects that are data search targets, and blob information indicating a plurality of blobs that are classified differently from the plurality of clusters. generate. Specifically, the information processing device 100E generates a first number of first groups (blobs) and a second number of second groups (clusters) that is larger than the first number.

Thereby, the information processing apparatus 100E can generate two types of groups for classifying a plurality of objects, and can generate groups for classifying objects. For example, in the case of a data structure in which a plurality of blobs (first group) are included in one quantization cluster (second group), high search performance may not be achieved. Even in such a case, the information processing device 100E maintains an appropriate group relationship by generating a first group with a first number and a second group with a second number larger than the first number. It becomes possible to improve search performance.

[6-1-1. Other processing examples]
Note that the process shown in FIG. 44 is only an example, and the information processing apparatus 100E may perform any type of process to generate blob information and cluster information as long as it is capable of generating the blob information and cluster information.

For example, the information processing device 100E may generate cluster information for classifying into clusters and then generate blob information for classifying into blobs. In this case, the information processing apparatus 100E may generate cluster information for classifying each object into one of the plurality of clusters by performing clustering in which the plurality of objects are classified into clusters. For example, the information processing device 100E may classify each object into one of the clusters CL21 to CL30 using arbitrary clustering such as k-means.

Then, the information processing device 100E generates blob information that is classified into blobs BL21 to BL24, etc., such as clusters CL21 to CL30. The information processing device 100E may classify clusters CL21 to CL30, etc. into any of a plurality of blobs using arbitrary clustering such as k-means. Note that the information processing device 100E may use any method to generate blobs as long as it is capable of generating blobs. For example, the information processing device 100E may generate a blob using an index such as a graph.

For example, the information processing device 100E classifies cluster CL21, cluster CL22, and cluster CL23 into blob BL21. Furthermore, the information processing device 100E classifies cluster CL24 and cluster CL25 into blob BL22. For example, the information processing device 100E classifies clusters CL26, CL27, and cluster CL28 into blob BL23. The information processing device 100E classifies cluster CL29 and cluster CL30 into blob BL24. Thereby, the information processing device 100E generates the first group of information using the second group of information. In this way, the information processing device 100E may perform the first classification process using the second group information generated by the second classification process.

[6-2. Configuration of information processing device]
Next, the configuration of the information processing apparatus 100E according to the sixth embodiment will be described using FIG. 45. FIG. 45 is a diagram illustrating a configuration example of an information processing apparatus according to the sixth embodiment. As shown in FIG. 45, the information processing device 100E includes a communication section 110, a storage section 120C, and a control section 130B. Note that in the information processing device 100E, descriptions of the same points as the information processing device 100, the information processing device 100A, the information processing device 100B, the information processing device 100C, or the information processing device 100D will be omitted as appropriate.

(Storage unit 120C)
The storage unit 120C is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 45, the storage unit 120C according to the sixth embodiment includes an object information storage unit 121, a graph information storage unit 122, a quantization information storage unit 123, a codebook information storage unit 124, and a blob information storage unit 121. It has an information storage section 125, a blob connection graph information storage section 126, and a cluster information storage section 127.

(Control unit 130E)
The control unit 130E is a controller, and allows various programs (corresponding to an example of an information processing program) stored in a storage device inside the information processing device 100E to operate the RAM by, for example, a CPU, MPU, GPU, etc. This is achieved by executing it as a region. Further, the control unit 130E is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

As shown in FIG. 45, the control unit 130E includes an acquisition unit 131, a generation unit 132E, a search processing unit 133E, and a provision unit 134, and realizes or executes information processing functions and operations described below. do. Note that the internal configuration of the control unit 130E is not limited to the configuration shown in FIG. 45, and may be any other configuration as long as it performs information processing to be described later.

The acquisition unit 131 according to the sixth embodiment acquires object information indicating a plurality of objects that are data search targets. The acquisition unit 131 acquires index information indicating an index that searches for a plurality of objects.

The acquisition unit 131 according to the sixth embodiment acquires various information in the same way as the acquisition unit 131 according to the first to fifth embodiments. For example, the acquisition unit 131 acquires object information indicating a plurality of objects that are data search targets. For example, the acquisition unit 131 acquires index information indicating an index that searches for multiple objects.

(Generation unit 132E)
The generating unit 132E generates various information in the same way as the generating unit 132, 132A, 132B, or 132C.

Based on the object information acquired by the acquisition unit 131, the generation unit 132E generates first group information indicating a plurality of first groups, which are a first number of groups into which a plurality of objects are classified, and a plurality of first groups. generates second group information indicating a plurality of second groups, which are groups of different classifications and are a second number of groups that is larger than the first number for classifying the plurality of objects.

The generation unit 132E generates first group information that is a group that classifies objects belonging to the same second group among the plurality of second groups into the same first group. The generation unit 132E generates first group information indicating a plurality of first groups in which objects belonging to each of two or more second groups among the plurality of second groups are classified into one first group.

The generation unit 132E generates first group information indicating a plurality of first groups used for performing batch processing in a search process targeting a plurality of objects. The generation unit 132E generates first group information indicating a plurality of first groups used to collectively calculate distances regarding a plurality of objects. The generation unit 132E generates first group information indicating a plurality of first groups used to calculate the distance between the search query and the object used in the search process in parallel.

The generation unit 132E generates second group information indicating a plurality of second groups used to perform processing related to quantization targeting a plurality of objects. The generation unit 132E generates second group information indicating a plurality of second groups that are units of quantization. The generation unit 132E generates second group information indicating a plurality of second groups used to quantize vectors corresponding to each of the plurality of objects.

After classifying the plurality of second groups, the generation unit 132E performs classification of the plurality of first groups. The generation unit 132E generates second group information indicating a plurality of second groups by clustering a plurality of objects. The generation unit 132E uses the index information to generate second group information indicating a plurality of second groups. The generation unit 132E generates first group information indicating a plurality of first groups using second group information indicating a second group.

The generation unit 132E generates first group information indicating a plurality of first groups by clustering a plurality of second groups. After classifying the plurality of first groups, the generation unit 132E performs classification of the plurality of second groups. The generation unit 132E generates first group information indicating a plurality of first groups by clustering a plurality of objects.

The generation unit 132E generates second group information indicating a plurality of second groups using first group information indicating the first group. The generating unit 132E generates second group information indicating the plurality of second groups by dividing at least one first group among the plurality of first groups and generating two or more second groups. The generation unit 132E generates second group information indicating a plurality of second groups by clustering objects belonging to each first group and dividing each first group into two or more second groups.

The generation unit 132E performs a first classification process, which is a classification process of classifying a plurality of objects into a plurality of first groups, based on the object information acquired by the acquisition unit 131, and a first classification process that classifies a plurality of objects into a plurality of first groups. are groups with different classifications. Execute one of the second classification processes, which is a classification process to classify into a plurality of second groups, and after one classification process, perform the other classification process. By executing the classification process using the results, first group information indicating a plurality of first groups for classifying a plurality of objects and second group information indicating a plurality of second groups for classifying a plurality of objects are generated. generate.

The generation unit 132E generates first group information that is a group that classifies objects belonging to the same second group among the plurality of second groups into the same first group. The generating unit 132E indicates a plurality of first groups in which other objects are classified into another first group different from the one first group to which one object belongs among the objects belonging to at least one second group. Generate first group information.

The generation unit 132E generates the first group information and the second group information by performing the first classification process, which is the other classification process, after the second classification process, which is one of the classification processes. The generation unit 132E generates second group information indicating a plurality of second groups by a second classification process that clusters a plurality of objects. The generation unit 132E generates second group information by a second classification process using index information.

The generation unit 132E generates first group information through a first classification process using the second group information generated through the second classification process. The generation unit 132E generates first group information indicating a plurality of first groups by performing a first classification process using a clustering result obtained by clustering a plurality of objects.

The generation unit 132E generates first group information through a first classification process using second centroids corresponding to each of the plurality of second groups. The generation unit 132E generates first group information by performing a first classification process that causes each of the second groups to belong to a candidate first group, which is the first group to which the largest object belongs among the first groups.

The generation unit 132E generates first group information by performing a first classification process using partial centroids corresponding to partial groups obtained by dividing each of the second groups into a plurality of parts. The generation unit 132E generates first group information through a first classification process that causes each of the second groups to belong to a candidate first group, which is the first group to which the partial centroid to which it belongs is the largest among the first groups. .

When there are a plurality of candidate first groups in one second group, the generation unit 132E selects the first group whose centroid is closest to the second centroid corresponding to one second group among the plurality of candidate first groups. First group information is generated by a first classification process that assigns one second group to the second group.

(Search processing unit 133E)
The search processing unit 133E executes the search process shown in FIG. 48. For example, the search processing section 133E performs various processes related to search processing in the same way as the search processing section 133, the search processing section 133A, the search processing section 133B, the search processing section 133C, or the search processing section 133D. The search processing unit 133E performs processing using the information stored in the storage unit 120C. The search processing unit 133E performs processing using the information generated by the generation unit 132E.

For example, the search processing unit 133E searches for a group connection graph using a first search range that indicates a range that corresponds to a search for multiple groups, and a second search range that indicates a range that corresponds to a search for multiple objects. By doing so, a search process is executed to extract nearby objects corresponding to the search query from a plurality of objects. The search processing unit 133E executes a search process using a first search range and a second search range that is wider than the first search range. Note that the search processing executed by the search processing unit 133E is similar to that of the search processing unit 133D, except that the second number of the second group is larger than the first number of the first group, so the details will not be explained. Further explanation will be omitted.

[6-3. Transposed index example]
An example of a transposed index in the sixth embodiment will now be described using FIG. 46. FIG. 46 is a diagram illustrating an example of a transposed index according to the sixth embodiment. Note that descriptions of points similar to the transposed index shown in FIGS. 46 and 36 will be omitted as appropriate.

First, the transposed index IND3 shown in FIG. 46 will be explained. The transposed index IND3 shown in FIG. 46 includes items such as "blob ID", "number of clusters", "cluster #1", and "cluster #2". Note that although two clusters, “cluster #1” and “cluster #2” are illustrated in FIG. 46, the transposed index IND3 corresponds to the number of clusters (second group) belonging to each blob (first group). Contains several items. Hereinafter, when "cluster #1", "cluster #2", etc. are collectively referred to, they may be written as "cluster".

“Clusters” such as “cluster #1” and “cluster #2” include items such as “cluster ID”, “quantization object number”, and “quantization object”. Note that although FIG. 46 shows a case where the "quantization object" stores information regarding a plurality of quantization objects in a format such as an array (list), "quantization object #1" in FIG. , "Quantization Object #2," "Quantization Object #3," and "Quantization Object #4" may also be included.

“Blob ID” indicates identification information for identifying blobs that are the first group. "Number of clusters" indicates the number of clusters belonging to the corresponding blob. “Cluster” indicates information regarding clusters belonging to the corresponding blob.

"Cluster ID" indicates identification information for identifying the cluster that is the second group. "Number of quantized objects" indicates the number of objects belonging to a cluster. "Quantized object" indicates a quantized object. For example, in "quantization object", an object is subjected to product quantization, and a list of information indicating the codebook corresponding to each partial vector is stored.

The example in FIG. 46 shows that four clusters belong to the blob with blob ID "1". This indicates that a cluster with cluster ID "2", a cluster with cluster ID "5", etc. belong to the blob with blob ID "1".

It also shows that the number of objects belonging to the cluster with cluster ID "2" is four. Information indicating an object belonging to the cluster with cluster ID "2" is associated with cluster ID "2". For example, information indicating a list in which objects belonging to the cluster with cluster ID "2" are arranged in order is associated with cluster ID "2".

Furthermore, for the cluster with cluster ID "2", "4, 2, 8, 10" are stored in the "quantization object", and the object is composed of four partial vectors, each with codebook "4", It shows that it is quantized with "2", "8", and "10". For example, for the first object belonging to the cluster with cluster ID "2", the first partial vector among the four divided partial vectors is quantized with codebook "4", and the second The subvector of is quantized with codebook "2", the third subvector is quantized with codebook "8", and the last subvector is quantized with codebook "10".

Furthermore, for the cluster with the cluster ID "2", "5, 6, 2, 0" are stored in the "quantization object", and the object is composed of four partial vectors, each of which has the codebook "5", It shows that it is quantized with "6", "2", and "0". For example, for the second object belonging to the cluster with cluster ID "2", the first partial vector of the four divided partial vectors is quantized with codebook "5", and the second The subvector is quantized with codebook "6", the third subvector is quantized with codebook "2", and the last subvector is quantized with codebook "0".

Note that the transposed index IND3 is not limited to the above, and may store various information depending on the purpose. For example, the transposed index IND3 may store information indicating each quantization object (object ID, etc.) in association with each quantization object.

As described above, information for identifying clusters (cluster ID) is attached to the transposed index, and the information processing device 100E calculates the residual vector using the transposed index. For example, the residual vector is used during index generation and search, but since the process is similar to that in conventional techniques such as Patent Document 3, detailed explanation will be omitted as appropriate.

[6-4. Information processing flow]
Next, the information processing procedure according to the sixth embodiment will be explained using FIG. 47. FIG. 47 is a flowchart illustrating an example of information processing according to the sixth embodiment.

First, FIG. 47 will be explained. For example, FIG. 47 is a flowchart illustrating an example of group generation processing. As shown in FIG. 47, the information processing apparatus 100E acquires object information indicating a plurality of objects that are data search targets (step S1001). For example, the information processing device 100E obtains object information indicating a plurality of objects from the object information storage unit 121.

The information processing device 100E generates first group information indicating a plurality of first groups, which are a first number of groups into which a plurality of objects are classified, based on the object information (step S1002). For example, the information processing device 100E generates blob information indicating a plurality of blobs that classify a plurality of objects based on the object information.

The information processing device 100E is configured to display a plurality of second groups that are classified into a group different from the plurality of first groups and have a second number larger than the first number for classifying the plurality of objects. Group information is generated (step S1003). For example, the information processing device 100E generates second groups with a second number greater than the first number of first groups.

[6-5. Search processing example]
Here, an example of the search process according to the sixth embodiment will be described using FIG. 48 as an example. FIG. 48 is a flowchart illustrating an example of search processing according to the sixth embodiment. The search processing described below is performed by the search processing unit 133E of the information processing device 100E. Each process of steps S1101 to S1128 in FIG. 48 corresponds to each of steps S901 to S928 in FIG. 43. Therefore, in each process of steps S1101 to S1128, descriptions of points similar to those of steps S901 to S928 will be omitted as appropriate. For example, the process in FIG. 48 and the process in FIG. 43 are the same as the process in FIG. 43 except that step S1121 is partially changed from the process in step S921.

The information processing device 100E obtains the search blob starting point set and calculates the distance between each object and the query (step S1101). The information processing device 100E sets the nearest neighbor blob of the query of the search blob origin set as the current nearest neighbor blob B (blob B) (step S1102). The information processing device 100E sets various parameters (step S1103).

The information processing device 100E adds search blob origin sets other than blob B to the excluded blob set (step S1104). The information processing device 100E adds the search blob origin set to the unsearched blob set and the distance calculated set (step S1105).

The information processing device 100E determines whether the unsearched blob set is empty (step S1106). If the unsearched blob set is not empty (step S1106: No), the information processing device 100E acquires the blob T closest to the query from the unsearched blob set (step S1107).

The information processing device 100E determines whether the blob T is within the blob search range (step S1108). If the blob T is within the blob search range (step S1108: Yes), the information processing device 100E determines whether all blobs connected by the edges of the blob T have been processed (step S1109). For example, the information processing device 100E processes all connected blobs of blob T using information (processing presence/absence information) indicating whether or not to process blobs (connection blobs) that are connected to blob T by an edge. Determine whether or not. For example, the information processing device 100E may manage whether or not a connected blob of the blob T is processed using information such as a flag. For example, the information processing device 100E updates the processing presence/absence information stored in the storage unit 120C according to the processing status, and refers to the processing presence/absence information in the storage unit 120C to determine whether all connected blobs of the blob T have been processed. You may decide whether

If the information processing device 100E has not processed all the blobs connected by the edges of the blob T (step S1109: No), the information processing device 100E acquires the blob J connected by the edges from the blob T (step S1110).

The information processing device 100E determines whether blob J exists in the distance calculation completed set (step S1111). If the blob J exists in the distance calculation completed set (step S1111: Yes), the information processing device 100E returns to step S1109 and performs the process.

Furthermore, if blob J does not exist in the distance calculated set (step S1111: No), the information processing device 100E adds blob J to the distance calculated set (step S1112), and adds blob J to the unsearched blob set. (Step S1113). The information processing device 100E then calculates the distance De between the centroid of the blob J and the query (step S1114).

The information processing device 100E determines whether the distance De is smaller than the blob search radius Rb (step S1115). If the distance De is not smaller than the blob search radius Rb (step S1115: No), the information processing device 100E adds blob J to the excluded blob set (step S1116), and returns to step S1109 for processing.

Further, if the distance De is smaller than the blob search radius Rb (step S1115: Yes), the information processing device 100E adds the current nearest neighbor blob B to the excluded blob set and sets the blob J as the current nearest neighbor blob B ( Step S1117). In this way, the information processing device 100E updates the current nearest neighbor blob B to the blob J when the distance De is smaller than the blob search radius Rb.

The information processing device 100E then updates the parameters regarding the blob based on the current nearest neighbor blob B that has been updated to the blob J (step S1118). In FIG. 48, the information processing device 100E updates the blob search radius Rb to "d(q,B)" based on the current nearest neighbor blob B updated to blob J. Furthermore, the information processing device 100E updates the blob search radius Eb to “Rb×(1+eb)” based on the updated blob search radius Rb. The information processing apparatus 100E then returns to step S1109 and performs the process.

When the information processing device 100E has processed all the blobs connected by the edges of the blob T (step S1109: Yes), the information processing device 100E returns to step S1106 and performs the processing. If the unsearched blob set is empty (step S1106: Yes), the information processing device 100E performs the process of step S1120. Further, if the blob T is not within the blob search range (step S1108: No), the information processing device 100E returns the blob T to an unsearched blob (step S1119), and performs the process of step S1120.

In step S1120, the information processing device 100E adds 1 to the number of blob searches (step S1120). Then, the information processing device 100E calculates approximate distances of all objects in the cluster for each cluster in the blob B, and sets the shortest distance as the blob distance of the blob B (step S1121). In this way, in FIG. 48, the information processing device 100E calculates approximate distances of all objects in the cluster for each cluster belonging to blob B, and calculates the shortest distance among all the objects for which the approximate distances have been calculated. Let the distance be the blob distance of the blob B.

The information processing device 100E adds all the objects in the blob B to the search result set, sorts them, and sets only the designated top search result set as the search result set (step S1122). Then, the information processing device 100E updates parameters related to the object based on the updated search result set (step S1123). The information processing device 100E then determines whether termination criteria #1 to #3 are met (step S1124).

If the termination criteria #1 to #3 are not applicable (step S1124: No), the information processing device 100E obtains the blob with the shortest distance from the excluded blob set and sets it as the current nearest neighbor blob B (step S1125).

Then, the information processing device 100E updates parameters regarding the blob based on the current nearest neighbor blob B that has been updated to the blob with the shortest distance (step S1126). In FIG. 48, the information processing device 100E updates the blob search radius Rb to "d(q,B)" based on the current nearest neighbor blob B that has been updated to the blob with the shortest distance. Furthermore, the information processing device 100E updates the blob search radius Eb to “Rb×(1+eb)” based on the updated blob search radius Rb. The information processing apparatus 100E then returns to step S1106 and performs the process.

Further, if the termination criteria #1 to #3 are met (step S1124: Yes), the information processing device 100E terminates the process. For example, if the termination criteria #1 to #3 apply, the information processing device 100E determines the objects included in the search result set at that time to be objects in the vicinity of the query q. Then, the information processing device 100E provides information indicating the objects determined to be objects in the vicinity of the query q to the requestor of the search that specified the query q.

[6-6. Group generation example]
Note that the first generation example described above is only an example of group generation, and the information processing apparatus 100E may generate groups in various ways. In FIG. 44, the information processing apparatus 100E uses hierarchical clustering to generate (clustering) a group by further dividing the first group as the second group. However, the information processing apparatus 100E is limited to hierarchical clustering. The first group and the second group may be generated by various processes. The second to fourth generation examples illustrating this point will be explained using FIGS. 49 to 51. FIGS. 49 to 51 are diagrams showing examples of generation of the first group and the second group. Note that descriptions of points similar to those described above will be omitted as appropriate.

[6-6-1. Second generation example]
First, a second generation example will be described using FIG. 49. FIG. 49 is a diagram illustrating an example of generation of the first group and the second group. Specifically, FIG. 49 is a diagram showing a second generation example of the first group and the second group in the sixth embodiment.

The illustrated aspect in FIG. 49 will be briefly described. In FIG. 49, the boundaries of clusters are shown by solid lines, and the boundaries of blobs are shown by dashed lines. Further, in FIG. 49, which blob each cluster belongs to is shown by connecting the centroid CN of the cluster CL and the centroid C of the blob BL with a dotted line. For example, since centroid CN34 of cluster CL34 and centroid CN35 of cluster CL35 are connected to centroid C33 of blob BL33, FIG. 49 shows a case where cluster CL34 and cluster CL35 belong to blob BL33. Note that FIG. 49 conceptually shows objects, centroids, etc. in order to explain the generation process, and the positions of centroids, etc. do not indicate exact positions.

From here, information processing regarding the second generation example by the information processing apparatus 100E will be described. The information processing device 100E generates cluster information that classifies multiple objects into multiple clusters. In the example of FIG. 49, the information processing device 100E generates cluster information for classifying a plurality of objects into clusters CL31 to CL40, etc., using an arbitrary method, as shown in spatial information SP72. The information processing device 100E may classify each node into one of clusters CL31 to CL40 using arbitrary clustering such as k-means. Note that the information processing apparatus 100E may use any method to generate clusters as long as the clusters can be generated.

The information processing device 100E then generates blob information that classifies the multiple clusters into multiple blobs. For example, the information processing device 100E uses the centroids CN31 to CN40 of each of the clusters CL31 to CL40 to generate blob information indicating a blob that classifies the clusters CL31 to CL40. In this way, the information processing device 100E generates first group information through the first classification process using the second centroids corresponding to each of the plurality of second groups. For example, the information processing device 100E classifies each of the centroids CN31 to CN40 of the clusters CL31 to CL40 into one of a plurality of blobs using arbitrary clustering such as k-means. For example, the information processing device 100E generates a first number of centroids CN that is smaller than the second number of clusters CL, and associates the generated centroid CN with the centroid CN of the blob BL, thereby converting the cluster CL into a blob BL. Classify into one of the following.

In the example of FIG. 49, the information processing device 100E assigns each of the centroids CN31 to CN40, etc. to any of the centroids C31 to C34, etc. of the blobs BL31 to BL34. Generate blob information that is classified into etc. Through the process described above, each centroid CN of the cluster CL is assigned to each blob BL.

For example, the information processing device 100E classifies cluster CL31, cluster CL38, and cluster CL40 into blob BL31 by assigning centroid CN31, centroid CN38, and centroid CN40 to centroid C31. Furthermore, the information processing device 100E classifies cluster CL32 and cluster CL33 into blob BL32 by assigning centroid CN32 and centroid CN33 to centroid C32. Furthermore, the information processing device 100E classifies cluster CL34 and cluster CL35 into blob BL33 by assigning centroid CN34 and centroid CN35 to centroid C33. Furthermore, the information processing device 100E classifies cluster CL36, cluster CL37, and cluster CL39 into blob BL34 by assigning centroid CN36, centroid CN37, and centroid CN39 to centroid C34.

Although not shown in FIG. 49, the information processing device 100E generates a blob connection graph that connects a plurality of blobs such as blobs BL31 to BL34. For example, the information processing apparatus 100E generates a blob connection graph by processing using information similar to that shown in FIGS. 23, 29, 44, etc., but detailed description thereof will be omitted.

For example, the information processing device 100E executes processing related to quantization using blob information. The information processing device 100E performs processing related to quantization targeting a plurality of objects using blob information. For example, the information processing device 100E performs quantization for each object belonging to each of the blobs BL31 to BL34, but a description of the same processing as described above will be omitted as appropriate.

As described above, the information processing device 100E stores cluster information indicating a plurality of clusters that classify a plurality of objects that are data search targets, and blob information indicating a plurality of blobs that are classified differently from the plurality of clusters. generate. Thereby, the information processing apparatus 100E can generate two types of groups for classifying a plurality of objects, and can generate groups for classifying objects.

[6-6-2. Third generation example]
Next, a third generation example will be described using FIG. 50. FIG. 50 is a diagram illustrating an example of generation of the first group and the second group. Specifically, FIG. 50 is a diagram showing a third generation example of the first group and the second group in the sixth embodiment. Note that descriptions of points similar to those described above will be omitted as appropriate.

In the third generation example, the information processing device 100E performs a first classification process that causes each of the second groups to belong to a candidate first group to which the object belongs to the largest first group among the plurality of first groups. , generates first group information. For example, the information processing device 100E generates the first group information by performing a first classification process in which each of the clusters CL belongs to a candidate blob to which the object belongs to the largest blob BL among the plurality of blobs BL. For example, as shown in spatial information SP73 in FIG. 50, the information processing device 100E generates a first number of centroids CN that is smaller than the second number of clusters CL, and uses the generated centroid CN as the centroid of the blob BL. By associating the CNs, the cluster CL is classified into one of the blobs BL. Note that FIG. 50 conceptually shows objects, centroids, etc. in order to explain the generation process, and the positions of centroids, etc. do not indicate exact positions.

The information processing device 100E classifies the clusters CL using an arbitrary method. For example, the information processing device 100E may assign the cluster CL to the centroid C of the blob BL to which the centroid CN of the cluster CL is closest. For example, the information processing device 100E clusters all objects to generate blobs BL, and assigns a quantization cluster (cluster CL) to a blob BL having the largest number of objects belonging to each quantization cluster (cluster CL). For example, the information processing device 100E determines, as a candidate blob, the blob BL corresponding to the centroid C with the largest number of objects having the shortest distance for objects belonging to one cluster CL. For example, if the maximum number of individual blobs BL is the same, the information processing device 100E may assign the cluster CL to the blob BL with a closer centroid. In FIG. 50, for cluster CL31, the two objects corresponding to each of nodes N9 and N126 are closest to centroid C33. On the other hand, for cluster CL31, one object corresponding to node N7 is closest to centroid C32, and one object corresponding to node N85 is closest to centroid C31. That is, for cluster CL31, centroid C33 is the centroid C that has the largest number of objects with the shortest distance.

Therefore, in FIG. 50, the information processing device 100E determines the blob BL33 corresponding to the centroid C33 as the candidate blob for the cluster CL31. Since the candidate blob of the cluster CL31 is one of the blobs BL33, the information processing device 100E classifies the cluster CL31 into the blob BL33 by assigning the centroid CN31 of the cluster CL31 to the centroid C33 of the blob BL33. In FIG. 50, the clustering results for other clusters CN32 to CN40, etc. are similar to those in FIG. 49, so detailed explanations will be omitted.

Note that when there are a plurality of candidate first groups in one second group, the information processing device 100E selects a centroid whose centroid is closest to a second centroid corresponding to one second group among the plurality of candidate first groups. A second group is made to belong to the first group. For example, when there are multiple candidate blobs in one cluster CL, the information processing apparatus 100E causes the centroid CN corresponding to one cluster CL to belong to the blob BL closest to the centroid C among the multiple candidate blobs. In FIG. 50, for example, when the candidate blobs of cluster CL33 are two blobs BL31 and BL32, the information processing apparatus 100E determines that among blobs BL31 and BL32, centroid C is closest to centroid CN33 corresponding to cluster CL33. Cluster CL33 is made to belong to blob BL32.

The above is just an example, and the information processing device 100E may perform the generation process using various processes. For example, the information processing device 100E may randomly extract the same number of objects from each quantization cluster (cluster CL), or randomly extract objects according to the ratio of the number of objects in the clusters CL, instead of all objects. , clustering may be performed to generate the blob BL. Then, the information processing device 100E assigns a quantization cluster to the blob having the largest number of objects belonging to each quantization cluster (cluster CL).

[6-6-3. Fourth generation example]
Next, a fourth generation example will be described using FIG. 51. FIG. 51 is a diagram illustrating an example of generation of the first group and the second group. Specifically, FIG. 51 is a diagram showing a fourth generation example of the first group and the second group in the sixth embodiment. Note that descriptions of points similar to those described above will be omitted as appropriate.

In the fourth generation example, the information processing device 100E generates first group information by a first classification process using partial centroids corresponding to partial groups obtained by dividing each of the second groups into a plurality of groups. Specifically, the information processing apparatus 100E performs a first classification process that causes each of the second groups to belong to a candidate first group, which is the first group to which the partial centroid to which it belongs is the largest among the first groups. 1 group information is generated.

For example, the information processing device 100E divides each of the clusters CL into a plurality of subgroups, and for each cluster CL, out of the plurality of blobs BL, the information processing device 100E makes the candidate blob belong to the blob BL with the largest partial centroid. First group information is generated by the first classification process. The information processing device 100E generates a plurality of blobs BL using information on centroids (partial centroids) of partial clusters (partial groups). The information processing device 100E generates a plurality of blobs BL by clustering the partial centroids of the partial groups. In this way, the information processing device 100E further performs hierarchical clustering on each quantization cluster (cluster CL) to generate a centroid of the cluster (subgroup).

In addition, in FIG. 51, as shown in the spatial information SP74, the division of each cluster CL into subgroups is indicated by a chain double-dashed line within the cluster CL. Further, in FIG. 51, partial centroids corresponding to a plurality of partial groups of the cluster CL are illustrated only in a portion necessary for explanation. Specifically, in FIG. 51, the cluster CL31 is divided into two subgroups, the partial centroid PC11 corresponding to one subgroup (lower side in FIG. 51), and the partial centroid PC11 corresponding to the other subgroup (upper side in FIG. 51). Only the partial centroid PC12 corresponding to is shown. When partial centroids such as partial centroid PC11 and partial centroid PC12 are described without distinction, they may be referred to as partial centroid PC. Note that although FIG. 51 shows an example in which each cluster CL is divided into two subgroups for explanation, the number of subgroups (number of divisions) of each cluster CL is not limited to two, but may be any number. good. For example, each cluster CL may be determined according to the number of objects to which the cluster CL belongs. For example, each cluster CL may be divided into as many subgroups as the number of objects to which the cluster CL belongs.

For example, for partial centroid PCs belonging to one cluster CL, the information processing apparatus 100E determines, as a candidate blob, a blob BL corresponding to a centroid C having the largest number of partial centroid PCs with the shortest distance. In FIG. 51, for cluster CL31, partial centroid PC11 is closest to centroid C32, and partial centroid PC12 is closest to centroid C33. That is, for cluster CL31, two centroids C, centroid C32 and centroid C33, are the centroids C with the shortest distance and the largest number of partial centroids PC. Note that the above is just an example, and the information processing apparatus 100E may allocate each cluster to a blob using any method. Since the information processing device 100E clusters each cluster to generate partial clusters and clusters the centroids thereof to generate blobs, the relationship between blobs and clusters is necessarily known. Therefore, the information processing device 100E may assign each cluster to a blob based on information indicating the relationship between the blob and the cluster obtained by generating the blob.

In this way, in FIG. 51, since there are multiple candidate blobs in the cluster CL31, the information processing device 100E selects a blob BL whose centroid C is closest to the centroid CN corresponding to the cluster CL31 among the multiple candidate blobs. let In FIG. 51, the information processing apparatus 100E makes the cluster CL31 belong to the blob BL33 whose centroid C is closest to the centroid CN31 corresponding to the cluster CL31 among the blobs BL32 and BL33. In FIG. 51, the clustering results for other clusters CN32 to CN40, etc. are also similar to those in FIG. 50, so a detailed explanation will be omitted.

Note that when there is one candidate first group in one second group, the information processing device 100E may determine the first group to which one second group belongs to the candidate first group. Therefore, detailed explanation will be omitted.

[6-6-4. Information processing flow]
Next, the information processing procedure according to the sixth embodiment will be described using FIG. 52. FIG. 52 is a flowchart illustrating an example of information processing according to the sixth embodiment.

As shown in FIG. 52, the information processing apparatus 100E acquires object information indicating a plurality of objects that are data search targets (step S1201). For example, the information processing device 100E obtains object information indicating a plurality of objects from the object information storage unit 121.

The information processing device 100E performs a first classification process that classifies a plurality of objects into a plurality of first groups, and a plurality of second classification processes that classify a plurality of objects into a plurality of second groups that are different from the plurality of first groups. One of the second classification processes, which is a classification process for classifying into groups, is executed (step S1202). For example, the information processing device 100E executes a second classification process that is a classification process of classifying into a plurality of second groups.

The information processing device 100E performs one classification process and then performs the other classification process using the result of one classification process, thereby obtaining first group information indicating a plurality of first groups into which a plurality of objects are classified, and a plurality of first group information. second group information indicating a plurality of second groups for classifying the objects (step S1203). For example, the information processing device 100E generates first group information and second group information by performing a first classification process, which is the other classification process, after performing two classification processes.

[7. effect〕
As described above, the information processing apparatus 100E according to the sixth embodiment includes the acquisition section 131 and the generation section 132E. The acquisition unit 131 acquires object information indicating a plurality of objects that are data search targets. Based on the object information acquired by the acquisition unit 131, the generation unit 132E generates first group information indicating a plurality of first groups, which are a first number of groups into which a plurality of objects are classified, and a plurality of first groups. generates second group information indicating a plurality of second groups, which are groups of different classifications and are a second number of groups that is larger than the first number for classifying the plurality of objects.

In this way, the information processing apparatus 100E according to the sixth embodiment has a first number of first groups for classifying a plurality of objects, and a group that is classified differently from the plurality of first groups, It is possible to generate two types of groups, the second group having the second number, which is larger than the number, and the group for classifying objects.

The generation unit 132E generates first group information that is a group that classifies objects belonging to the same second group among the plurality of second groups into the same first group.

In this way, the information processing apparatus 100E can generate two types of groups that have an inclusive relationship, and can generate groups that classify objects.

The generation unit 132E generates first group information indicating a plurality of first groups in which objects belonging to each of two or more second groups among the plurality of second groups are classified into one first group.

In this way, the information processing apparatus 100E can generate two types of groups that have an inclusive relationship, and can generate groups that classify objects.

The generation unit 132E generates first group information indicating a plurality of first groups used for performing batch processing in a search process targeting a plurality of objects.

In this way, the information processing device 100E can generate two types of groups including a plurality of first groups used for performing batch processing in a search process targeting a plurality of objects, and can classify objects. Groups can be created.

The generation unit 132E generates first group information indicating a plurality of first groups used to collectively calculate distances regarding a plurality of objects.

In this way, the information processing apparatus 100E can generate two types of groups including the first group used to collectively calculate distances, and can generate groups for classifying objects.

The generation unit 132E generates first group information indicating a plurality of first groups used to calculate the distance between the search query and the object used in the search process in parallel.

In this way, the information processing device 100E can generate two types of groups including the first group used for parallel processing of calculating the distance between the search query and the object used in the search process, and It is possible to generate groups for classification.

The generation unit 132E generates second group information indicating a plurality of second groups used to perform processing related to quantization targeting a plurality of objects.

In this way, the information processing device 100E can generate two types of groups including the second group used to perform processing related to quantization for a plurality of objects, and can generate a group for classifying objects. can do.

The generation unit 132E generates second group information indicating a plurality of second groups that are units of quantization.

In this way, the information processing apparatus 100E can generate two types of groups including the second group that is the unit of quantization, and can generate groups for classifying objects.

The generation unit 132E generates second group information indicating a plurality of second groups used to quantize vectors corresponding to each of the plurality of objects.

In this way, the information processing device 100E can generate two types of groups including the second group used to quantize vectors corresponding to each of a plurality of objects, and generate a group for classifying objects. can do.

After classifying the plurality of second groups, the generation unit 132E performs classification of the plurality of first groups.

In this way, the information processing apparatus 100E can generate groups for classifying objects by performing the process of generating group information in the order of the second group classification and the first group classification.

The generation unit 132E generates second group information indicating a plurality of second groups by clustering a plurality of objects.

In this way, the information processing apparatus 100E can generate groups into which objects are classified by classifying the plurality of second groups by clustering the plurality of objects.

The acquisition unit 131 acquires index information indicating an index that searches for a plurality of objects. The generation unit 132E uses the index information to generate second group information indicating a plurality of second groups.

In this way, the information processing apparatus 100E can generate groups for classifying objects by classifying the plurality of second groups using the index.

The generation unit 132E generates first group information indicating a plurality of first groups using second group information indicating a second group.

In this way, the information processing device 100E can classify the first group based on the classification of the second group by generating first group information indicating a plurality of first groups using the second group information. , it is possible to generate groups to classify objects.

The generation unit 132E generates first group information indicating a plurality of first groups by clustering a plurality of second groups.

In this way, the information processing device 100E can classify the first group based on the classification of the second group by clustering the second group and generating first group information indicating a plurality of first groups, Groups can be created to categorize objects.

After classifying the plurality of first groups, the generation unit 132E performs classification of the plurality of second groups.

In this way, the information processing apparatus 100E can generate groups for classifying objects by performing a process of generating group information in the order of first group classification and second group classification.

The generation unit 132E generates first group information indicating a plurality of first groups by clustering a plurality of objects.

In this way, the information processing apparatus 100E can generate groups into which objects are classified by classifying the plurality of first groups by clustering the plurality of objects.

The generation unit 132E generates second group information indicating a plurality of second groups using first group information indicating the first group.

In this way, the information processing device 100E can classify the second group based on the classification of the first group by generating second group information indicating a plurality of second groups using the first group information. , it is possible to generate groups to classify objects.

The generating unit 132E generates second group information indicating the plurality of second groups by dividing at least one first group among the plurality of first groups and generating two or more second groups.

In this way, the information processing device 100E can classify the second group based on the classification of the first group by dividing the first group and generating second group information indicating a plurality of second groups. Groups can be created to categorize objects.

The generation unit 132E generates second group information indicating a plurality of second groups by clustering objects belonging to each first group and dividing each first group into two or more second groups.

In this way, the information processing device 100E clusters objects belonging to each first group, divides each first group into two or more second groups, and generates second group information indicating a plurality of second groups. As a result, the second group can be classified based on the first group classification, and groups for classifying objects can be generated.

[8. Hardware configuration]
The information processing apparatuses 100, 100A, 100B, 100C, 100D, and 100E according to each of the embodiments described above are realized by, for example, a computer 1000 having a configuration as shown in FIG. 53. FIG. 53 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface (I/F) 1500, an input/output interface (I/F) 1600, and a media interface (I/F). ) has 1700.

CPU 1100 operates based on a program stored in ROM 1300 or HDD 1400, and controls each section. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, programs depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. Communication interface 1500 receives data from other devices via network N and sends it to CPU 1100, and sends data generated by CPU 1100 to other devices via network N.

The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and mouse via an input/output interface 1600. CPU 1100 obtains data from an input device via input/output interface 1600. Further, CPU 1100 outputs the generated data to an output device via input/output interface 1600.

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1200. CPU 1100 loads this program from recording medium 1800 onto RAM 1200 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the information processing device 100 according to the first embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the RAM 1200. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from another device via the network N.

As mentioned above, some embodiments of the present application have been described in detail based on the drawings, but these are merely examples, and various modifications and variations can be made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure section of the invention. It is possible to carry out the invention in other forms with modifications.

[9. others〕
Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of the process can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured.

Furthermore, the processes described in the embodiments described above can be combined as appropriate within a range that does not conflict with the process contents.

Further, the above-mentioned "section, module, unit" can be read as "means", "circuit", etc. For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

1 Information processing system 10 Terminal device 50 Information providing device 100, 100A, 100B, 100C, 100D, 100E Information processing device 120, 120A, 120B, 120C Storage unit 121 Object information storage unit 122 Graph information storage unit 123, 123A Quantization information Storage unit 124 Codebook information storage unit 125, 125A Blob information storage unit 126, 126A Blob connected graph information storage unit 130, 130A, 130B, 130C, 130D, 130E Control unit 131 Acquisition unit 132, 132A, 132B, 132C, 132E Generation Department (processing department)
133, 133A, 133B, 133C, 133D, 133E Search processing unit 134 Providing unit

Claims (21)

an acquisition unit that acquires object information indicating multiple objects to be searched for;
First group information indicating a plurality of first groups that are a first number of groups for classifying the plurality of objects based on the object information acquired by the acquisition unit is different from the plurality of first groups. a generation unit that generates second group information indicating a plurality of second groups, which are groups that are classified and are a second number of groups that are larger than the first number of groups that classify the plurality of objects;
An information processing device comprising: The generation unit is
The first group information is generated that is a group that classifies objects belonging to the same second group among the plurality of second groups into the same first group among the plurality of objects. information processing equipment. The generation unit is
Generating the first group information indicating the plurality of first groups in which a group of objects belonging to each of two or more second groups among the plurality of second groups is classified into one first group. The information processing device according to claim 1. The generation unit is
The information processing apparatus according to claim 1, wherein the information processing apparatus generates the first group information indicating the plurality of first groups used for performing batch processing in a search process targeting the plurality of objects. The generation unit is
The information processing apparatus according to claim 4, wherein the information processing apparatus generates the first group information indicating the plurality of first groups used to collectively calculate distances regarding the plurality of objects. The generation unit is
5. The method according to claim 4, further comprising: generating the first group information indicating the plurality of first groups used for parallel processing calculation of a distance between a search query and an object used in the search process. Information processing device. The generation unit is
The information processing apparatus according to claim 1, wherein the information processing apparatus generates the second group information indicating the plurality of second groups used to perform processing related to quantization for the plurality of objects. The generation unit is
The information processing apparatus according to claim 7, wherein the information processing apparatus generates the second group information indicating the plurality of second groups that are the units of the quantization. The generation unit is
The information processing apparatus according to claim 7, wherein the second group information indicating the plurality of second groups used for quantizing the vectors corresponding to each of the plurality of objects is generated. The generation unit is
The information processing apparatus according to claim 1, wherein the plurality of first groups are classified after the plurality of second groups are classified. The generation unit is
The information processing apparatus according to claim 10, wherein the second group information indicating the plurality of second groups is generated by clustering the plurality of objects. The acquisition unit includes:
obtaining index information indicating an index for searching the plurality of objects;
The generation unit is
The information processing apparatus according to claim 10, wherein the second group information indicating the plurality of second groups is generated using the index information. The generation unit is
The information processing apparatus according to claim 10, wherein the first group information indicating the plurality of first groups is generated using the second group information indicating the second group. The generation unit is
The information processing apparatus according to claim 10, wherein the first group information indicating the plurality of first groups is generated by clustering the plurality of second groups. The generation unit is
The information processing apparatus according to claim 10, wherein the plurality of second groups are classified after the plurality of first groups are classified. The generation unit is
The information processing apparatus according to claim 15, wherein the first group information indicating the plurality of first groups is generated by clustering the plurality of objects. The generation unit is
The information processing apparatus according to claim 15, wherein the second group information indicating the plurality of second groups is generated using the first group information indicating the first group. The generation unit is
The second group information indicating the plurality of second groups is generated by dividing at least one first group among the plurality of first groups and generating two or more second groups. The information processing device according to claim 15. The generation unit is
The second group information indicating the plurality of second groups is generated by clustering objects belonging to each first group and dividing each first group into two or more second groups. The information processing device according to claim 15. An information processing method performed by a computer, the method comprising:
an acquisition step of acquiring object information indicating a plurality of objects to be searched for;
First group information indicating a plurality of first groups that are a first number of groups for classifying the plurality of objects based on the object information acquired in the acquisition step is different from the plurality of first groups. a generation step of generating second group information indicating a plurality of second groups, which are groups to be classified and are a second number of groups larger than the first number for classifying the plurality of objects;
An information processing method characterized by comprising: an acquisition procedure for acquiring object information indicating multiple objects to be searched for;
First group information indicating a plurality of first groups that are a first number of groups for classifying the plurality of objects based on the object information acquired by the acquisition procedure is different from the plurality of first groups. a generation procedure for generating second group information indicating a plurality of second groups, which are groups that are classified and are a second number of groups that are larger than the first number of groups that classify the plurality of objects;
An information processing program that causes a computer to execute.

Images