JPWO2013153725A1 - Data search device, data search method, and data search program - Google Patents

Abstract

The spatial mapping generation unit 81 generates a spatial attribute mapping that maps the data to be searched in a space where similarity with other data can be determined. The viewpoint map generation unit 82 generates a viewpoint attribute map that maps one or more pieces of given data under a totally ordered set. The space attribute information generating unit 83 generates space attribute information corresponding to the data by converting the search target data using the space attribute mapping. The subset generation unit 84 divides the set of spatial attribute information according to the allowable neighborhood range criterion, and creates a set of search target data corresponding to each of the divided spatial attribute information as a subset of the search target data. The mapping unit 85 maps each subset to a fully ordered set using the viewpoint attribute mapping.

Description

  The present invention relates to a data extraction apparatus, a data search method, and a data search program for extracting a subset of data of high interest from a data group to be searched.

  Nearest neighbor search is known as a method for searching for data close to the characteristics of data of interest. This is a method of extracting a plurality of pieces of data having feature vectors similar to the feature vectors used as search (query) keys, such as image data represented by feature vectors.

  In particular, as a method for searching for a plurality (K) of similar vectors, a KNN (K-Nearest Neighbor) search or the like is known. Further, R-tree using a tree structure, ANN (Applied Nearest Neighbor), and the like are also used as a search method.

  In a method such as R-tree, a spatial index in a feature vector space is created in advance in a tree structure, and data in which the distance between feature vectors is within a specified range is searched. This tree structure is created for the purpose of improving the search speed.

  ANN is a method of further speeding up the search process by performing an approximate search. In addition, LSH (Locality Sensitive Hashing) has been proposed for the purpose of effective search processing even for high-dimensional feature vectors (see Non-Patent Document 1).

  Patent Document 1 describes a search device that searches for high-dimensional feature vectors. The search device described in Patent Document 1 does not index feature vectors themselves, but uses a quantization code obtained by quantizing feature vectors, thereby reducing the memory usage rate.

JP 2011-257970 A

Datar, M., Immorlica, N., Indyk, P., and Mirrokni, V., "Locality-Sensitive Hashing Scheme Based on p-Stable Distributions", SCG '04 Proceedings of the ACM Symposium on Computational Geometry, ACM Press, p.253-262, 2004

  In the method described above, it is implicitly assumed that the data to be searched (i.e., data of high interest) is represented in a feature vector space that is considered to determine the similarity between the data. That is, in the method described above, the data to be searched is expressed as one point in the feature vector space (that is, the searcher's interest is expressed by the feature vector). Therefore, in the method described above, there is a problem of how to efficiently search for the vicinity of a point expressed in space.

  However, for example, when a certain feature vector space is created, it is assumed that data that is distributed (distributed) in an “enclave” manner is searched in the space. In this case, in the method described above, it is necessary to create a complicated query for the search.

  For example, if the query to be executed is a query that searches for “data having feature A or feature B” in the same feature vector space, each query may be merged after issuing a plurality of queries independently. . However, in the first place, there are cases where the feature A and the feature B should not be considered in the same space as the feature vector. In this case, it becomes very difficult to create the query itself, and it is also necessary to recreate the feature vector space itself.

  In general, the work of structuring the index of the feature vector space has a heavy calculation load. Therefore, when performing a free search flexibly and interactively, it is not preferable to reconfigure the feature vector space too frequently.

  The above problem occurs when there are a plurality of viewpoints for judging similarity. The above problem occurs when the viewpoint of similarity of data is different from the viewpoint representing the level of interest in data. Here, the similarity of data is an amount used for limiting a condition in data search and used for determining whether data existing in the vicinity can be regarded as approximately the same data. The high interest in data is an amount used to determine what kind of data the searching user is interested in.

  For example, a case is assumed in which “an area where similar text information is as dense as possible” is searched for text data to which position information is added. In this case, when determining the similarity of data, it is necessary to consider not only the similarity of text information but also the positional similarity.

  In order to express this in a single feature vector space using a general method, it is necessary to create a feature vector space including both text information and position information. When considering only text information, it is necessary to search for data groups that are similar in position information among data existing at a short distance (that is, similar) according to the conditions of the permitted range. Further, when the process for performing this search is created in a format used in general neighborhood search, it is very complicated to create the process.

  Furthermore, if the feature of interest is a region where similar text information is concentrated, data that is only concentrated as similar text information may be used. However, the viewpoint may be changed, and it may be assumed that it is not just the similarity of text information, but that it is desired to make a selection based on a more complicated viewpoint. In this case, it may be necessary to recreate the feature vector space itself.

  As described above, in the data search process, there are cases where a plurality of similar viewpoints exist, and in each data search process, the viewpoints indicating a plurality of similarities to be considered may be used differently. When the neighborhood search processing in the feature vector space is performed by a general method, there are problems that the query becomes complicated or that restructuring for data search is necessary. For this reason, it is desirable that data search can be performed at high speed and flexibly even when a complicated search is performed based on a viewpoint showing a plurality of similarities.

  Therefore, the present invention provides a data search apparatus, a data search method, and a data search method that can perform data search at high speed and flexibly even when searching based on a viewpoint that shows a plurality of similarities between data. The purpose is to provide a program.

  The data search apparatus according to the present invention includes a proximity criterion used as a criterion for determining similarity of data to be searched, an allowable neighborhood range criterion indicating a range in which the data to be searched are similar, and a viewpoint criterion indicating a search viewpoint. Spatial mapping generation that generates spatial attribute mapping that maps data to be searched on a space where similarity with other data can be determined based on the neighborhood criteria and allowable neighborhood range criteria in response to a query input including Means for generating viewpoint attribute mapping for mapping one or more pieces of given data under a total ordered set based on the viewpoint criterion in response to an input of the query, and the spatial attribute mapping The space attribute information generating means for generating the space attribute information corresponding to the data by converting the data to be searched using, and the space according to the allowable neighborhood range criterion A subset generation means for dividing a set of sex information and creating a set of search object data corresponding to each divided space attribute information as a subset of the search object data, and using the viewpoint attribute mapping, A mapping means for mapping a subset to a fully ordered set and a search result for identifying each subset corresponding to the mapped total ordered set according to the order determined based on the element And a specifying means.

  The data search method according to the present invention includes a proximity criterion used as a criterion for determining the similarity of data to be searched, an allowable neighborhood range criterion indicating a range in which the data to be searched are similar, and a viewpoint criterion indicating a viewpoint for searching. In response to an input of a query including a spatial attribute mapping that maps data to be searched on a space where similarity with other data can be determined based on the neighborhood criterion and the allowable neighborhood range criterion, In response to the input, generates a viewpoint attribute map that maps one or more pieces of given data under a total ordered set based on the viewpoint criteria, and converts the search target data using the spatial attribute map Thus, spatial attribute information corresponding to the data is generated, the set of spatial attribute information is divided according to the allowable neighborhood range criterion, and a search pair corresponding to each divided spatial attribute information is generated. A set of data is created as a subset of search target data, and each of the subsets is mapped to a fully ordered set using the viewpoint attribute mapping, and each portion corresponding to the mapped totally ordered set A search result is specified according to the order in which the set is determined based on the element.

  The data search program according to the present invention provides a computer with a viewpoint of searching for a vicinity reference used as a reference for determining similarity of data to be searched, an allowable vicinity range reference indicating a range in which the data to be searched is similar, and Generates a spatial attribute mapping that maps the data to be searched in a space where similarity with other data can be determined based on the neighborhood criteria and allowable neighborhood range criteria in response to a query input including the viewpoint criteria shown A spatial mapping generation process that generates a viewpoint attribute mapping that maps one or more pieces of given data under a total ordered set based on the viewpoint criterion in response to an input of the query, the space A spatial attribute information generation process for generating spatial attribute information corresponding to the data by converting the data to be searched using the attribute mapping; The set of spatial attribute information is divided according to the above, and a subset generation process for creating a set of search target data corresponding to each of the divided spatial attribute information as a subset of the search target data, using the viewpoint attribute mapping, Mapping processing for mapping each subset to a fully ordered set, and specifying a search result for each subset corresponding to the mapped total ordered set element according to the order determined based on the element The search result specifying process is executed.

  According to the present invention, even when a search is performed based on a viewpoint showing a plurality of similarities between data, the data search can be performed at high speed and flexibly.

It is a block diagram which shows the structural example of 1st Embodiment of the data search device by this invention. It is explanatory drawing which shows the example of a data reconstruction process. It is a flowchart which shows the operation example of the data search device of 1st Embodiment. It is explanatory drawing which shows the example of a space attribute map and a viewpoint attribute map. It is explanatory drawing which shows the example of mapping. It is a block diagram which shows the structural example of 2nd Embodiment of the data search device by this invention. It is a block diagram which shows the structural example of the 1st Example of the data search device by this invention. It is explanatory drawing which shows the example which divided | segmented two-dimensional space. It is explanatory drawing which shows the example which performs a data search via a communication network. It is a block diagram which shows the structural example of the 3rd Example of the data search device by invention. It is a block diagram which shows the outline | summary of the data search device by this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1. FIG.
FIG. 1 is a block diagram showing a configuration example of a first embodiment of a data search apparatus according to the present invention. The data search device of this embodiment includes a data registration unit 110 and a data search unit 120. The data search apparatus is an apparatus that outputs output subset data 103 that is a subset of data 101 input (registered) in advance when a query 102 according to a user request is input.

  Data 101 is data to be searched, and is input data registered and updated by a user in advance or sequentially. The format of the data 101 is arbitrary. The data 101 is typically represented as data having a plurality of arbitrary attributes for one data entry, as represented by a relational database. In the present embodiment, this attribute is designated to perform data search.

  The query 102 is a search condition that a searcher (hereinafter referred to as a user) inputs to the data search apparatus according to the present invention in order to acquire data of interest. The query 102 includes at least a neighborhood criterion used as a criterion for determining the similarity of data, an allowable neighborhood range criterion indicating a range in which search target data is similar, and a user's interest (that is, a search viewpoint). Including viewpoint criteria. It can be said that the allowable neighborhood range criterion is information defining a range of data that can be identified.

  Specifically, the neighborhood criterion is a criterion indicating the similarity of a plurality of given data. Desirably, the neighborhood criterion is a criterion by which the similarity of data can be determined based on the relationship of the distance in the space by mapping the data to the space in which the distance is defined. This distance can take various forms such as Euclidean distance and Hamming distance.

  Further, even if the neighborhood criterion is non-quantitative data such as category data, the neighborhood criterion may be handled by defining a distance relationship using, for example, a multidimensional scaling method.

  Specifically, the allowable neighborhood range criterion is a range parameter that designates a range of data handled as one data group having a feature with very high similarity. For example, in a D-dimensional space in which the distance of data to be searched can be represented by a distance metric, the allowable neighborhood range criterion is represented by a D-dimensional sphere designated with an appropriate radius.

  Specifically, the viewpoint criterion indicates which data group has a higher interest when comparing data groups belonging to areas that can be judged to be highly similar (hereinafter referred to as neighborhood areas). It is a parameter to specify. The viewpoint criterion may be, for example, information specifying which data set the searcher prefers when comparing two arbitrary data sets. This parameter can be used to map an arbitrary data group to the base of a totally ordered set.

  Here, the total ordered set can define binary relations ≦ and = (equivalent) for any element a, b, c, and further, the reflection rule (a ≦ a), the transition rule (a ≦ b) And if b ≦ c, a ≦ c), an asymmetric rule (a = b if a ≦ b and b ≦ a), and a complete set (all original comparisons are possible).

  The output subset data 103 is a set of a subset of all data output for the query 102 input by the user.

  The data registration unit 110 includes a database management unit 111 and a database 112. The database 112 stores the attributes included in the data 101 so that they can be identified. For example, the database 112 may manage the attributes of the data 101 in a table format, may be managed in a key-value format, or may be managed in a graph format.

  The database management unit 111 manages the database 112. Specifically, when the data 101 is input, the database management unit 111 registers the input data in the database 112. Further, the database management unit 111 performs necessary processing in response to a data update or data acquisition request.

  The data registration unit 110 is typically realized by an RDBMS (relational database management system) represented by a database system constructed by MYSQL (registered trademark). The data registration unit 110 may be realized using a NoSQL database such as Hadoop, Cassandra, and Graph database, for example. When the data size is not large, the data registration unit 110 may be realized by an array management system developed on a memory or the like.

  The data search unit 120 includes a query analysis unit 121, a data reconstruction unit 122, a reconstruction data management unit 123, a reconstruction data storage unit 124, and a sampling unit 125.

  The query analysis unit 121 analyzes the query 102 and transmits information necessary for the reconstruction process of the database 112 to the data reconstruction unit 122. Note that the sampling unit 125, which will be described later, samples data that matches the query 102 from the reconstructed data based on the viewpoint criteria. Therefore, the query analysis unit 121 transmits information necessary for the processing based on the viewpoint criterion to the sampling unit 125. Details of the information transmitted by the query analysis unit 121 to each unit will be described later.

  The data reconstruction unit 122 reconstructs data included in the database 112 according to the parameters input from the query analysis unit 121. Specifically, the data reconstruction unit 122 converts the original data using the parameters input from the query analysis unit 121, and adds the converted information to the base data. Hereinafter, information obtained by converting the original data is referred to as space attribute portion or space attribute information. That is, the reconstructed data is data in which the space attribute portion extracted from the original data is added to the original data.

  FIG. 2 is an explanatory diagram illustrating an example of the data reconstruction process. The example shown in FIG. 2 shows that a space attribute part is added to data stored in the relational database format. A method for the data reconstruction unit 122 to reconstruct data will be described later.

  The reconstructed data storage unit 124 stores the data reconstructed by the data reconstructing unit 122 (that is, the original data and the data including the space attribute part).

  The reconfiguration data management unit 123 manages the reconfiguration data storage unit 124. Specifically, the reconfiguration data management unit 123 performs processing to save, update, or acquire the reconfigured data in the reconfiguration data storage unit 124 in response to a request. Similar to the database 112, the reconfiguration data storage unit 124 is typically realized by an RDBMS. However, the method of realizing the reconfiguration data storage unit 124 is not limited to RDBMS. The reconstruction data storage unit 124 is realized by, for example, a magnetic disk.

  The sampling unit 125 extracts the output subset data 103 from the reconstructed data using the information received from the query analysis unit 121. Specifically, the sampling unit 125 receives from the query analysis unit 121 information used for division processing (hereinafter referred to as division information) and information used for viewpoint evaluation. In addition, the sampling unit 125 acquires reconstruction data from the reconstruction data management unit 123.

  The sampling unit 125 extracts the output subset data 103 based on these pieces of information. A specific method by which the sampling unit 125 extracts the output subset data 103 will be described later.

  The query analysis unit 121, the data reconstruction unit 122, the reconstruction data management unit 123, and the sampling unit 125 are realized by a CPU of a computer that operates according to a program (data search program). For example, the program is stored in a storage unit (not shown) of the data search apparatus, and the CPU reads the program, and in accordance with the program, the query analysis unit 121, the data reconstruction unit 122, the reconstruction data management unit 123, and The sampling unit 125 may be operated.

  Further, the query analysis unit 121, the data reconstruction unit 122, the reconstruction data management unit 123, and the sampling unit 125 may each be realized by dedicated hardware.

  Next, the operation of the data search apparatus of this embodiment will be described. FIG. 3 is a flowchart illustrating an operation example of the data search apparatus according to the first embodiment. In the example shown in FIG. 3, the data search is started when the user inputs a query.

  First, the user inputs a query necessary for search (step S301). The query includes a neighborhood criterion, an allowable neighborhood range criterion, and a viewpoint criterion for the data stored in the database 112 illustrated in FIG.

  The query analysis unit 121 analyzes the received query (step S302), and based on the neighborhood criterion, the allowable neighborhood criterion, and the viewpoint criterion, the spatial attribute mapping f: E → Φ, the viewpoint attribute mapping g: E → V, and Create split information. Here, E is a data set having one data in the database 112 as a source e. Φ indicates a space in which similarity is defined. V indicates a totally ordered set.

  The query analysis unit 121 transmits information indicating the spatial attribute mapping f to the data reconstruction unit 122 (step S303). Further, the query analysis unit 121 transmits information indicating the division information and the viewpoint attribute mapping g to the sampling unit 125 (step S304).

  The processing of the query analysis unit 121 will be further described. The query analysis unit 121 creates the spatial attribute map f from the neighborhood criterion and the allowable neighborhood range criterion. That is, information necessary for creating the spatial attribute map f is specified in the neighborhood criterion and the allowable neighborhood range criterion. The spatial attribute map f converts attribute information of data to be searched. This conversion is performed for the purpose of quantitatively evaluating the similarity between the data using the attribute information of the data.

  Specifically, in the query, as the details of the spatial attribute mapping, which attribute is used and which conversion is applied among the attribute information of the data is described. The query also describes a similarity index defined in the mapping destination space. As such a conversion method, for example, a coordinate system of a mapping destination and a conversion formula to the coordinate system are designated. In this case, a plurality of pieces of attribute information having appropriate numerical values may be used as vectors, and an appropriate conversion formula and parameter values may be designated as vector conversion methods. In this case, the allowable neighborhood range criterion is mainly used to determine the value of the parameter.

  For example, an arithmetic expression for designating vector data with designated attribute information and performing an appropriate algebraic operation on the vector data is designated. Further, a coefficient matrix used as a parameter of the arithmetic expression is determined from the specified allowable neighborhood range criterion. Since the spatial attribute mapping is uniquely determined by these pieces of information, the data to be searched can be mapped by this spatial attribute mapping. For the mapped data, for example, the similarity between the data is evaluated using the Euclidean distance in the mapped space.

  This space attribute map f is generally defined as a map that converts a data set E including a plurality of data into a set Φ of space attributes φ. However, this spatial attribute map f may be defined as a map φ = f (e) for converting a single data e into a single spatial attribute φ.

  In other words, this space attribute map f can be said to be a map that defines a method for converting a data attribute into a metric space in which a quantitative index capable of determining the similarity between arbitrary data is defined.

  For example, the query analysis unit 121 prepares a plurality of options in advance as a mapping creation method, and creates a spatial attribute map f based on designation of keyword information indicating which mapping is selected and a parameter value determination method. May be. Alternatively, the administrator may implement mapping as a method in advance, and the query analysis unit 121 may create the spatial attribute mapping f in accordance with designation of pointer information that allows selection of a corresponding method.

  The viewpoint attribute mapping g is a mapping that maps a given data set to the base of a totally ordered set based on viewpoint criteria. In other words, the viewpoint attribute map g can be said to be a map that defines a method for converting data attributes into a totally ordered set. The query analysis unit 121 may create the viewpoint attribute map g by a method similar to the method of creating the space attribute map f.

  The division information is information describing a method of dividing the space mapped by the space attribute map f. The query analysis unit 121 creates division information using information specified in the neighborhood criterion and the allowable neighborhood range criterion. The query analysis unit 121 may create the division information by a method similar to the method of creating the space attribute map f. In addition, when the division process can be performed based on the allowable neighborhood range criterion described later, the query analysis unit 121 does not have to create the division information.

  FIG. 4 is an explanatory diagram showing an example of the space attribute map f and the viewpoint attribute map g. In the example shown in FIG. 4, it is shown that the data sets E1 and E2 each include four data (e1, e2, e3, e4).

  The spatial attribute map f illustrated in FIG. 4 is a map for converting the data set E into a spatial attribute set Φ. From the four data (e1, e2, e3, e4), four spatial attributes φ (φ1, φ2). , Φ3, φ4).

  Also, the viewpoint attribute map g illustrated in FIG. 4 is a map for converting the data set E into a totally ordered set V, and v satisfying the properties of the totally ordered set from four data (e1, e2, e3, e4). Indicates that it has been converted to.

  When the data reconstruction unit 122 receives the information of the spatial attribute map f, the data reconstruction unit 122 calculates φ∈Φ (spatial attribute Φ = f (E)) corresponding to all e∈E included in the data set E (FIG. 3). Step S305). Specifically, the data reconfiguration unit 122 requests all data stored in the database 112 from the database management unit 111. Then, the data reconstruction unit 122 calculates the spatial attribute information φi of each data ei belonging to the data group E of the acquired database 112.

  FIG. 5 is an explanatory diagram illustrating an example of mapping. The mapping result 501 illustrated in FIG. 5 shows an example in which the data reconstruction unit 122 maps the data in the database 112 to the two-dimensional Euclidean space.

  Then, the data reconfiguration unit 122 transmits (ei, φi) to the reconfiguration data management unit 123 and makes a request for saving to the reconfiguration data storage unit 124.

  The reconfiguration data management unit 123 stores the data information (ei illustrated in FIG. 5) and the spatial attribute information (φi illustrated in FIG. 5) received from the data reconfiguration unit 122 in the reconfiguration data storage unit 124. When the process is completed, a storage completion notification is sent to the data reconstruction unit 122.

  When the data reconfiguration unit 122 receives the storage end notification for all the reconfiguration data, the data reconfiguration unit 122 notifies the query analysis unit 121 of the data reconfiguration completion.

  Upon receiving the data reconstruction completion notification, the query analysis unit 121 notifies the sampling unit 125 of sampling start (step S306).

  The sampling unit 125 acquires reconstruction data from the reconstruction data management unit 123. Then, the sampling unit 125 divides the data information (ei illustrated in FIG. 5) of the reconstructed data into a set of subsets using the spatial attribute information and the allowable neighborhood range information of the reconstructed data (step S307). ). The sampling means 125 divides all data into each subset according to the conditions described in the allowable neighborhood range criterion. Note that, when the query analysis unit 121 creates division information, the sampling unit 125 may divide all data into each subset based on the division information. Each subset forms an allowed neighborhood area according to an allowed neighborhood range criterion.

  For the allowable neighborhood range criterion, for example, a parameter for designating a specific distance criterion is designated. In this case, the sampling unit 125 divides the space so that the Euclidean space distance between all data included in the subset is within a certain distance. In addition, for example, a parameter that designates a probabilistic distance criterion is designated as the allowable neighborhood range criterion. In this case, the sampling unit 125 divides the space based on the average distribution spread.

  However, the method by which the sampling unit 125 divides the data space is not limited to the above method. For example, when an appropriate probability distribution and its parameters are designated as the allowable neighborhood range criterion, the sampling unit 125 may divide the space according to the probability distribution. In other words, the sampling unit 125 may divide the data using another method as long as the space can be divided so that the subset of the reconstructed data is included in the range indicated by the allowable neighborhood range criterion.

  The mapping result 502 illustrated in FIG. 5 shows an example in which the sampling unit 125 divides the mapping result 501 into four neighboring regions (N1 to N4). For example, in the example shown in FIG. 5, the neighborhood area N1 includes three data, and these three data are a subset of the reconstructed data.

  The sampling means 125 calculates a total ordered set P = {pi | pi = g (Ei)} using the viewpoint attribute mapping g for each obtained subset Ei belonging to each neighboring region Ni (step S308). ). Since P is a fully ordered set, any element of P is comparable. Therefore, the sampling means 125 can perform sampling such as selecting in descending order such as the top Q.

  The mapping result 503 illustrated in FIG. 5 shows an example in which viewpoint evaluation values (v1 to v4) are calculated for each of four neighboring regions (N1 to N4). This set of viewpoint evaluation values is a totally ordered set.

  The sampling policy may be a predetermined static parameter. If the query 102 includes a sampling policy, the sampling unit 125 may perform sampling using the policy. In the sampling policy, for example, information such as “select top Q” and “number of upper limit data” is set. However, the content of the policy is not limited to the exemplified content. Other contents may be used as long as they specify the size of data actually sampled from the data stored in the original database.

  The sampling means 125 outputs the subset set Ψ = {Ψ1, Ψ2,..., ΨQ} obtained in this way as the output subset data 103, and the process ends.

  In the example illustrated in FIG. 5, v1 to v4 are evaluation values (real values) obtained by mapping the neighboring regions N1 to N4. Therefore, the sampling unit 125 may select a subset illustrated in FIG. 5 based on the evaluation value. Each element of the set Ψ corresponds to a set of data included in each neighboring area.

  Each subset included in the output subset data 103 obtained in this way is a set of data satisfying the concept of the allowable neighborhood range designated by the user through the query, and becomes data of high user interest. These data do not necessarily have to be similar to each other, and each data has desirable characteristics and is extracted as a grouped group of similar parts. It becomes possible.

  Further, as described above, in the present embodiment, the data reconstruction unit 122 and the sampling unit 125 can select the processing to be performed with the calculation amount of order N with respect to the total number of data N. It becomes possible.

  Because of this characteristic, even if the data that the user wants is not always clear, the data that is really necessary among the data on the database can be accessed at high speed. Specifically, the user first makes a hypothesis what kind of data he wants to acquire, inputs a query to the hypothesis, and analyzes the output subset data of the return value. Then, the user may verify the hypothesis and perform verification again with a new hypothesis corrected. In this embodiment, such a hypothesis verification loop can also be performed interactively.

  As described above, according to the present embodiment, the query analysis unit 121 is based on the neighborhood criterion and the allowable neighborhood range criterion according to the input of the query including the neighborhood criterion, the allowable neighborhood range criterion, and the viewpoint criterion. A space attribute map f is generated, and a viewpoint attribute map g is generated based on the viewpoint criteria. The data reconstruction unit 122 generates the spatial attribute information corresponding to the data by converting the search target data using the spatial attribute map f.

  The sampling unit 125 divides the space attribute information according to the allowable neighborhood range criterion, and creates a set of search target data corresponding to each divided space attribute information as a subset of the search target data. The sampling means 125 maps each subset to a total ordered set using the viewpoint attribute mapping, and the order in which each subset corresponding to the mapped total ordered set is determined based on the element. The search result is specified according to

  Therefore, even when a search is performed based on a viewpoint showing a plurality of similarities between data, the data search can be performed at high speed and flexibly.

  Specifically, according to this embodiment, even if there are a plurality of similarities between data and an attribute indicating a viewpoint at the time of search, it is possible to suppress the query from becoming complicated. Further, according to the present embodiment, data reconstruction is not performed. Therefore, it is possible to perform a highly flexible search.

  In addition, for example, in a general neighborhood search process, there is a problem in that data cannot be efficiently acquired if there is no prior knowledge that allows a searcher to clearly specify a desired data condition. This is because, when prior knowledge is scarce, it is difficult to specify appropriate conditions in advance, such as the restriction at the time of retrieval and the viewpoint of desired data. Therefore, it is not always optimally structured, and it is difficult to acquire desired data by changing the search condition interactively while viewing the data acquired according to the search condition. In particular, when the data size becomes enormous, this operation takes a very long time. As described above, in a general method, it has been difficult to interactively and quickly specify a place where a lot of necessary information is included from target data while repeating trial and error.

  However, according to the present embodiment, since it has the flexibility described above, it is possible to repeat setting of hypotheses and search processing (that is, trial and error). Since such an interactive approach can be performed, it is possible to access appropriate data even when there is little prior knowledge about search.

Embodiment 2. FIG.
FIG. 6 is a block diagram showing a configuration example of the second embodiment of the data search apparatus according to the present invention. In addition, about the structure similar to 1st Embodiment, the code | symbol same as FIG. 1 is attached | subjected and description is abbreviate | omitted. In the data search apparatus of this embodiment, the data search unit 120 of the first embodiment further includes a condition determination unit 601. The condition determination means 601 is also realized by a CPU of a computer that operates according to a program (data search program).

  The query 102 of this embodiment includes conditions (hereinafter, referred to as output conditions) used by the condition determination unit 601 for determination in addition to the vicinity criterion, the allowable vicinity range criterion, and the viewpoint criterion. The query analysis unit 121 extracts an output condition from the input query 102 and notifies the condition determination unit 601 of the output condition. For example, the upper limit number of data and the characteristics of the subset actually extracted are set as the output condition. Note that when the output condition is determined in advance, the query 102 may not include the output condition.

  The condition determination unit 601 receives an output condition from the query analysis unit 121. The condition determining unit 601 checks whether the output result from the sampling unit 125 satisfies the output condition before actually outputting.

  When the output result satisfies the output condition, the condition determination unit 601 outputs the output result as it is as the output subset data 103. On the other hand, when the output result does not satisfy the output condition, any or all of the spatial attribute mapping, the viewpoint attribute mapping, and the allowable neighborhood range criterion are updated. By updating these, conditions can be relaxed or restricted.

  For example, if the upper limit number of data is not reached, the condition may be relaxed and adjusted so that more data can be obtained. Specifically, the condition determination unit 601 may relax the condition by changing the spatial attribute mapping so that more data can be obtained. In addition, the condition determining unit 601 may relax the condition by changing the method by which the sampling unit 125 divides the space so that a larger number of data can be obtained.

  Such a feedback mechanism may be realized by changing a parameter in a predetermined method, and by adopting a change corrected in a preferable direction by a parameter change that introduces a random process such as a genetic algorithm. It may be realized.

  With the above configuration, a search result more suitable for the user's requirements can be output.

  Hereinafter, the present invention will be described with reference to specific examples, but the scope of the present invention is not limited to the contents described below.

  FIG. 7 is a block diagram showing a configuration example of this embodiment of the data search apparatus according to the present invention. The data search apparatus according to this embodiment includes a data registration unit 710 and a data search unit 720. The data registration unit 710 and the data search unit 720 of this example correspond to the data registration unit 110 and the data search unit 120 of the first embodiment (or the second embodiment), respectively.

  The data registration unit 710 includes a database management unit 711 and a database 712. The database management unit 711 and the database 712 correspond to the database management unit 111 and the database 112 of the first embodiment, respectively.

  The data search unit 720 includes a query analysis unit 721, a data reconstruction unit 722, a reconstruction data management unit 723, a reconstruction data storage unit 724, a clustering unit 725, a cluster selection unit 726, and a data shaping unit. 727.

  The query analysis unit 721, the data reconstruction unit 722, the reconstruction data management unit 723, and the reconstruction data storage unit 724 are the query analysis unit 121, the data reconstruction unit 122, and the reconstruction unit 724 of the first embodiment. It corresponds to the configuration data management means 123 and the reconfiguration data storage means 124, respectively.

  The clustering unit 725 and the cluster selection unit 726 correspond to the sampling method 125 of the first embodiment.

  In this embodiment, it is assumed that the data 701 includes a vector x on the D-dimensional Euclidean space, time t, appropriate category data u, and text information q. The vector x is, for example, a vector composed of coordinate values of a place where the user with the user ID u created the text information q at a certain time t. Here, when a place is defined by latitude and longitude, the place is expressed as a two-dimensional coordinate, so x is a two-dimensional vector.

  In this embodiment, information indicating a data group that the user wants to collect is described in the query 702. Here, the query 702 is “text data including a certain word w as much as possible among text data distributed in a region having an average spatial extent of about 1 km and created for about one hour. I want to collect a group of data created for a simple user.

  This query is assumed to be created in order to increase the reliability of information in a situation where an action is performed to find a place where an event deeply related to the word w is performed. Specifically, in consideration of real-time characteristics, it is assumed that "information is assumed to have spread to users in the vicinity of the event in about an hour, and information is emitted from multiple sources as much as possible rather than by a limited number of limited users. If the user wants to search for "", the above query is prepared.

  The description format of the query 702 is arbitrary. For example, an API (Application Program Interface) using an abstract class that defines an interface function may be provided to the user application side. In this case, the user may implement an application using this API, and generate the query 702 using the application.

  Further, input parameters may be defined on the data search device side, and the query 702 may be generated according to the parameters input by the user. Note that the user may select a parameter from preset values, may input a value as a parameter, or may specify a value in a predetermined range as a parameter. In this embodiment, a method in which a user designates input parameters will be described as an example.

  In the present embodiment, it is assumed that the allowable neighborhood range criterion indicates an average spatial extent of about 1 km. In addition, it is assumed that a subset is created by dividing the space by randomization. A method of dividing space and time based on a fixed grid is also conceivable. However, when such a method is used, it may be desirable depending on the division method. Therefore, it is more preferable to divide the space by randomization. Further, other division methods may be used as necessary.

  Note that the division method may be predetermined in the data search apparatus, or may be specified in detail in the query 702. In the present embodiment, it is assumed that a randomization parameter defined in the data search apparatus is used by specifying “random with an average width of 1 km and a time width of 1 hour” in the query 702.

  Further, it is assumed that the neighborhood criterion is designated as “similarity evaluation by linear transformation of time and place including the word w in the text”. This neighborhood criterion assumes that texts that do not include w have infinite dissimilarity, and that texts that include w are considered the same from the viewpoint of text. In addition, this neighborhood criterion further assumes that a spatial attribute map is generated by linear transformation in consideration of temporal and locational distances.

  Further, it is assumed that “user diversity” is specified in the viewpoint standard.

  When the query 702 is input, the query analysis unit 721 determines the spatial attribute mapping, the division information, and the viewpoint attribute mapping. Hereinafter, these determination methods will be described.

<Spatial attribute mapping>
For convenience, it is assumed that the word w is included in the text t of some data e. This is expressed as t. First, the query analysis means 721 starts with a set Ew (Ew = {e∈E | w∈et}) consisting only of data containing the word w in the text t from the set E of all data stored in the database 712. A mapping θ: E → E is defined as Ew = θ (E).

Furthermore, the query analyzing unit 721, the Ew, conversion to 3-dimensional Euclidean space ^{R 3} obtained by synthesizing the spatial coordinates X∈R ² time coordinate t∈R σ: exemplified as Ew → ^{R 3,} in formula 1 below Define the mapping σ to be. Here, R represents a real number field.

  φ = σ (e) = Ay (Formula 1)

  Y in Equation 1 is a three-dimensional vector obtained by combining x and t. A in Formula 1 is a matrix generated based on the allowable neighborhood range criterion, and is a matrix having coefficients for normalizing 1 km and 1 hour to unit length 1 as diagonal components at corresponding positions. .

  In the present embodiment, since the values obtained by scaling the time and place are used as they are, the calculation can be performed simply by extracting the time and place components from the result of the mapping σ (e) for each element e. More generally, for example, linear transformation is performed using a plurality of attributes of a single data e, or a plurality of spatial attribute information groups are calculated for a plurality of data groups. It is also possible to define a mapping.

  Hereinafter, for all the elements of Ew, the mapping for performing the conversion shown in the above equation 1 is denoted as Φ = σ (Ew) for convenience. However, Φ = {φ = σ (e) | e∈Ew}. That is, the spatial attribute map f can be expressed by a map Φ = f (E) = σ (θ (E)) obtained by combining θ and f.

<Division information>
The data reconstruction unit 722 randomly divides the space to which the spatial attribute information belongs according to the keyword “random” specified in the neighborhood criterion and the information “average spatial extent of about 1 km” specified in the allowable neighborhood range information. Create split information to do

  In the present embodiment, the dimension of the space to be divided is three dimensions. Further, in this embodiment, as parameters necessary for the division processing described later, a positive integer B, a positive integer M, an integer C of 2 or more, and a probability density distribution h having an average value of 1 for a random variable W is 1 Assume that (W) is preset. Note that these values may be acquired from the query 702.

  The data reconstruction unit 722 considers dividing a three-dimensional space using these parameters. Specifically, the data reconstructing means 722 divides the parallel vector so that the direction B of random direction vectors is uniformly directed and the interval takes a random variable W in each direction.

  First, an index calculation formula represented by Formula 2 shown below is defined.

  In Equation 2, a is a two-dimensional vector that uniformly faces a random direction. Assuming that ζ is a two-dimensional random vector composed of random numbers according to an independent standard normal distribution N (0, 1), a is calculated as a = ζ / | ζ |. In the present embodiment, the target space is a three-dimensional space, but in general, a is calculated in the same manner even in the case of the D dimension.

  In Equation 2, u is a random number that follows a uniform distribution in the range [0, 1). W is an actual value of the random variable W according to the probability density distribution h (W) having an average value of 1.

  In Expression 2, the size restriction parameter C is added to limit the upper limit value of the index number to C-1. When C = ∞ is designated, mod C is omitted.

  FIG. 8 is an explanatory diagram showing an example of dividing a two-dimensional space. The geometric meaning represented by Equation 2 is that the space is divided by a plane in which the normal direction faces a and is arranged at an interval w. Arbitrary vectors having the same index number calculated using Expression 2 belong to the same divided area.

  If this division process is repeated B times independently, the space can be divided by B planes arranged in parallel in a uniform random direction. Each divided area is expressed by a set of index numbers. Further, by performing this process of dividing the space M times independently, a result of dividing the space into M spaces in different directions and intervals can be obtained.

  Let each realized value of the random variable in Equation 2 be sjk = (ajk, wjk, ujk). Here, j∈ {1,..., M}, and k∈ {1,. In this case, the query analysis unit 721 uses the division parameters S = {{s11, s12,..., S1B},..., {SM1,.

<Viewpoint attribute map>
In order to express “user diversity” specified by the query, the query analysis unit 721 is given a frequency distribution {r1, r2,..., Rk} of the user attribute u in a certain data group Eu. The viewpoint attribute map g is defined by Equation 3 shown below.

  In Equation 3, r = r1 + r2 +... + Rk. Equation 3 is an example of expressing user diversity. The query analysis unit 721 may express the user diversity by other methods as long as it can specify the order of diversity, such as Simpson's diversity index. In this way, the query analysis unit 721 selects the viewpoint attribute mapping according to the contents described in the query viewpoint reference.

  The query analysis unit 721 transmits information indicating the above θ (E) and σ (Ew) to the data reconstruction unit 722 as information on the spatial attribute map f. In addition, the query analysis unit 721 transmits the above-described division parameters S and C to the clustering unit 725 as the division information. Further, the query analysis unit 721 transmits information indicating the above-described Expression 3 to the cluster selection unit 726 as information on the viewpoint attribute mapping g.

  Note that the query analysis unit 721 may share the mapping contents with the transmission destination in advance, and transmit the mapping information by transmitting only the parameters to the transmission destination. Further, the query analysis means 721 may implement mapping as a method on the transmission side and provide mapping information as a callback function. Further, the query analysis unit 721 may issue a command for selecting a method corresponding to the content of the mapping among methods pre-installed on the receiving side.

  In addition, the query analysis unit 721 may transmit the division information by sharing in advance the content of Equation 2 above with the transmission destination and transmitting only the parameters to the transmission destination.

  Further, the query analysis unit 721 can transmit the viewpoint attribute mapping information by providing a method indicating the contents of “user diversity” shown in the above-described Expression 3 to the cluster selection unit 726 as a callback function. Good. In addition, the query analysis unit 721 transmits information on the viewpoint attribute mapping by issuing a command for selecting a method corresponding to the processing of Expression 3 from the evaluation functions implemented in the cluster selection unit 726. May be.

  When the data reconstruction unit 722 receives the information of the spatial attribute map f, the data reconstruction unit 722 acquires the data of the database 712 through the database management unit 711. The data reconstruction unit 722 acquires all data including w in the text information using a normal query defined in the database. Then, the data reconstruction unit 722 obtains φ by performing the process shown in Equation 1 using the three-dimensional vector y composed of space and time information for each piece of acquired data.

  The data reconstruction unit 722 adds the acquired φ to the attribute of the original data e as space attribute information, and requests the reconstruction data management unit 723 to save the data.

  The reconfiguration data management unit 723 stores the requested data in the reconfiguration data storage unit 724. The reconfiguration data management unit 723 returns a completion notification to the data reconfiguration unit 722 when the storage process is completed. When this process is repeated and registration of all data in the reconfiguration data storage unit 724 is completed, a data reconfiguration completion notification is sent to the query analysis unit 721.

  Upon receiving the data reconstruction completion notification, the query analysis unit 721 requests the clustering unit 725 to cluster the data stored in the reconstruction data storage unit 724.

  The clustering means 725 performs space division M times using the received division information. At this time, the spatial attribute information of the reconstruction data is also used. The clustering means 725 groups data belonging to the area divided by each division process (hereinafter referred to as trial) as one subset.

  As a result, the total number of areas divided in each trial i (i = 1,..., M) is Ki, and a subset of data belonging to each of the divided areas is represented by Ei, j (j = 1,... , Ki), the output Ω is Ω = {{E1,1,..., E1, K1}, {E2,1,..., E2, K2},. ,... EM, KM}}.

  The cluster selection means 726 creates a frequency distribution of user types for the subset Ei, j in each trial i from this output Ω. Then, the cluster selection unit 726 calculates the user diversity index Pi, j using Equation 3 shown above. As a result, {{P1,1, ..., P1, K1}, {P2,1, ..., P2, K2}, ..., {PM, 1, ..., PM, KM}} Is obtained.

  The cluster selection means 726 selects the upper Q from the user diversity index Pi, j having a large value. The selected result (hereinafter referred to as sampling subset data ΩQ) is ΩQ = {{E1,1,..., E1, Q}, {E2,1,..., E2, Q},. .., {EM, 1,... EM, Q}}. By this processing, it is possible to sample a subset of data in a region with high user diversity in the considered space. Note that Q may be a predetermined value or a value specified by a query.

  Finally, the data shaping means 727 generates the sampled subset data ΩQ = {{E1,1,..., E1, Q}, {E2,1,. ,..., {EM, 1,... EM, Q}} are processed, shaped, and output as output subset data 703.

  In general, each subset {Ei, 1,..., Ei, Q} and {Ej, 1,..., Ej, Q} extracted for different trial numbers i and j are in similar spatial regions. There is a possibility of distribution. In addition, these subsets may be exactly the same or may include partially overlapping data. In such a case, it may be better to format and output the data as one.

  The data formatting method depends on the policy. For example, assume a policy that “a subset having common data is integrated into one subset”. In this case, when the data shaping means 727 is given an input set {Xi} (i = 1,..., L), its output {Yj} is “Yj = Union ({Xi}) {i∈ Lj} ”and {Yj} may be created so as to satisfy“ Yj1∩Yj2 = φ (empty set) for j1 ≠ j2. ”

  Here, “Union ({Zi}) {range of i}” means taking the union of Zi corresponding to i within the specified range in the set {Zi}. Also, Lj⊆ {1,..., L} satisfies Xi1∩Xi2 ≠ φ (empty set) for different i1εLj and i2εLj, and Lj1 異 な る for different j1 and j2. This is an index set that satisfies Lj2 = φ (empty set).

  In this case, the data shaping unit 727 searches for another Xj including an overlapping element for a certain Xi. The data shaping unit 727 creates a set obtained by integrating a certain Xi and another Xi, and performs predetermined processing on the set {Xi}. The data shaping unit 727 repeats this operation until all sets have no overlapping elements.

  Note that the method by which the data shaping unit 727 shapes the data is not limited to the above method. The data shaping unit 727 may output the data as it is without shaping the data according to the policy. That is, the data shaping unit 727 may output data in a format that matches the user's request.

  In the present embodiment, the method of randomly dividing the space on the (super) plane has been described. Note that the clustering unit 725 may divide the space on a plane other than the plane by performing plane division after performing nonlinear conversion once.

  Further, the clustering unit 725 may divide the space by a method other than random division. For example, the clustering unit 725 may divide the space in consideration of the localization of the data distribution. Further, the clustering means 725 may perform Voronoi division. Further, the clustering unit 725 may divide the space using a space index using a tree structure such as R-Tree. Various other clustering methods can be applied to the method in which the clustering means 725 divides the space.

  However, the response time is important when considering repeated hypothesis testing. Generally, since the clustering process takes time, it is more preferable in the present invention to use a method that is faster than accuracy.

  In the second embodiment, an example in which the database 112 illustrated in FIG. 1 is arranged as a distributed database, and a user uses a data search device from a client PC (Personal Computer) via a communication network represented by the Internet. To do. FIG. 9 is an explanatory diagram illustrating an example in which data search is performed via a communication network. A data management apparatus 907 illustrated in FIG. 9 corresponds to the data search apparatus of the present invention.

  A user uses the client PC 901 to access the application 903 via a communication network 902 represented by the Internet. At this time, a query necessary for data acquisition is issued from the client PC 901, and data is acquired from the distributed database 910.

  The application 903 provides a service to the client PC 901 by accessing a database storing large-scale data and performing desired processing. The application 903 is typically implemented as a Web application. In this embodiment, it is assumed that the application 903 does not have a function for processing large-scale data itself, and processes small-scale data necessary to respond to a request from the client PC 901.

  In this embodiment, it is assumed that the application 903 is an application that processes text information to which position information and time are added. For example, the application 903 displays a certain range on the map requested from the client PC 901 by changing the color according to the position distribution of diversity indicated by the text information. However, the processing performed by the application 903 is not limited to this example.

  It is assumed that data to be searched is stored and managed on a distributed database 910 connected via a communication network 909. Further, it is assumed that the data management apparatus 907 can access the distributed database 910 using the database driver 908. The data management device 907 provides an application program interface (API) 905 to the application 903, and the application 903 issues a search query using the API 905.

  The client PC 901 has a function of selecting a display range and a zoom level when displaying a map on a browser. The application 903 automatically calculates an allowable neighborhood range that can be considered to be the same region based on human sensitivity from the display range and zoom level. In this embodiment, the application 903 sets the neighborhood reference described in the first embodiment.

  Further, by mounting the interface 906 on the API 905 provided by the data management device 907, user diversity is specified as a viewpoint reference.

  The interface 906 is provided as a comparator that provides for comparing any two sets. Here, when T is a data class and a set is expressed by a list of List <T> and T, the comparator is described as “public int evaluator (List <T> lst1, List <T> lst2)).

  As shown by the interface implementation 904 in FIG. 9, the application 903 designates the reconfigured data class DataClass that is actually handled as T and implements the comparison process.

  The data management apparatus 907 receives a data search query issued from the application 903 when the display range is selected for the client PC 901. This data search query includes a data display range, a neighborhood criterion, an allowable neighborhood criterion, and a viewpoint criterion. Then, the data management device 907 uses the database driver 908 to acquire the corresponding data, perform data reconstruction and sampling, and return a set of data subsets to the application 903.

  The result is output as a set of data clusters sorted by viewpoint criteria for each neighborhood. For example, when the results are displayed on the browser, the results are displayed in a list form in a list format, and the diversity is displayed at a corresponding location on the map based on a predetermined color coding. For example, the higher the diversity, the brighter the color may be displayed.

  In addition, the application 903 may implement a function capable of interactively searching for customer behavior trends as a marketing tool. In this case, it is assumed that the user is a marketer and analyzes customer information such as past log data when considering a marketing strategy. Hereinafter, an example of a method for analyzing customer behavior trends will be described.

  The customer information includes a residence position, purchase item ID, purchase item category, purchase time, customer age, and the like. Based on this information, the marketer is supposed to make a store opening plan and a sales plan.

  First, the marketer narrows down the purchase item category to the category of the main product (for example, food). In addition, the marketer uses the residence position as a neighborhood reference. Further, the marketer sets the allowable neighborhood range reference to, for example, 5 km. The marketer also sets the viewpoint standard to low customer age diversity. If the data management device 907 is used under such an assumption, it is possible to extract only data of residential areas where the purchase layer is concentrated in a specific age part as a subset of the original data. By analyzing the extracted information, it is possible to determine which age group is the main purchase layer in each region.

  Furthermore, the marketer can obtain more information by using the extracted subset. For example, marketers use age and residence location as neighborhood criteria. Further, the marketer sets the allowable neighborhood range reference to, for example, 10 years old and a position range of 5 km, respectively. In addition, the marketer sets the viewpoint criterion as to which purchase item ID is purchased in which purchase time zone (for example, every hour) from the viewpoint of low diversity.

  When the data management device 907 is used under such an assumption, it can be determined whether a specific age group purchases a specific purchase item at a specific time in a specific area. For example, it is determined that a resident in his 20s who lives in a certain region often purchases instant foods around 23:00.

  If these pieces of information can be acquired, for example, it is possible to obtain information for determining which store is opened in which region and what product should be received at that store.

  The example shown in the present embodiment is a simple example of information set in a query. Information (for example, neighborhood criteria and viewpoint criteria) set in the query may be more complicated. For example, when a general function for extracting data from a database is used, when searching for a complicated case, it is necessary to describe a complicated search sentence. However, according to the present embodiment, even when searching for a case where a complicated search sentence must be described, the search processing can be performed flexibly and at high speed.

  In addition, when a search condition that is not clear in advance is necessary, a trial and error process for changing the combination of search conditions little by little is essential. According to the present embodiment, hypothesis verification can be efficiently performed by repeatedly issuing a search query with the above-described flexibility.

  In the third example, an example in which the data search device connects the data search units 120 of the first embodiment in multiple stages will be described. FIG. 10 is a block diagram illustrating a configuration example of the data search apparatus according to the present embodiment. The data search apparatus illustrated in FIG. 10 includes a data registration unit 110, a data search unit 1020, and a data search unit 1030.

  As illustrated in FIG. 10, the data search apparatus includes a plurality of data search units, so that a search process can be performed based on a plurality of criteria.

  For example, it is assumed that the query 1001 specifies two sets of neighborhood criteria, allowable neighborhood range criteria, and viewpoint criteria, respectively. In this case, the data search unit 1020 creates a spatial attribute based on the first neighborhood criterion and the first allowable neighborhood range criterion. Further, the data search unit 1020 samples a set of subsets of data divided using the space attribute based on the first viewpoint criterion.

  The sampled data is input to the data search unit 1030. The data search unit 1030 creates a spatial attribute for the sampled data based on the second neighborhood criterion and the second allowable neighborhood range criterion. Further, the data search unit 1030 samples a set of subsets of data divided using the space attribute based on the second viewpoint criterion. In this embodiment, the data search apparatus includes a plurality of data search units, so that the sampling process can be repeated.

  In the example illustrated in FIG. 10, two data search units are illustrated, but the number of data search units is not limited to two and may be three or more. Moreover, the configuration shown in the second embodiment may be used instead of the configuration in which the data search device includes a plurality of data search units. In this case, the query analysis unit 121 updates conditions such as an appropriate neighborhood criterion, an allowable neighborhood range criterion, and a viewpoint criterion from the sampled result, and notifies the data reconstruction unit 122 and the sampling unit 125 of the information again. It is sufficient to perform a loop process.

  Next, the outline of the present invention will be described. FIG. 11 is a block diagram showing an outline of a data search apparatus according to the present invention. The data search apparatus according to the present invention includes a proximity criterion used as a criterion for determining similarity of data to be searched, an allowable neighborhood range criterion indicating a range in which the data to be searched are similar, and a viewpoint criterion indicating a search viewpoint. In response to an input of a query including the query (for example, the query 102), based on the neighborhood criterion and the allowable neighborhood range criteria, a spatial attribute mapping (mapping the search target data on a space in which similarity with other data can be determined ( For example, a spatial mapping generation unit 81 (for example, a query analysis unit 121) that generates a spatial attribute mapping f) and one or more pieces of given data based on viewpoint criteria in accordance with the input of the query A viewpoint mapping generation unit 82 (for example, query analysis unit 121) that generates a viewpoint attribute mapping (for example, viewpoint attribute mapping g) to be originally mapped, and a search object using the spatial attribute mapping By converting the data, the space attribute information generating unit 83 (for example, the data reconstructing unit 122) that generates the spatial attribute information corresponding to the data, and the set of the spatial attribute information are divided according to the allowable neighborhood range criterion. Each subset is generated using a subset generation means 84 (for example, sampling means 125) that creates a set of search target data corresponding to each spatial attribute information as a subset of the search target data, and a viewpoint attribute map. And mapping means 85 (for example, sampling means 125) for mapping each of the subsets corresponding to the elements of the mapped total ordered set according to the order determined based on the elements. Search result specifying means 86 (for example, sampling means 125) for specifying search results (for example, selecting Q items in descending order) is provided.

  With such a configuration, even when a search is performed based on a viewpoint showing a plurality of similarities between data, the data search can be performed at high speed and flexibly.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Supplementary Note 1) A query including a neighborhood criterion used as a criterion for judging similarity of search target data, an allowable neighborhood range criterion indicating a range in which the search target data is similar, and a viewpoint criterion indicating a search viewpoint In response to the input, based on the neighborhood criterion and the allowable neighborhood range criterion, a spatial mapping generation unit that generates a spatial attribute mapping that maps data to be searched on a space where similarity with other data can be determined; and In accordance with the input of a query, based on the viewpoint criteria, a viewpoint mapping generating means for generating viewpoint attribute mapping that maps one or more pieces of given data under a total ordered set, and a search using the spatial attribute mapping By converting target data, spatial attribute information generating means for generating spatial attribute information corresponding to the data, and dividing the set of spatial attribute information according to the allowable neighborhood range criterion A subset generation means for creating a set of search object data corresponding to each divided space attribute information as a subset of the search object data, and using the viewpoint attribute mapping, Mapping means for mapping to the original, and search result specifying means for specifying the search results for each subset corresponding to the elements of the mapped total ordered set according to the order determined based on the elements A data search device characterized by the above.

(Additional remark 2) It is provided with the data storage means which memorize | stores the data of search object, The data extraction means which extracts the data which satisfy | fills the designated conditions from the data memorize | stored in the said data storage means, Spatial attribute information generation means The data search device according to appendix 1, wherein the extracted data is converted using a spatial attribute mapping to generate spatial attribute information corresponding to the data.

(Supplementary Note 3) The spatial mapping generation unit generates a spatial attribute mapping that maps the data specified as the search result based on the new neighborhood criterion and the allowable neighborhood range criterion, and the viewpoint mapping generation unit generates the new viewpoint criterion. The viewpoint attribute mapping is generated based on the spatial attribute information generating means, and the spatial attribute information generating means generates the spatial attribute information corresponding to the data by converting the data specified as the search result using the spatial attribute mapping. Or the data search device according to appendix 2.

(Supplementary Note 4) When the specified search result does not satisfy the specified condition, the search result output means is either a spatial attribute map, a viewpoint attribute map, or an allowable neighborhood range criterion, or a spatial attribute map, a viewpoint attribute map, and The data search device according to supplementary note 3, wherein all of the allowable neighborhood range criteria are updated.

(Supplementary note 5) The data according to any one of supplementary notes 1 to 4, wherein the subset generation unit creates a subset of the search target data by randomly dividing the spatial attribute information according to the allowable neighborhood range criterion. Search device.

(Supplementary note 6) The data search device according to supplementary note 2, wherein the data storage means manages the attribute of each data in a table format, a key value format, or a graph format.

(Supplementary note 7) The data search device according to supplementary note 6, wherein the data storage means can access the stored data using SQL or NoSQL.

(Supplementary note 8) The data search device according to supplementary note 6 or supplementary note 7, wherein the data storage means is arranged to be distributed.

(Supplementary note 9) Spatial attribute mapping is a mapping that prescribes a method for converting a data attribute to a metric space in which a quantitative index that can determine the similarity between arbitrary data is defined. The range of data that can be identified is defined, and the viewpoint attribute map is a map that defines a method for converting data attributes into a fully ordered set, according to any one of appendix 1 to appendix 8. Data retrieval device.

(Supplementary note 10) The viewpoint criterion is any one of supplementary notes 1 to 9 in which information indicating which data set the searcher prefers when comparing two arbitrary data sets is specified. The data search device described in 1.

(Supplementary Note 11) A query including a neighborhood criterion used as a criterion for determining similarity of search target data, an allowable neighborhood range criterion indicating a range in which the search target data is similar, and a viewpoint criterion indicating a search viewpoint In response to the input, generates a spatial attribute mapping that maps the search target data on a space that can be determined to be similar to other data based on the neighborhood criterion and the allowable neighborhood range criterion, and responds to the query input. , Generating a viewpoint attribute map that maps one or more pieces of given data based on a total ordered set based on the viewpoint criteria, and converting the data to be searched using the spatial attribute map, Generating spatial attribute information corresponding to the data, dividing the set of spatial attribute information according to the allowable neighborhood range criterion, and collecting a set of search target data corresponding to each of the divided spatial attribute information, Create as a subset of search target data, use the viewpoint attribute mapping, map each subset to a total ordered set, and each subset corresponding to the mapped total ordered set, A data search method characterized by specifying a search result according to an order determined based on the above.

(Supplementary note 12) By extracting data satisfying a specified condition from the data stored in the data storage means for storing data to be searched, and converting the extracted data using a spatial attribute mapping, The data search method according to appendix 11, wherein spatial attribute information corresponding to the data is generated.

(Additional remark 13) The neighborhood reference used as a reference | standard which judges the similarity of the data of search object to a computer, the permissible neighborhood range reference | standard which shows the range where the data of search object are similar, and the viewpoint reference | standard which shows the viewpoint to search Spatial mapping generation processing for generating a spatial attribute mapping that maps data to be searched on a space where similarity with other data can be determined based on the neighborhood criterion and the allowable neighborhood range criterion according to an input of a query including In response to the input of the query, based on the viewpoint criteria, a viewpoint mapping generation process for generating a viewpoint attribute mapping that maps one or more pieces of given data under a total ordered set, using the spatial attribute mapping Spatial attribute information generation processing for generating spatial attribute information corresponding to the data by converting search target data, and the spatial attribute information according to the allowable neighborhood range criterion A subset generation process for dividing a set and creating a set of search target data corresponding to each divided space attribute information as a subset of the search target data, and using the viewpoint attribute mapping, A mapping process for mapping to an ordered set, and a search result specifying process for identifying each subset corresponding to the elements of all mapped ordered sets according to the order determined based on the element Data retrieval program to be executed.

(Supplementary note 14) A computer is caused to execute a data extraction process for extracting data satisfying a specified condition from the data stored in the data storage means for storing data to be searched. 14. The data search program according to appendix 13, wherein the converted data is converted using a spatial attribute map to generate spatial attribute information corresponding to the data.

  Although the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  This application claims the priority on the basis of the JP Patent application 2012-091085 for which it applied on April 12, 2012, and takes in those the indications of all here.

  The present invention is suitably applied to a data search apparatus that extracts a subset of highly interested data from a data group to be searched.

DESCRIPTION OF SYMBOLS 101 Data 102 Query 103 Output subset data 110 Data registration part 111 Database management means 112 Database 120 Data search part 121 Query analysis means 122 Data reconstruction means 123 Reconstruction data management means 124 Reconstruction data storage means 125 Sampling means 601 Condition Judgment means

Claims (10)

In response to a query input including a proximity criterion used as a criterion for determining the similarity of data to be searched, an allowable neighborhood range criterion indicating a range in which the data to be searched are similar, and a viewpoint criterion indicating a point of search, Spatial mapping generation means for generating a spatial attribute mapping that maps data to be searched on a space where similarity with other data can be determined based on the neighborhood criterion and the allowable neighborhood range criterion;
In accordance with the input of the query, viewpoint mapping generation means for generating viewpoint attribute mapping that maps one or more pieces of given data under a total ordered set based on the viewpoint criterion;
Space attribute information generating means for generating space attribute information corresponding to the data by converting the search target data using the space attribute map;
A subset generation means for dividing the set of spatial attribute information according to the allowable neighborhood range criterion, and creating a set of search target data corresponding to each of the divided spatial attribute information as a subset of the search target data;
Mapping means for mapping each subset to a fully ordered set using the viewpoint attribute mapping;
A data search apparatus comprising: search result specifying means for specifying a search result for each subset corresponding to an element of a mapped total ordered set according to an order determined based on the element. Data storage means for storing data to be searched;
Data extraction means for extracting data satisfying a specified condition from the data stored in the data storage means;
The data search apparatus according to claim 1, wherein the space attribute information generation means generates space attribute information corresponding to the data by converting the extracted data using a space attribute map. The spatial mapping generation means generates a spatial attribute mapping that maps the data specified as the search result based on the new neighborhood criterion and the allowable neighborhood range criterion,
The viewpoint map generation means generates a viewpoint attribute map based on the new viewpoint criterion,
The data search device according to claim 1 or 2, wherein the space attribute information generation means generates space attribute information corresponding to the data by converting the data specified as a search result using the space attribute map. . When the specified search result does not satisfy the specified condition, the search result output means is either a spatial attribute mapping, a viewpoint attribute mapping, or an allowable neighborhood range criterion, or a spatial attribute mapping, a viewpoint attribute mapping, and an allowable neighborhood range criterion. The data search device according to claim 3, wherein all of the data are updated. The data search device according to any one of claims 1 to 4, wherein the subset generation means generates a subset of search target data by randomly dividing the spatial attribute information according to an allowable neighborhood range criterion. . The data search device according to claim 2, wherein the data storage means manages the attribute of each data in a table format, a key-value format, or a graph format. In response to a query input including a proximity criterion used as a criterion for determining the similarity of data to be searched, an allowable neighborhood range criterion indicating a range in which the data to be searched are similar, and a viewpoint criterion indicating a point of search, Based on the neighborhood criteria and the allowable neighborhood range criteria, generate a spatial attribute mapping that maps the data to be searched on a space where similarity with other data can be determined,
In response to the input of the query, based on the viewpoint criteria, generates a viewpoint attribute mapping that maps one or more pieces of given data under a fully ordered set;
By converting the search target data using the spatial attribute mapping, to generate spatial attribute information corresponding to the data,
Dividing the set of spatial attribute information according to the allowable neighborhood range criterion, creating a set of search target data corresponding to each of the divided spatial attribute information as a subset of the search target data;
Map each subset to a fully ordered set using the viewpoint attribute mapping;
A data search method comprising: specifying a search result for each subset corresponding to an element of a mapped total ordered set according to an order determined based on the element. Extract the data that satisfies the specified conditions from the data stored in the data storage means for storing the data to be searched,
The data search method according to claim 7, wherein the extracted attribute data is converted using a spatial attribute map to generate spatial attribute information corresponding to the data. On the computer,
In response to a query input including a proximity criterion used as a criterion for determining the similarity of data to be searched, an allowable neighborhood range criterion indicating a range in which the data to be searched are similar, and a viewpoint criterion indicating a point of search, Spatial mapping generation processing for generating a spatial attribute mapping that maps data to be searched on a space where similarity with other data can be determined based on the neighborhood criterion and the allowable neighborhood range criterion;
A viewpoint mapping generation process for generating a viewpoint attribute mapping that maps one or more pieces of given data under a total ordered set based on the viewpoint criterion in response to the input of the query;
A spatial attribute information generation process for generating spatial attribute information corresponding to the data by converting the data to be searched using the spatial attribute mapping;
A subset generation process for dividing the set of spatial attribute information according to the allowable neighborhood range criterion and creating a set of search target data corresponding to each of the divided spatial attribute information as a subset of the search target data;
A mapping process for mapping each subset to a fully ordered set using the viewpoint attribute mapping; and
A data search program for executing a search result specifying process for specifying a search result for each subset corresponding to an element of a mapped total ordered set according to an order determined based on the element. On the computer,
From the data stored in the data storage means for storing the search target data, the data extraction process for extracting the data satisfying the specified condition is executed,
The data search program according to claim 9, wherein the spatial attribute information corresponding to the data is generated by converting the extracted data using the spatial attribute mapping in the spatial attribute information generation process.

Images