JP3571162B2 - Similar object search method and apparatus - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an object similar to an object (reference object) input by a user through some means from a large number of digitized objects (images, sounds, music, texts, etc.) such as an electronic museum or an electronic catalog. And a similar object search device that presents some of the top ranks as search results in the order in which they are determined to be similar.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a similar object search device that searches for and displays an object similar to a reference object specified by a user calculates some feature amounts of the objects, and determines the closeness (distance) of the feature amounts between the objects. The similarity is obtained by the calculation, and the similarity is ranked. At this time, as the types of feature amounts used to quantify the object, for example, if the object is an image, various types such as color tone, lightness, saturation, and the direction of a line in the image are used.
[0003]
The distance is calculated as a Euclidean distance between the feature values represented by a multidimensional vector for each type of the feature value, and further, the distance between a plurality of types of feature values for the same object combination is calculated. By performing weighting, the final similarity is determined in consideration of the user's viewpoint on similarity (for example, searching for a similar image object from the viewpoint of similar colors). At this time, the weight of the distance of the feature amount is input by the user from the search condition input unit.
[0004]
In this method, the distance calculation for each type of the feature amount is performed for all the stored objects, and the final weighting is performed on each object based on the distance evaluation value for each type of the feature amount. Calculating the degree of similarity, ranking, and returning the top object.
[0005]
Alternatively, there is a method in which all types of feature amounts are handled as one multidimensional vector space and distance calculation is performed there without dividing distance calculation for each type of feature amount. In this case, the reflection of the weight, which is the user's search viewpoint, preliminarily limits the weight pattern specified by the user, and takes into account the weight of each vector component of the multidimensional vector according to the pattern. It is done in such a way.
[0006]
In this method, in order to speed up the search, an index is calculated for a multidimensional vector space in which weights are added in advance, and at the time of search, candidates can be narrowed down using the index.
[0007]
[Problems to be solved by the invention]
In the conventional similar object search apparatus based on the similarity calculation as described above, there is a problem that the calculation cost is high because the distance calculation has to be performed for every type of the feature amount for all the objects.
[0008]
Also, in the method of adding a weight to the multidimensional vector space of the feature amount in advance with respect to the determined pattern, since the index is assigned and the candidates are narrowed down, the calculation cost can be suppressed, but There is a problem that an index must be provided for all of the weights of the given pattern, and the weight can be specified only for a limited number of patterns.
[0009]
SUMMARY OF THE INVENTION An object of the present invention is to solve the above-described problem in the prior art, eliminate the trouble of performing distance calculation for all objects, and search for similar objects with an arbitrary weight. And equipment.
[0010]
[Means for Solving the Problems]
The similar object search method of the present invention includes:
The feature amount of each type of the accumulated object is calculated and stored as a point in the multidimensional vector space for each individual feature amount type. Calculating the required index,
Obtaining a weight between the types of feature amounts that are search viewpoints;
Calculating the feature amount of the reference object for each type;
For each type of feature amount, with respect to the distance between the reference object and each of the stored objects in the multidimensional vector space, a neighborhood object search step of obtaining a specific number of objects in the vicinity of the reference object using the index. ,
A similarity calculation step of creating similar object candidates by grouping the neighboring objects obtained for each type of feature amount for all types of feature amounts, and calculating the similarity by adding the weight to the candidates,
And ordering the slave have candidate object to the similarity, and determining a final result to match the number specified by the user,
Presenting the final result to the user.
[0011]
The present invention provides a method of dividing a calculation for each feature amount type, narrowing down a nearby object by using an index in each feature amount multidimensional vector space, and a method of dividing a nearby object from all types of feature amount multidimensional vector space. Is performed for the distance calculation for the candidates that have been combined and the similarity calculation using the weights.
[0012]
According to the second aspect of the present invention, in the neighborhood object search step and the similarity calculation step, the distance of the feature type of the neighborhood object in the multidimensional vector space is calculated, and the neighborhood obtained from all the feature quantity types is calculated. At the time when the objects are combined and the candidate objects are obtained, the distance in the multidimensional vector space of the characteristic amount type is calculated for the feature amount for which the distance has not been obtained for each candidate object. The similarity is calculated by taking into account the similarity.
[0013]
According to the third aspect of the present invention, in the neighborhood object search stage and the similarity calculation stage, the distance of the feature type of the neighbor object in the multidimensional vector space is not calculated, and the distance is obtained from all feature types. At the time when the candidate objects are obtained by grouping the neighboring objects, the distance in the multidimensional vector space of each feature amount type is calculated for each candidate object, and then the similarity is calculated by adding the weight to all the candidate objects.
[0014]
According to the fourth aspect of the present invention, in the neighborhood object search step, the neighborhood objects searched for in the multidimensional vector space of each feature amount type from the reference object with the feature amount of the type of interest at that stage. The distance from the reference object is calculated for all types of feature amounts, not only the distance of the object, but the distance calculated once for each object is marked so that the distance calculation does not overlap.
[0015]
The case where the object is an image object will be briefly described as an example. It is assumed that hue, lightness, and saturation are used as the types of the feature amounts. Now, it is assumed that neighboring objects A, B, and C are obtained from the hue feature quantity multidimensional vector space, neighboring objects B, C, and D are obtained from the brightness space, and neighboring objects C, D, and E are obtained from the saturation space. I do. Candidate objects that combine these are A, B, C, D, and E. For each of these, three types of distances of a hue space, a lightness space, and a saturation space are obtained, weighted, and the final similarity is calculated. For the neighboring objects A, B, and C, the hue distance is calculated at the stage of obtaining the neighboring objects. Others are the same. "For the feature amount types for which the distance has not been determined for each candidate object,... Calculate the distance" means that the brightness and saturation distances have not been determined for the neighboring object A, and This is because the saturation distance has not been determined. All distances have been determined for the nearby object C. Such an undetermined distance is obtained.
[0016]
According to the third aspect, all the distances have not been obtained at the stage when the candidate objects A, B, C, D, and E have been obtained. According to the second aspect, the distance is partially obtained, and the distance of the missing portion is obtained later. In the third aspect, the entire distance is obtained later.
[0017]
According to the fourth aspect, at the stage where the nearby objects A, B, and C are obtained from the hue space, the distances of all the hues, lightness, and saturation are obtained for the nearby objects A, B, and C. Thus, at the stage when the candidate objects A, B, C, D, and E have been obtained, the distances of the three types of feature spaces have been obtained for all five objects.
[0018]
The similar object search device of the present invention
Object data storage means for storing object data;
Reference object input means for a user to input the reference object;
Means for specifying the number of similar objects desired by the user by the user;
Means for obtaining a weight between the types of feature amounts that are search viewpoints;
A feature value calculating unit that calculates a feature value of each type for each of the stored objects and reference objects;
A feature amount storage unit for storing, in the calculated feature amounts, feature amounts of an object stored in advance as points in a multidimensional vector space for each feature amount type;
Of the calculated feature amounts, for a feature amount of an object stored in advance, in a multidimensional vector space of each feature amount type, an index required for data search is calculated and the index is stored. Index generation and storage means,
For each type of feature amount, with respect to the distance in the multidimensional vector space between the reference object and each of the stored objects, a neighborhood object search means for determining a specific number of objects in the vicinity of the reference object using the index. ,
Similarity calculating means for summarizing neighboring objects obtained for each type of feature amount for all types of feature amounts, creating candidates for similar objects, and calculating the similarity by adding the weight to the candidates;
Means for ordering the candidate objects according to the similarity and obtaining a final result according to the number specified by the user;
Object presenting means for presenting the final result to the user.
[0019]
BEST MODE FOR CARRYING OUT THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings.
[0020]
FIG. 1 is a configuration diagram of a similar object search device according to an embodiment of the present invention, and FIG. 2 is a flowchart showing the operation thereof.
[0021]
The similar object search device according to the present embodiment includes an object data storage unit 10 that stores digitized object data Ob1 to Obm, a feature amount calculation unit 11 that calculates feature amounts of the objects Ob1 to Obm, A feature quantity / index storage unit 12 for storing separately generated indexes, a reference object input unit 131 for a user to newly input a reference object from outside, and a user for measuring similarity between objects A weight specifying unit 132 for specifying a weight, which is a viewpoint of the above, an object number specifying unit 133 for specifying the number of objects obtained as a result by the user, and an object presenting unit 134 for presenting the object to the user. Interface 13 with the user, and a multi-dimensional vector space An index generation unit 14 for generating an index; and a neighbor object search unit 15 for obtaining, for each type of feature quantity, an object that is close to a reference object in a multidimensional vector space by a distance in the multidimensional vector space. Upon receiving a search request from the user, the feature amount of the reference object is divided into feature amount types, the search request is delivered to the neighboring object search unit 15 for each feature amount type, and the returned results are grouped into a candidate object. For each candidate object, if necessary, determine the distance for each feature type from the missing reference object, calculate the similarity by taking into account the weight, and calculate the number of similar objects by the specified number from the upper object. A coordinator unit 16 that returns a result as a temporary data storage unit 17 that temporarily stores an intermediate result. It is configured.
[0022]
Note that the index is an auxiliary data structure for searching for a point at a specific position in a multidimensional space or for searching for a point near a specified position at high speed. Typically, there are R-tree and k-dtree. These indexes are searched by creating a hierarchical data structure depending on the position of the point and manipulating it from the source (root node). In a familiar example, meshes are cut in the vertical and horizontal directions on a road map (two-dimensional space) or the like, and this sightseeing spot is located in the vertical and horizontal directions of 3 and can be used as a kind of index.
[0023]
Next, the operation of this embodiment will be described with reference to the flowchart of FIG.
[0024]
The feature amounts Ob11, Ob12,. . . , Ob1n, the feature amounts Ob21, Ob22,. . . , Ob2n,. . . , Obm1, Obm2,. . . , Obmn (n is the number of types of feature amounts) are calculated in advance by the feature amount calculation unit 11 and stored in the feature amount / index storage unit 12. Each object Ob1 to Obm itself is stored in the object data storage unit 10. In addition, in the multidimensional vector space of each feature amount type, an index is generated by the index generation unit 14 for search. The index is stored in the feature / index storage unit 12.
[0025]
First, the user inputs a reference object from the reference object input unit 131 and weights w1,. . . , Wn, and the number K of objects from the number-of-objects specifying unit 133. If there is no input from the user, default values are used for the weights w1 to wn and the number K of objects (step 21). Next, the feature amounts VR1,. . . , VRn are calculated by the feature value calculator 11 (step 22).
[0026]
Next, the coordinator unit 16 determines the type i of the feature quantity, the feature quantity VRi of the i-th type of the reference object for each feature quantity type, and the number (a function of K) f (K) (the function of K) specifying the number of neighboring objects. K or more) and requests the nearby object search unit 15 to search for nearby objects (step 23).
[0027]
The neighboring object search unit 15 that has received the request uses, for each feature amount type i, f (K) points that are close to the point corresponding to the reference object in the multidimensional vector space at a distance in the space using an index. And returns it to the coordinator 16 (step 24).
[0028]
The coordinator unit 16 that has returned the neighboring objects collects the returned neighboring objects for all feature types and creates a candidate object group (step 25). The candidate object group is obtained, for example, as a union of neighboring objects. In this case, the number of candidate objects is a maximum of f (K) × n (when there is no overlap between neighboring objects for each feature amount type), and a minimum f (K) (when the neighboring objects are exactly the same for each feature amount type). become.
[0029]
Next, for the candidate object Obk, if the distance dki (i is the type of the feature amount) between the point VRi and the Obki corresponding to the reference object for each feature amount type is not determined, the shortage is calculated (step 26). .
[0030]
For each candidate object Obk, the distances dk1,. . . , VRn and Obkn are found, and weights w1,. . . , Wn are taken into account to calculate the similarity (step 27). For example, the similarity is calculated as w1 / dk1 +... + Wn / dkn.
[0031]
Next, the candidate objects are arranged in descending order of the obtained similarity, and the top K objects are returned as a result (step 28).
[0032]
When the result is returned from the coordinator unit 16, the object presenting unit 134 presents the result to the user (step 29).
[0033]
Note that, for the feature amount type designated as having a weight of 0, the neighboring object search and the distance calculation can be omitted. Further, as the weights, not only those directly specified by the user but also those obtained by some method from the specified ones may be used. Steps 24 and 26 can be performed in parallel with respect to the feature type.
[0034]
According to another embodiment of the present invention, the distance calculation in step 26 is performed together with the step of obtaining a nearby object in step 24. In this case, step 26 is unnecessary.
[0035]
【The invention's effect】
As described above, according to the present invention, for each type of feature amount, an index is used to narrow down the nearby objects of the reference object, create similar object candidates from these nearby objects, calculate the distance to these candidates, Since the calculation of the similarity is performed by the formula (1), the trouble of performing the distance calculation for all the objects can be omitted, and the similar object can be searched with an arbitrary weight.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a similar object search device according to an embodiment of the present invention.
FIG. 2 is a flowchart showing an operation of the similar object search device of FIG. 1;
[Explanation of symbols]
Reference Signs List 10 Object data storage unit 11 Feature amount calculation unit 12 Feature amount / index storage unit 13 Interface unit 131 Reference object input unit 132 Weight designation unit 133 Presented object number designation unit 134 Object presentation unit 14 Index generation unit 15 Nearby object search unit 16 Coordinator Unit 17 temporary data storage units 21 to 29 step

Claims (5)

A similar object search method in which objects similar to a reference object, which is an object designated by a user, are extracted from a large number of accumulated objects in a descending order of similarity and presented to the user,
The feature amount of each type of the accumulated object is calculated and stored as a point in the multidimensional vector space for each individual feature amount type. Calculating the required index,
Obtaining a weight between the types of feature amounts that are search viewpoints;
Calculating the feature amount of the reference object for each type;
For each type of feature amount, with respect to the distance between the reference object and each of the stored objects in the multidimensional vector space, a neighborhood object search step of obtaining a specific number of objects in the vicinity of the reference object using the index. ,
A similarity calculation is performed by grouping the neighboring objects obtained for each type of feature amount for all types of feature amounts to create similar object candidates, and calculating the similarity by adding the weight to the distance between the candidate and the reference object. Stages and
Ordering the candidate objects according to the similarity and obtaining a final result according to a number specified by a user;
Presenting the final result to a user. In the neighborhood object search step and the similarity calculation step, the distance of each feature amount type of the neighborhood object from the reference object in the multidimensional vector space is calculated, and the neighborhood objects obtained from all the feature amount types are collectively collected. At the time when the candidate object is obtained, the distance from the reference object in the multi-dimensional vector space is calculated for the feature type for which the distance has not been obtained for each candidate object. 2. The similar object search method according to claim 1, wherein the similarity is calculated by adding a weight to the distance to the object. In the neighborhood object search step and the similarity calculation step, the distance of each feature amount type of the neighborhood object from the reference object in the multidimensional vector space is not calculated, and the neighborhood objects obtained from all the feature amount types are collected. At the time when the candidate object is obtained by the calculation, the distance in the multidimensional vector space of each feature amount type is calculated for each candidate object, and then the similarity is calculated for all the candidate objects by adding a weight to the distance to the reference object. 2. The similar object search method according to claim 1, wherein: In the neighborhood object search step, not only the distance from the reference object in the feature amount of the type of interest in the stage but also all types of the neighborhood object searched in the multidimensional vector space of each feature amount type 2. A similar object search method according to claim 1, wherein the distance from the reference object is calculated for the characteristic amount of the object, but for each object, the distance calculated once is marked so that the distance calculation does not overlap. A similar object search device that extracts objects similar to a reference object, which is an object designated by a user, from a large number of accumulated objects in a descending order of similarity and presents them to the user,
Object data storage means for storing object data;
Reference object input means for a user to input the reference object;
Means for specifying the number of similar objects desired by the user by the user;
Means for obtaining a weight between the types of feature amounts that are search viewpoints;
A feature value calculating unit that calculates a feature value of each type with respect to each of the stored objects and the reference object;
A feature amount storage unit for storing, in the calculated feature amounts, feature amounts of an object stored in advance as points in a multidimensional vector space for each feature amount type;
Of the calculated feature amounts, for a feature amount of an object stored in advance, in a multidimensional vector space of each feature amount type, an index required for data search is calculated and the index is stored. Index generation and storage means,
For each type of feature amount, with respect to the distance in the multidimensional vector space between the reference object and each of the stored objects, a neighborhood object search means for determining a specific number of objects in the vicinity of the reference object using the index. ,
Creating candidate similar object be summarized for all of the feature types near objects found for the types of features, and similarity calculation means for calculating the similarity in consideration of the weight for that candidate,
And ordering the candidate object in accordance with the degree of similarity, and means for obtaining the final result according to the number specified by the user,
A similar object search device having object presentation means for presenting a final result to a user.

Images