JP3505393B2 - Similar object search method and apparatus, and recording medium storing similar object search program - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object (user input by some means) from a large number of digitized objects (images, sounds, music, texts, etc.) such as electronic museums and electronic catalogs. The present invention relates to a similar object search device that presents, as a search result, some of the higher order objects that are similar to a reference object).

[0002]

2. Description of the Related Art Conventionally, in a similar object search apparatus which searches for and displays an object similar to a reference object designated by a user, some feature quantities are calculated for each object, and the feature quantities are approximated to each other. The degree of similarity is calculated by calculating the distance (distance), and ranking is performed according to the degree of similarity. At this time, various types of feature quantities used for quantifying the object include, for example, if the object is an image, color tone, lightness, saturation, and direction of lines in the image.

The distance is calculated as a Euclidean distance between feature amounts represented by a multidimensional vector for each type of feature amount, and further, a plurality of types of feature amounts for the same combination of objects. By weighting the distance of, the final similarity can be determined by taking into consideration the user's viewpoint of similarity (for example, wanting to search for similar image objects from the viewpoint of similar colors). ing. At this time, the user inputs the distance weight of the feature amount from the search condition input means.

In this method, the distance calculation for each type of feature amount is performed for all accumulated objects, and each object is weighted based on the distance evaluation value for each type of feature amount. The final similarity was calculated, ranking was performed, and the higher-ranking objects were returned.

Alternatively, there is a method in which the distance calculation is not divided for each type of the feature amount but treated as one multidimensional vector space in which the feature amounts of all types are collected and the distance calculation is performed there. In this case, in the reflection of the weight, which is the user's search viewpoint, the weight pattern specified by the user is limited in advance, and the weight is added to each vector component of the multidimensional vector according to the pattern. Method.

In this method, in order to speed up the search, an index is calculated for a multidimensional vector space in which weights are added in advance, and at the time of search, the index can be used to narrow down the candidates. .

[0007]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
In the similar object search device based on the similarity calculation, the distance calculation must be performed for all the objects for each type of feature amount, which causes a problem of high calculation cost.

Further, in the method in which the weight is added to the multidimensional vector space of the feature quantity in advance for the determined pattern, the index is added to narrow down the candidates, so that the calculation cost can be suppressed. However, there was a problem that it had to have an index for all the weights of the determined pattern, and that the weights could still be specified only for a limited pattern.

The object of the present invention is to solve the above-mentioned problems in the prior art, to save the trouble of calculating the distance for all objects, and to search for similar objects with arbitrary weights. An object search method and apparatus are provided.

[0010]

According to the similar object search method of the present invention, the feature amount of each type of accumulated objects is calculated and accumulated as points of the multidimensional vector space for each feature amount type. In addition, in the multidimensional vector space of each feature type, a step of calculating an index required for data retrieval, a step of obtaining a weight between the types of feature amounts which is a search viewpoint, and a user A step of calculating the feature amount of the input reference object for each type, and referring to the distance in the multidimensional vector space between the reference object and each accumulated object for each type of feature amount using the index The number of objects in the neighborhood of the object is determined for each feature type using the weights between the feature types, and the determined number of objects in the neighborhood is determined. And a neighboring object search step for finding the target, and a similar object candidate is created by summarizing the neighboring objects found for each feature amount type for all feature amount types, and the similarity is obtained by adding the weight to the candidate. A degree of similarity calculation step, a step of ordering candidate objects according to the degree of similarity, and obtaining a final result in accordance with the number specified by the user or a default value ,
Presenting the final result to the user.

The present invention divides the calculation for each feature quantity type, and
Narrowing down neighboring objects using the index in each feature multidimensional vector space, distance calculation for candidates summarizing neighboring objects coming from all types of feature multidimensional vector space, and similarity calculation by weight Perform processing to perform.

References K. Curtis, N. Taniguchi, J. Nakagaw
a, M. Yamamuro, “A ComprehensiveImage Similarity R
etrieval System that recover Multiple Feature Vec
tors in High Dimensional Space, "In the Information Processing Society of Japan Database System Study Group 113-28, 1997.7, when the results of neighborhood search with multiple types of features are integrated and a similar object is searched by the overall similarity, The overall search accuracy depends on the number of top searches in each feature type.As reported in this document, the larger the number of searches in each feature type, the higher the overall search accuracy. On the other hand, in the neighborhood search, the number of top neighborhoods obtained by the search affects the time required for the search.
It is generally known that the larger the number of requests, the longer it takes. Therefore, according to the present invention, the number of objects in the vicinity to be obtained is determined for each feature amount type by using the weight between the feature amount types.

According to claim 2 of the present invention, in the neighborhood object searching step, the search accuracy when finding the neighborhood object for each feature quantity type is determined for each feature quantity type by using the weight between the feature quantity types. Set. Here, the search accuracy refers to how accurate the obtained k items are in the neighborhood search that determines the top k items in descending order of similarity, and how accurately the obtained k items are based on the true top k items that are calculated accurately. It is a measure to show.

References S. Arya, DM Mount, NS Netany
ahu, R. Silverman, and AY Wu, “An optimal algor
ithm for approximate nearest neighbor searching,
“5th Ann. ACM-SIAM Symposium on Discrete Algorith
As reported in ms, 1994, pp.573-582., the search accuracy is generally lowered (in this document, the value of ε is increased).
And, the search time will be reduced.

Therefore, with respect to the feature quantity type having a small weight, by lowering the search accuracy, it is possible to reduce the time required for calculation as a whole without impairing the overall search accuracy.

According to the fifth aspect of the present invention, in the neighborhood object searching step, the distance of the feature type of the neighborhood object in the multidimensional vector space is calculated, and the neighborhood objects obtained from all the feature types are calculated. At the time when the candidate objects are collectively obtained, the distance in the multidimensional vector space of the feature amount type for which the distance has not been obtained for each candidate object is calculated, and then at the similarity calculation stage, all candidate objects are calculated. Is added to calculate the degree of similarity.

According to the sixth aspect of the present invention, in the neighborhood object searching step, the distance of the neighborhood object in the multidimensional vector space of the feature quantity type is not calculated, and the neighborhood object obtained from all the feature quantity types is calculated. When the candidate objects are collectively obtained, the distance in the multidimensional vector space of each feature type is calculated for each candidate object, and then the similarity is calculated by adding weights to all candidate objects in the similarity calculation step. calculate.

According to claim 7 of the present invention, in the neighborhood object search step, for the searched neighborhood objects in the multidimensional vector space of each feature quantity type, the feature quantity of the type of interest at that stage Not only the distance from the reference object but also the distance from the reference object is calculated for all types of feature values, but for each object, once the distance calculation is made, a mark is added so that the distance calculation does not overlap.

The case where the object is an image object will be briefly described as an example. It is assumed that hue, lightness, and saturation are used as the types of feature quantities. Now, it is assumed that neighboring objects A, B, and C are obtained from the hue feature multidimensional vector space, neighboring objects B, C, and D are obtained from the lightness space, and neighboring objects C, D, and E are obtained from the saturation space. To do. Candidate objects that combine these are A, B, C, D, and E.
Becomes For each of these, three types of distances of a hue space, a lightness space, and a saturation space are obtained and weighted to calculate the final similarity. However, in the case of claim 5, , For neighboring objects A, B, C,
It means that the distance of hue is calculated at the stage of obtaining a neighboring object. Others are the same. "For a feature amount type for which the distance has not been obtained for each candidate object ... Calculate the distance" means that the lightness and saturation distances have not been obtained for the neighboring object A,
For the neighboring object B, the saturation distance has not been obtained, so this is obtained. All the distances have been obtained for the neighborhood object C. This is to calculate such an undetermined distance.

In claim 6, the candidate objects A, B,
This means that when C, D, and E are obtained, all distances are not obtained. In claim 2, the distance is calculated partially, and the distance of the missing portion is calculated later. In claim 3, the distance is calculated later.

In the seventh aspect, when the neighboring objects A, B and C are obtained from the hue space, the distances of all the hues, lightnesses and saturations of the neighboring objects A, B and C are obtained. In this way, the candidate objects A, B, C, D,
At the stage of obtaining E, for all five objects,
This means that the distances of the three types of feature spaces have been obtained.

The similar object search device according to the present invention includes object data storage means for storing object data, reference object input means for the user to input the reference object, and the number of similar objects desired by the user. Object number designating means, a means for obtaining a weight between types of feature quantities which are search viewpoints, and a feature quantity calculating means for calculating feature quantities for each type of accumulated objects and reference objects And a feature quantity storage unit for storing, as the points of the multi-dimensional vector space for each feature quantity type, the feature quantity of the object accumulated in advance among the calculated feature quantities, The feature quantity of the object that has been accumulated in advance In the multidimensional vector space of, the index generation / storage means for calculating the index required for data retrieval and storing the index, the reference object and each accumulated object for each type of feature Regarding the distance in the multi-dimensional vector space between the numbers, the number of the objects in the vicinity of the reference object obtained by using the index is determined for each feature amount type by using the weight between the feature amount types, and the determined number A neighbor object search means for finding nearby objects and a neighbor object found for each feature quantity type are combined for all feature quantity types, candidates for similar objects are created, and the weight is added to the candidates to determine the degree of similarity. A similarity calculation unit that obtains the candidate objects, and uses the candidate objects in order according to the similarity. There has means for determining the final result to fit the specified number, and object presenting means for presenting the final results to the user.

[0023]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of a similar object retrieval apparatus according to an embodiment of the present invention, and FIG. 2 is a flow chart showing its operation.

The similar object search apparatus according to the present embodiment uses digitized object data Ob1 to Ob.
an object data storage unit 10 for storing m, a feature amount calculation unit 11 for calculating the feature amounts of the objects Ob1 to Obm, a feature amount / index storage unit 12 for storing the feature amount and an index separately generated, A reference object input unit 131 for a person to newly input a reference object from the outside, a weight designation unit 132 for a user to designate a weight that is a viewpoint for measuring the similarity between objects, and a user Object number designating section 133 for designating the number of objects obtained as
4 including an interface unit 13 with the user, an index generation unit 14 that generates an index of the multidimensional vector space for each type of feature amount, and a reference object in the multidimensional vector space for each type of feature amount. On the other hand, in response to a search request from a nearby object search unit 15 that finds a nearby object at a distance in the multidimensional vector space, the feature quantity of the reference object is divided into feature quantity types, and A search request is passed to the neighboring object search unit 15 for each type, and the returned results are collectively set as a candidate object. For each candidate object, if necessary, a distance from the missing reference object for each feature type is calculated. And add the weight to calculate the similarity,
It is composed of a coordinator unit 16 that returns the results as similar objects from a specified number of upper objects, and a temporary data storage unit 17 that temporarily stores intermediate results.

The index is an auxiliary data structure for searching for a point at a specific position in the multidimensional space or for searching for a point near the specified position at high speed. Typically, there are R-tree and k-dtree. These indexes are searched by creating a hierarchical data structure according to the position of the point and manually manipulating it from the original (root node). In a familiar example, the mesh is cut vertically and horizontally on a road map (two-dimensional space), and this tourist attraction is vertically D horizontally 3, so it can be said that this index is also a kind of index.

Next, the operation of this embodiment will be described with reference to the flowchart of FIG.

Characteristic quantities Ob11, O of the object Ob1
b12 ,. ． ． , Ob1n, feature quantities Ob21, Ob22 ,. ． ． , Ob2n ,. ． ． , Obm1, Obm, which are the feature quantities of the object Obm
2 ,. ． ． , Obmn (n is the number of types of feature quantity) are calculated in advance by the feature quantity calculation unit 11 and stored in the feature quantity / index storage unit 12. Each object O
b1 to Obm themselves are stored in the object data storage unit 10. Further, in the multidimensional vector space of each feature type, an index is generated by the index generation unit 14 for searching. This index is stored in the feature quantity / index storage unit 12.

First, the user uses the reference object input unit 1
31 from the weight designation unit 132 to the weights w1 ,. ． ． , Wn to the object number designation unit 133
Enter the number K of objects. The weights w1 to wn and the number of objects K use default values when there is no input from the user (step 21). Next, the feature quantities VR1 ,. ． ． , VRn are calculated by the feature amount calculation unit 11 (step 22).

Next, the coordinator unit 16 specifies the i-th type feature quantity VRi of the reference object and the number (K and w1, w2, ... , Wn) f (K,
w1, w2, ..., wn) (however, the number is K or more),
The search precision e is specified, and the near object search unit 15 is requested to search for near objects (step 23). At this time, as f (K, w1, w2, ..., Wn), for example, wi × f (K) can be considered.

At this time, although the retrieval accuracy e varies depending on the index used, for example, the reference S. Arya,
DM Mount, NS Netanyasu, R. Silverman, and
AYWu, “An Optimal algorithm for approximate ne
arest neighbor searching, “5th Ann. ACM-SIAM Sympo
ε in sium on Discrete Algorithms, 1994, pp.573-582. The present invention lowers the search accuracy for a feature amount type having a small weight and searches the neighborhood. That is,
The value of the precision e is set to a value of 0 to 1, and when it is 1, it is completely the same as the true neighborhood. In this case, as a method of determining e, the e is larger (closer to 1) as the weight wi is larger. For example, wi is weight w
When normalized as a whole of 1, ..., Wn, wi is 0 to
It takes a value of 1, but the value of e is 0.5 + wi / 2 or sq.
It is possible to use a value of r (wi) (where sqr (x) is the square root of x).

Neighboring object retrieval unit 15 that has received the request
Is in the multidimensional vector space for each feature type i,
F (K) objects that are close in space to the point corresponding to the reference object are obtained using the index and returned to the coordinator unit 16 (step 24).

The coordinator unit 16 that has returned the neighboring objects collects the returned neighboring objects for all the feature quantity types and creates a candidate object group (step 25). The candidate object group is obtained, for example, as a union of neighboring objects. In this case, the number of candidate objects is maximum f (K) × n (when there is no overlap between neighboring objects for each feature amount type) and minimum f (K) (when the neighboring objects are exactly the same for each feature amount type). become.

Next, regarding the candidate object Obk,
Point V corresponding to the reference object for each feature type
If the distance dki between Ri and Obki (i is the type of feature quantity) is not found, the shortfall is calculated (step 2
6).

For each candidate object Obk, VR
1 and Obk1 have a distance dk1 ,. ． ． , VRn and Obk
When the distance dkn from n is obtained, the weight w
1 ,. ． ． , Wn are added to calculate the similarity (step 27). For example, w1 / dk1 + ... + wn / dkn
To calculate the similarity.

Next, the candidate objects are arranged in descending order of similarity, and the top K pieces are returned as a result (step 28).

When the result is returned from the coordinator unit 16, the object presenting unit 134 presents the result to the user (step 29).

In the above, for the feature quantity type for which the weight is designated as 0, the neighborhood object search and distance calculation can be omitted. Further, as the weight, not only the weight designated by the user is directly used, but also the weight obtained by some method from the designated weight may be used. Further, steps 24 and 26 can perform processing in parallel for each type of feature quantity.

According to another embodiment of the present invention, the distance calculation in step 26 is also performed in the step of obtaining the neighboring object in step 24. In this case, step 26 is unnecessary.

Referring to FIG. 3, a similar object retrieval apparatus according to another embodiment of the present invention comprises an input device 31, storage devices 32 to 34, an output device 35, a recording medium 36 and a data processing device 37.

The input device 31 corresponds to the reference object input unit 131, the weight designation unit 132, and the presentation object number designation unit 133 in FIG. The storage devices 32, 33, and 34 correspond to the object data storage unit 10, the feature quantity / index storage unit 12, and the temporary data storage unit 17 in FIG. 1, respectively. The output device 35 is the object presenting unit 1 in FIG.
Equivalent to 34. The recording medium 36 is a recording medium such as a floppy disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, etc., and records the processing shown in FIG. 2 as a similar object search program. Data processing device 37
Is a CPU of a computer, which reads a similar object search program from the recording medium 36 and executes it.

[0042]

As described above, according to the present invention, the neighboring objects of the reference object are narrowed down by using the index for each kind of the feature amount, candidates of the similar objects are made from these neighboring objects, and the candidates for the similar objects are obtained. Since distance calculation and similarity calculation by weight are performed,
It is possible to search for similar objects with arbitrary weights without having to calculate distances for all objects, and reduce the number of searches for feature types that do not significantly affect the overall accuracy. By increasing the number of searches for the feature type that greatly affects the accuracy of, it is possible to reduce the time required for the entire search without lowering the overall accuracy.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a similar object search device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing an operation of the similar object search device of FIG.

FIG. 3 is a configuration diagram of a similar object search device according to another embodiment of the present invention.

[Explanation of symbols]

10 Object data storage 11 Feature calculation unit 12 Feature quantity / index storage 13 Interface part 131 Reference Object Input Section 132 Weight designation section 133 presentation object number designation part 134 Object presentation unit 14 Index generator 15 Neighborhood object search section 16 Coordinator section 17 Temporary data storage 21-29 steps 31 Input device 32-34 storage device 35 Output device 36 recording media 37 Data processing device

Continuation of the front page (56) References JP-A-10-154149 (JP, A) K. Curtis et al., A Comp rehensive Image Simility Retrieval System system theattributes multiple Multiple Vectors in, Information Processing Society of Japan, Research Report (97-DBS-113), March 16, 1997, Japan Society for Society, 1997, Japan Society for Incorporated. Vol. 97, No. 64, pp. 167 to 172, Continuation of article titles, High Dimensional Space Sail Arya et al., An optimal algorithm near east neighbor learning, 5th. , ACM-SIAM Symposium on Discrete Algorithms, USA, Society for Industrial and Applied Mathematics, 1994, pages 573 to 582, Kazuhiko Kushima et al. Japan Telecommunications Association, February 1, 1998, VOL. 10 NO. 2, Pages 78 to 80 Masayoshi Umeda et al., Request for query language in image retrieval processing, IPSJ research report (database system), Japan, Information Processing Society of Japan, January 20, 1998, VOL ． 98 No. 2, pages 1 to 5 (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06F 17/30 JISST file (JOIS)

Claims (15)

(57) [Claims] 1. A similar object retrieval method for presenting to a user a similar number to a reference object, which is an object designated by a user, out of a large number of accumulated objects and presenting the same to the user. And calculate the features of each type of accumulated object,
It is the stage of storing the points of the multi-dimensional vector space for each feature type and calculating the index required for data retrieval in the multi-dimensional vector space of each feature type, and the search viewpoint. The steps of obtaining the weights among the types of feature quantities, the steps of calculating the feature quantities of the reference object input by the user for each type, and the steps of comparing the reference object and each accumulated object for each type of feature quantity Regarding the distance in the multi-dimensional vector space between, the number of objects that are close to the reference object using the index is determined for each feature type using the weights between the feature types, and only the determined number is determined. The neighborhood object search step for finding the neighboring objects, and the neighborhood object found for each feature quantity type for all feature quantity types. A similar object candidate is created by summarizing, the similarity calculation step of calculating the similarity by adding the weight to the distance between the candidate and the reference object, and the candidate objects are ordered according to the similarity, and the user A similar object search method comprising: obtaining a final result in accordance with a designated number or a default value; and presenting the final result to a user. 2. The method of claim 1, wherein in the neighborhood object searching step, a search accuracy for obtaining a neighborhood object for each feature quantity type is determined for each feature quantity type by using a weight between the feature quantity types. Similar object search method. 3. The number of objects in the neighborhood is calculated as
The feature quantity type with a small weight is reduced, and the feature quantity type with a large weight is increased.
The method described. 4. The method according to claim 2, wherein the search accuracy is lowered for the feature quantity type having the smaller weight. 5. The neighborhood object searching step calculates a distance of each feature type of a neighborhood object from a reference object in a multidimensional vector space, and collects the neighborhood objects obtained from all the feature types as candidates. At the time when the object is obtained, the distance from the reference object in the multidimensional vector space of the feature type for which the distance has not been obtained for each candidate object is calculated. The similar object search method according to claim 1, wherein the similarity is calculated by adding a weight to the distance from the reference object for the candidate object. 6. The neighborhood object search step does not calculate the distance of each feature type of the neighborhood object from the reference object in the multidimensional vector space, but collects the neighborhood objects obtained from all the feature types. At the time of obtaining the candidate objects, the distance in the multidimensional vector space of each feature type is calculated for each candidate object, and then, in the similarity calculation step, weights are added to the distances from the reference object for all candidate objects. 5. The similar object search method according to claim 1, wherein the degree of similarity is calculated by calculating the similarity. 7. In the neighboring object searching step, for the searched neighboring objects in the multidimensional vector space of each feature amount type, only the distance from the reference object in the feature amount of the type of interest at that stage is used. 5. The distance from the reference object is calculated for all types of feature amounts, but the distance calculation is marked once for each object so that the distance calculation does not overlap. The similar object search method according to item 1. 8. A similar object retrieval apparatus for extracting, from a large number of accumulated objects, objects similar to a reference object, which is an object designated by a user, in a descending order of similarity and presenting them to the user. And object data storage means for storing object data, reference object input means for the user to input the reference object, and object number designating means for the user to specify the number of similar objects desired as a result. A means for obtaining a weight between types of feature quantities which is a search point of view, a feature quantity calculating means for calculating a feature quantity for each accumulated object and reference object, and the calculated feature quantity Among the features, the features A feature quantity storage unit for storing as a point of a multidimensional vector space for each quantity type, and a multidimensional vector of each feature quantity type for a feature quantity of an object accumulated in advance among the calculated feature quantities In the space, index generation / storing means for calculating the index required for data retrieval and storing the index, and a large number of objects between the reference object and each accumulated object for each type of feature With respect to the distance in the dimensional vector space, the number of neighbor objects to the reference object using the index is determined for each feature quantity type using the weights between the feature quantity types, and the determined number of neighbor objects are obtained. Summarizes the search object and the neighboring objects found for each feature type for all feature types. Make candidates of similar objects, the similarity calculation means for calculating the similarity in consideration of the weight for that candidate, in accordance with the degree of similarity, and orders the candidate object,
A similar object search device having means for obtaining a final result in accordance with a number designated by a user or a default value , and object presenting means for presenting the final result to the user. 9. The neighborhood object search means determines the search accuracy when obtaining the neighborhood object for each feature quantity type, for each feature quantity type using weights between the feature quantity types. Similar object retrieval device. 10. The neighborhood object search means according to claim 8, wherein the number of objects in the neighborhood is calculated to be small for the feature quantity type with the small weight and large for the feature quantity type with the large weight. apparatus. 11. The apparatus according to claim 9, wherein the neighboring object search means reduces the search accuracy for the feature quantity type with the smaller weight. 12. The neighborhood object search means calculates a distance from a reference object in a multidimensional vector space of each feature quantity type of the neighborhood object, and collects the neighborhood objects obtained from all the feature quantity types as candidates. At the time of obtaining the object, the distance from the reference object in the multidimensional vector space of the feature amount type for which the distance has not been obtained for each candidate object is calculated, and then the similarity calculating means The similar object search device according to claim 8, wherein the similarity is calculated by adding a weight to a distance between the candidate object and the reference object. 13. The neighborhood object search means collects the neighborhood objects obtained from all the feature quantity types without calculating the distance of each feature quantity type of the neighborhood object from the reference object in the multidimensional vector space. At the time of obtaining the candidate objects, the distance in the multidimensional vector space of each feature type is calculated for each candidate object, and then the similarity calculation means weights the distances with respect to the reference object for all candidate objects. The similar object search device according to claim 8, wherein the similarity is calculated by calculating the similarity. 14. The neighborhood object means not only measures the distance from the reference object in the feature quantity of the type of interest at that stage for the searched neighborhood objects in the multidimensional vector space of each feature quantity type. , Calculate the distance from the reference object for all types of features, but mark each object once it has been calculated so that the distance calculations do not overlap,
The similar object search device according to any one of claims 8 to 11. 15. The method according to any one of claims 1 to 7.
The similar object search program for causing the computer to execute the similar object detection method of is recorded ,
Computer-readable recording medium.