JP2009294855A - Similar data retrieval system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce the load of disk scanning by arranging feature value vectors so that featured value vectors whose similarity is high can be closely gathered on a secondary storage, and that the order of the number of closely gathering data can be turned into O(N<SB>c</SB>) depending on the number N<SB>c</SB>of all data. <P>SOLUTION: In the case of registering new data, the prescribed number of clusters having a cluster average whose inter-vector distance with a featured vector extracted from new data is close is retrieved as a proximity cluster, and optimization by a k-means method with the member of the retrieved proximity cluster and new data as an object. When the number of members of the cluster exceeds the upper limit, the cluster is divided into two part, and the newly created cluster is added to the group of approximate clusters. The update of the cluster management is performed by making energy functions have the relation of the adjacent clusters and information showing where each cluster should be arranged in the whole cluster array. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a similar data search system, and to a data arrangement optimization method on a computer in similar data search.

  The amount of images and videos that can be accessed by the user is increasing day by day as the network becomes broadband and the storage capacity increases. In such a flow, a problem arises as to how to access a desired one from a vast amount of image / video content. In the past, keyword search was made possible by manually or semi-automatically adding text information to the content to solve this problem. However, this is possible only for a part of a large amount of content, and there is also a problem that information possessed by an image cannot be accurately represented by text information alone.

  Therefore, the information that the image itself, such as the color distribution and shape of the image, is automatically extracted, and the search is performed by evaluating the similarity between images based on the image feature amount expressed as high-dimensional vector information. Image retrieval techniques have been proposed. In a similar vector search process, which is a main process of similar image search, a search for data serving as a search key and data having a short distance in the vector space, that is, a neighborhood search in the vector space is performed. For example, in a similarity search of still images, a color distribution histogram or shape pattern in an image is used as a feature vector that characterizes data.

  In the neighborhood search, it is necessary to know the distance between vectors. If the number of target data for distance calculation is N and the number of dimensions of the feature vector is M, N distance calculations are required between the search key data and the target data for distance calculation, and each distance calculation is required. Is proportional to M. Therefore, when the neighborhood search is realized by a linear search, a calculation time proportional to N × M is required. Therefore, when a large-scale high-dimensional data is to be searched, enormous calculation time is required, so that it is essential to speed up the nearest neighbor search process.

  As a technique for speeding up the nearest neighbor search process, a plurality of methods for narrowing the target data for distance calculation according to the search query data have been proposed. A group of techniques collectively referred to as multidimensional indexing is an extension of the concept of balanced trees used in the field of databases to multidimensional space processing. In multidimensional indexing, regions in space are managed in a tree structure. When the search query data is given, the search is performed with the order of leaf Log N in which the area including the search query data is defined. On the other hand, in the field of pattern recognition, speeding up based on clustering is often used in which objects that are close to each other are classified in advance. As a specific classification method, the k-means method is common.

  In general database systems, feature vectors are often stored in secondary storage. Therefore, it is necessary to retrieve similar vectors while performing I / O on the feature vector in the secondary storage as needed. However, since the I / O on the secondary storage requires much time compared with the I / O on the primary storage, the target feature amount even if the distance calculation between the feature amount vectors is accelerated. If the access speed to the vector is slow, it makes sense.

  In order to improve the I / O speed of data on the secondary storage, the following method has been proposed.

  In general, a feature vector having high similarity is highly likely to be referred to at the same time when searching and when updating search target data. Therefore, it is considered that the load of scanning the disk can be reduced if the feature quantity vectors having high similarity are arranged as close as possible to each other on the secondary storage.

  In general, disk access is performed for all data in a certain range at a time. Therefore, if similar data are gathered in the vicinity of the secondary storage, the data that was accessed when the search was performed but was not displayed as the final search result is read into the disk cache. Become. Therefore, since similar data is read into the disk cache when searching using the similar search key data a plurality of times, the number of disk accesses is reduced, and the search speed can be increased. Searching a plurality of times using similar search key data is likely to occur when an image displayed on the search result screen is newly used as search key data.

  Therefore, the secondary storage medium is regarded as a one-dimensional array, similar feature quantity vectors are arranged in the vicinity on the one-dimensional arrangement, and dissimilar feature quantity vectors are arranged in distant areas on the one-dimensional arrangement. It is considered that high-speed disk access can be achieved by performing an appropriate data arrangement process.

  However, with existing tree structures, hierarchical clustering, and the like, similar feature quantity vectors cannot be arranged in the vicinity on a one-dimensional array. For example, in a tree structure, when a tree structure is constructed with each cluster as a leaf node in the lowest layer, it can be said that the clusters existing under each node are similar. However, the tree structure is an abstract structure that prescribes only the hierarchical relationship of nodes, and does not realize a relationship in which similar feature quantity vectors are arranged in the vicinity.

  Patent Document 1 proposes a technique for mapping feature vectors onto a one-dimensional array using a self-organizing map that is a kind of neural network while preserving the distance in a high-dimensional vector space. . Here, the self-organizing map uses the feature of arranging data in a low-dimensional space while maintaining the neighborhood relation in a high-dimensional space as much as possible. If the number of units in the output layer of the self-organizing map is k, all search target feature vectors are divided into k clusters, and similar clusters are arranged in the vicinity on a one-dimensional array. The similarity between clusters is defined by the distance between reference vectors of the clusters. And all the search object feature-value vectors are stored in the nearest cluster. At this time, the reference vector of each cluster is held in the memory, and the feature vector belonging to each cluster is stored in a continuous area on the secondary storage.

  In Patent Document 2, a cluster in which similar feature vectors are created using a clustering method based on the k-means method is created, and data arrangement is performed so that similar clusters are arranged in a neighboring region on a one-dimensional array. Process. Only the representative value of the cluster is held on the primary storage, and the feature vector is stored on the secondary storage. Here, an average value of feature quantity vectors included in the cluster is referred to as a cluster representative value, and is hereinafter referred to as a cluster average. Further, in order to continuously secure storage areas on the secondary storage of each cluster member, a predetermined amount of storage areas is secured at the time of cluster generation. Accordingly, the maximum number of each cluster member is also limited.

  However, if clustering of all feature quantity vectors is performed each time a new feature quantity vector is registered, it is impossible to perform registration processing in a practical processing time. Therefore, the following sequential clustering is performed. When a new data is registered, a predetermined number of clusters having a cluster average whose vector distance is close to the new feature vector extracted from the new data are searched as neighboring clusters, and the feature vector and new feature included in the searched neighboring cluster Perform optimization by k-means method only for quantity vectors. Further, an upper limit is set for the number of members of each cluster, and when the upper limit is exceeded, the cluster is divided into two, and the newly generated cluster is added to the set of adjacent clusters. Therefore, as the number of registered data increases, the number of clusters also increases.

Then, using the energy function E ₁ shown below, similar clusters are arranged in the vicinity region on the one-dimensional array.

Equation (1) is an energy function E ₁ for defining a cluster position rearrangement procedure. N _c represents the number of clusters, and v _i represents an average vector of clusters at the i-th position. This energy function is defined as the sum of the squared distances of the cluster averages at successive positions.

However, positions of both ends, i.e., the boundary condition for the first position and the N _c-th cluster defines one side only. Specifically, the distance between the cluster averages of the non-existing 0th position cluster and the first cluster is 0, and the non-existing N _c + 1st position cluster and the Nth position cluster average The distance between them is also zero. At this time, the amount of change in the energy function when the cluster at the i-th position and the cluster at the j-th position are exchanged is calculated by Expression (2). However, j> i.

  If the position of the cluster in the adjacent cluster set that is the current update target is updated so that the energy function decreases based on the amount of energy change described above, the distance between the clusters existing at adjacent positions in the array can be determined. A relatively small state can be realized.

  By repeating the above process every time new data is registered, similar clusters are arranged in the vicinity region of the one-dimensional array.

JP 2004-46612 A JP 2007-334402 A

  The conventional data placement optimization method that aims to place similar vector data in a nearby region on a storage medium has the following problems.

  When high-dimensional vector data is mapped onto a one-dimensional array using a self-organizing map, it is not possible to flexibly cope with the addition of new data. This is because, when mapping using a self-organizing map, a large amount of processing time is required, and therefore it is practically impossible to perform remapping every time new data is added. For example, when a person in an image captured by a surveillance camera is a search target, since a human image to be searched is sequentially generated, it is necessary to register new data sequentially. Will have to be remapped.

  When high-dimensional vector data is arranged on a one-dimensional array using an energy function defined as the sum of square distances of vector data at successive positions, feature vectors with large similarity are adjacent to each other. It is possible to arrange to some extent. However, the similarity between clusters whose distances on the one-dimensional array are more than a certain distance is randomly equal regardless of the distance on the one-dimensional array. In other words, this method is effective only for clusters within a certain range on the one-dimensional array. Further, since the order of the number of clusters gathered in the vicinity is constant (O (1)) regardless of the total number of clusters, the number of clusters gathered in the vicinity becomes relatively small as the number of clusters increases. Therefore, there arises a problem that locally optimized cluster groups are scattered in a plurality of locations. This is because the cluster position in the entire one-dimensional array becomes indefinite because the energy position is calculated from only the distance between the nearest clusters and the cluster position is determined.

Therefore, in order to improve the I / O speed of data on the storage medium, the present invention increases the order of the number of data gathered in the vicinity so as to become O (N _c ) depending on the total number of data N _c. An object of the present invention is to provide means for arranging dimensional vector data on a one-dimensional array.

  In order to avoid the indefinite cluster position in the entire cluster array when high-dimensional vector data is arranged on a one-dimensional array (hereinafter, this one-dimensional array is referred to as a cluster array) using an energy function, In addition to the relationship between adjacent clusters, information on where each cluster should be arranged in the entire cluster array needs to be added to the energy function. For this purpose, the cluster information similar to the neighborhood area of the cluster array is gathered using the distribution information of the feature vector, and the dissimilar clusters are arranged in a distant area in the cluster array.

  At the time of new data registration, sequential clustering is performed using the method of Patent Document 2. Search for a specified number of clusters that have a cluster average that is close to the feature vector extracted from the new data as adjacent clusters, and target only the feature vectors and new feature vectors that are included in the searched neighboring clusters Then, the optimization by the k-means method is executed. Further, an upper limit is set for the number of members of each cluster, and when the upper limit is exceeded, the cluster is divided into two, and the newly generated cluster is added to the set of adjacent clusters. Therefore, as the number of registered data increases, the number of clusters also increases.

When the number of clusters N _c is small (for example, N _c <4), the cluster arrangement is updated using the same energy function E ₁ as in Patent Document 2 for the update target cluster. In addition to this, a set of cluster average and cluster position is stored as cluster information. Here, the cluster position is an array element number on the cluster array.

When the number of clusters N _c increases (for example, N _c ≧ 4), the square of the inter-vector distance between the current cluster average and the stored past cluster average is added to the energy function E ₁ . Specifically, first, attention is paid to a past cluster arrangement at a certain point in time for past clusters stored. Next, the cluster number in the focused past cluster array is converted so that the minimum value is 1 and the maximum value is N _c . Then, the cluster in the past cluster array having the cluster number of the cluster in the current cluster array and the cluster number after the closest conversion is obtained, and the square of the inter-vector distance of the cluster average of those clusters is added to the energy function. To do. Further, the cluster information is stored as in the case where the number of clusters _Nc is small.

  In the present invention, a cluster having a cluster representative value with a close distance in the vector space with the feature vector extracted from the newly registered data is searched, and the data included in the newly registered data and similar clusters are classified into a plurality of clusters, An upper limit is set for the number of members of each cluster. When the upper limit is exceeded, the corresponding cluster is divided, and the feature vector included in each cluster is stored in a continuous area of the storage medium. Are managed as cluster numbers, cluster representative values that are vector data are calculated from feature vectors included in each cluster, cluster representative values and cluster numbers of each cluster are stored, and clusters of clusters having specific numbers The class of the cluster with a specific number is calculated by calculating the inter-vector distance 1 between the representative value and the cluster representative value of the preceding and following clusters. The distance 2 between vectors between the representative value and the cluster representative value of the stored cluster is calculated, and the feature amount vector included in the cluster is stored so that the sum of the distance 1 between the vectors and the distance 2 between the vectors becomes small. By exchanging the storage positions on the medium, clusters having similar cluster representative values are stored in the vicinity area of the storage medium.

According to the data arrangement optimization method in the similar data search according to the present invention, the storage medium is regarded as a one-dimensional array, and similar data on the storage medium is arranged by arranging similar data in a neighboring region on the one-dimensional array. I / O speed can be improved. Further, the number of similar data gathered in the vicinity region is O (N _c ) depending on the total number of data N _c .

  Hereinafter, as one embodiment of the present invention, data arrangement processing in a similarity search system for images will be described.

  FIG. 1 is a diagram showing a system configuration example of a similar data search system according to the present invention. The server computer 110 on which the search engine operates is connected to the client computer 130 on which the application program operates via the communication infrastructure 120, and provides services such as search to the client computer 130. The communication infrastructure 120 is a network (for example, an IP network) that connects the server computer 110 and the client computer 130. The server computer 110 includes at least an interface (I / F) 151, a CPU 152, a memory 153, and a secondary storage such as a hard disk 154 that are connected to each other.

  The I / F 151 is connected to the communication infrastructure 120 and is used for communication between the server computer 110 and the client computer 130. The CPU 152 is a processor that executes a program stored in the memory 153. The memory 153 is a storage device that stores a program executed by the CPU 152 and data referred to by the CPU 152. The memory 153 according to the present embodiment is a so-called main memory device, for example, a semiconductor memory device that can be randomly accessed. The memory 153 of this embodiment stores at least a server program and data for realizing the search server process 111.

  The hard disk 154 is a storage device including one or more hard disk drives (HDD). The hard disk 154 according to the present embodiment stores information about the feature vector of the image stored in the image server 140 as feature data 114 and cluster management information 115. Note that the hard disk 154 of this embodiment may be replaced with an optical disk device, a semiconductor storage device such as a flash memory, or any other type of storage device.

  The feature amount vector is obtained by digitizing the feature of an image stored in the image server 140 as vector data. The feature vector can be calculated by various conventionally known methods. For example, what is called an edge pattern feature amount can be used. In order to extract the edge pattern feature quantity, first, a plurality of characteristic edge patterns are set in advance. Then, the face image is divided into a grid pattern, and the number of edge patterns included in each region is counted. A feature quantity vector is obtained by generating a histogram from the number of edge patterns and making it a multidimensional vector. Further, composition division may be performed on the entire region of the search target image in a grid pattern, and a multidimensional vector created from the color histogram of each divided region may be used as the feature amount vector. Further, a combination of these multidimensional vectors may be used as the feature vector.

  In the present embodiment, each feature vector is classified into one of a plurality of clusters. It is desirable that feature vectors that are close to each other are classified into the same cluster. The feature amount vector may be classified into clusters by any method, but in the present embodiment, it is classified by the k-means method.

  The feature amount data 114 includes a set of a data ID that identifies one or more images classified into one cluster and a feature amount vector of the image data identified by the ID. Note that the images and feature vectors classified into each cluster are also referred to as cluster members.

The cluster management information 115 includes a set of a cluster ID for identifying each cluster and a representative value of the cluster identified by the cluster ID. Current information and information when the total number of clusters _Nc is a specific number are stored. In the present embodiment, cluster information is stored when the number of clusters N _c is a power of two.

  In the present embodiment, the representative value of each cluster is an average vector (cluster average) of feature quantity vectors included in each cluster. A value other than the average vector may be used as the representative value of the cluster. For example, a feature vector of cluster members close to the average vector may be used, or a seed value specified when performing optimization by the k-means method may be used. As a result of optimization by the k-means method, each feature vector is included in a cluster including a representative value closest to the feature vector.

  The image server 140 is a computer that includes a storage device that stores image data and is connected to the communication infrastructure 120. When image data is accessed from the client computer 130 or the server computer 110 through the communication infrastructure 120, input / output of image data is performed. The image server 140 may be included in the server computer 110 or the client computer 130.

  The search server process 111 in the server computer 110 manages search targets that are clustered (that is, classified into clusters). When the system is operating, the cluster management information 115 is expanded as the cluster management information 112 in the memory 153 of the server computer. As each cluster information 113, each cluster ID, an average vector which is a representative value of the cluster identified by the ID, a data ID string for identifying cluster members, and the like are stored. The feature vector of each cluster member is managed on the hard disk 154 collectively as feature data 114. For this reason, as the cluster information 113 in the memory 153, the position on the hard disk that stores the feature vector of each cluster member is further stored.

  The cluster management information 112 is a copy of the cluster management information recorded on the hard disk 154. Therefore, when an update to the cluster occurs, not only the cluster management information 112 but also the cluster management information 115 on the hard disk 154 is updated. However, the cluster management information 115 is not directly referenced in the search process.

  The client computer 130 is a computer connected to the communication infrastructure 120. Although two client computers 130 are shown in FIG. 1, the similar data search system may include any number of client computers 130. If the server computer 110 has a function equivalent to that of the client computer 130, all processing may be performed by the server computer.

  The client computer 130 may be a computer having any configuration. FIG. 1 shows a configuration of a typical client computer 130. That is, the client computer 130 of FIG. 1 includes a CPU 131, a memory 132, an I / F 133, an input device 134, and an output device 135.

  The CPU 131 is a processor that executes a program stored in the memory 132. The memory 132 is a storage device that stores programs executed by the CPU 131. The I / F 133 is an interface that is connected to the communication infrastructure 120 and is used for communication between the client computer 130 and the server computer 110. The input device 134 is a device that receives input from the user of the client computer 130. The input device 134 may include, for example, a keyboard, a pointing device, an image scanner, or the like. The output device 135 is a device that displays information to the user of the client computer 130. Specifically, for example, an image acquired as a result of the similar data search is displayed on the output device 135. The output device 135 is an image display device such as a CRT or a liquid crystal display.

  Next, processing at the time of similar data search in the search system of the present embodiment will be described. FIG. 2 is a block diagram showing an execution unit of processing executed when searching for similar data in the embodiment of the present invention. As shown in the figure, the similar data search processing according to the present embodiment includes an image input unit 210, an image feature amount extraction unit 211, a similar cluster search unit 212, a similar feature amount search unit 213, and a search result output unit 214. Is done.

  In the image input unit 210, search target image input processing is performed. Here, the search target image is image data in which an object or person to be searched is copied, or image data in which a texture to be searched is drawn. These image data are supplied by user drawing using a digital camera, an auxiliary storage device, a mouse, or the like.

  The image feature amount extraction unit 211 extracts an image feature amount from the search target image. In this embodiment, what is called an edge pattern feature amount is used as the image feature amount. In order to extract the edge pattern feature quantity, first, a plurality of characteristic edge patterns are set in advance. Then, composition division is performed on the image data in a grid pattern, and the number of edge patterns included in each region is counted. An image feature quantity is created by generating a histogram from the number of edge patterns and using it as a multidimensional vector. In addition, a color histogram may be used as the image feature amount.

  The similar cluster search unit 212 uses the cluster ID and the cluster average of the cluster management information 112 to search for a predetermined number of clusters having a cluster average that is close to the image feature vector extracted from the search target image and the vector space. To do.

  The similar feature quantity search unit 213 uses the feature quantity vector of the feature quantity data 114 to extract the image feature extracted from the search target image from among the cluster members included in the similar cluster acquired by the similar cluster search unit 212. Search for data having feature vectors close to the vector in the vector space.

  The search result output unit 214 acquires the image data of the data searched by the similar feature amount search unit 213 from the image server 140 and outputs it as a search result. Note that the data ID may be displayed as a search result without using the image server 140.

  Here, the image feature quantity extraction unit 211, the similar cluster search unit 212, and the similar feature quantity search unit 213 are realized as a part of the search server process 111 by a program developed in the memory 153 of the server computer 110. Executed.

  Next, processing at the time of new data registration in the similar data search system of the present embodiment will be described. FIG. 3 is a block diagram showing an execution unit of processing executed when new data is registered in the embodiment of the present invention. As shown in the figure, the new data registration process of the present embodiment includes an image input unit 310, an image feature amount extraction unit 311, a similar cluster search unit 312, a clustering unit 313, a cluster division unit 314, and an energy function calculation unit 315. , A data storage unit 316.

  The image input unit 310 performs input processing for a newly registered image. The image data is supplied by user drawing using a digital camera, an auxiliary storage device, a mouse, or the like. The image feature amount extraction unit 311 extracts image feature amounts for registered images in the same manner as the image feature amount extraction unit 211. The similar cluster search unit 312, clustering unit 313, and cluster dividing unit 314 perform sequential clustering using the method of Patent Document 2.

  In the similar cluster search unit 312, as in the similar cluster search unit 212, an image feature amount vector extracted from a newly registered image and a cluster having a short distance in the vector space using the cluster ID and cluster average of the cluster management information 112. A predetermined number of clusters having an average are searched. Note that the number of clusters obtained as a search result may be different from that of the similar cluster search unit 212. The clustering unit 313 acquires only the feature quantity vectors of the cluster members included in the similar clusters acquired by the similar cluster search unit 312 from the feature quantity data 114, and only the feature quantity data and the feature quantity vectors of the newly registered image are acquired. Clustering is performed on the target based on the k-means method.

  The cluster division unit 314 performs cluster division. A maximum number is set in advance for the number of cluster members included in the cluster, and a cluster whose number of cluster members exceeds the maximum value is divided into two. Since a new cluster is generated by this process, the number of clusters increases as the number of registered data increases. When a cluster is generated, a predetermined amount of disk area is secured in order to physically and continuously secure storage areas on the hard disks of the members of each cluster. The maximum number of cluster members is limited by the amount of disk area secured at this time.

  The energy function calculation unit 315 calculates energy based on an energy function described later. In the data storage unit 316, the feature quantity vector clustered by the clustering unit 313 is stored in the feature quantity data 114. At this time, the energy function, which will be described later, is stored at a position where it becomes the smallest. Further, when the number of clusters is a specific number, the cluster IDs and cluster averages of all the clusters are stored in the cluster management information 112.

  Here, the image feature quantity extraction unit 311, the similar cluster search unit 312, the clustering unit 313, the cluster division unit 314, the energy function calculation unit 315, and the data storage unit 316 are searched by a program developed in the memory 153 of the server computer 110. It is realized as a part of the server process 111, and its processing is executed.

  FIG. 4 is a flowchart showing processing executed on the server when new data is registered in the embodiment of the present invention. Here, processing executed by the similar cluster search unit 312, the clustering unit 313, and the cluster dividing unit 314 will be described in detail.

  The process shown in FIG. 4 is executed as part of a server program that implements the search server process 111. Therefore, the process shown in FIG.

  When the new data x to be registered is given, the CPU 152 first searches for neighboring clusters and acquires a set C * of neighboring clusters (S410). Specifically, the CPU 152 compares the average vector of each cluster with the new data x, and selects a predetermined number of clusters in order from the cluster represented by the average vector whose distance is close to the new data x. Get as *. Next, the CPU 152 adds the new data x to the nearest cluster c * (that is, the cluster represented by the average vector closest to the new data x) in the set C * of adjacent clusters (S420). .

  Next, the CPU 152 initializes the parameter t and the parameter i to “0” and “1”, respectively (S421, S422). The parameter t is used to count the number of iterations of the k-means method update. The parameter i is used to indicate a cluster included as an element in the set C * of neighboring clusters. Thereafter, an optimization loop based on the k-means method shown in step 430 and subsequent steps is entered.

First, the CPU 152 determines whether or not the number of cluster members exceeds the limit M _max for each cluster that is an element of the set C * of adjacent clusters. M _max is the maximum number of members in the cluster. Specifically, CPU 152 determines in step 430 whether parameter i is equal to or less than the number of elements in set C *. If it is determined in step 430 that the parameter i is less than or equal to the number of elements in the set C *, the CPU 152 targets the cluster c that is the i-th element of the set C * (S434), and the number of members of the cluster c is It is determined whether or not M _max is exceeded (S431). At the time of entering the optimization loop, it is assumed that the cluster other than the nearest cluster c * to which the new data x is added does not exceed the member number limit.

If the number of members of the closest cluster c * exceeds M _max , the CPU 152 divides the cluster into two (S432), and adds the newly generated cluster d to the elements of the neighboring cluster set C * (S433). ). Various methods can be considered as a method of dividing a cluster into two. In the present embodiment, the principal axis is obtained with respect to the vector distribution in the cluster, and by determining which side the projection of the vector of each member onto the principal axis is on the projection side of the cluster average vector, Divide into clusters.

After step 433 is executed, the process of the CPU 152 returns to step 431. In step 431, it is determined whether or not the number of members of the cluster after the division exceeds _Mmax . If it is determined that the number of members of the cluster after the division exceeds M _max , the process proceeds to step 432 in order to further divide the cluster. On the other hand, if it is determined that the number of members of the cluster after the division does not exceed M _max , the CPU 152 adds 1 to the value of the parameter i in order to execute the determination of step 431 for the next cluster ( S435), the process returns to step 430.

If it is determined in step 430 that the parameter i exceeds the number of elements of the set C *, it is confirmed that the number of members of all the clusters that are elements of the set C * is within M _max . In this case, the CPU 152 checks the number of optimization iterations t by the k-means method (S440).

In this system, the optimization shown in FIG. 4 is intended only for a subset of clusters, and does not mean optimization for the entire cluster. Further, it is assumed that the addition of data is repeatedly performed thereafter, and optimization is executed each time. Therefore, it is not necessary to place extreme importance on optimization at a certain point in time, and it is sufficient that the maximum number of iterations t _max is several times.

If it is determined in step 440 that the parameter t indicating the number of iterations is equal to or greater than the maximum number of iterations t _max , optimization by the k-means method has been performed a predetermined number of times, and the processing in FIG. 4 ends. Alternatively, if it is determined in step 440 that the set C * has not changed, it is considered that it is not necessary to perform further optimization. Therefore, also in this case, the processing of FIG.

On the other hand, if it is determined in step 440 that the parameter t indicating the number of iterations is smaller than the maximum number of iterations t _max and that the set C * is changing, it is necessary to perform cluster optimization. The CPU 152 updates the set C * by the k-means method (S450).

The processing in step 450 is the same as the normal k-means method. That is, all data included in the adjacent cluster is distributed to the cluster having the closest cluster average at that time. As a result, the members of the adjacent clusters and the cluster average are updated, and the process returns to Step 430. At this time, 1 is added to the value of the parameter t indicating the number of iterations in step 451. Unlike the point in time when the optimization loop is entered, this time, if the cluster state changes greatly, there is a possibility that a plurality of clusters may exceed the upper limit of the number of members. Moreover, since it is not sufficient to divide into two, there is a possibility that division may be necessary again, or a newly generated cluster may exceed the upper limit. For this reason, the CPU 152 proceeds to step 440 after performing processing (step 430 to step 435) so that the number of members of all clusters is equal to or less than the upper limit M _max . In step 440, if the number of iterations reaches the maximum number t _max , or if there is no change in the state of the set C * of neighboring clusters, the process ends.

  With the above processing, the number of clusters increases as the number of registered data increases. The new cluster is added when the number of members in the cluster exceeds the maximum number of cluster members, so the frequency is about once every (maximum number of cluster members). The series of processing shown in FIG. 4 is executed in the work area on the memory 153, and the set of adjacent clusters is updated.

  Next, a data arrangement process in the similarity search system will be described. In general, a cluster having high similarity is highly likely to be referred to at the time of update and search. Therefore, if the clusters having high similarity are arranged as close to each other as possible on the secondary storage medium, the load on the disk scanning should be reduced. Therefore, the secondary storage medium is regarded as a one-dimensional array, similar clusters are arranged in the vicinity on the one-dimensional array, and dissimilar clusters are arranged in a distant area on the one-dimensional array.

When the number of clusters N _c is small (for example, N _c <4), the cluster arrangement is updated using the same energy function E ₁ as in Patent Document 2 for the update target cluster.

When the number of clusters N _c increases (for example, N _c ≧ 4), the cluster arrangement is updated using the energy function E ₂ shown in Expression (3).

Equation (3) is an energy function E ₂ for defining a cluster position rearrangement procedure. N _c is the number of clusters, v _i is the average vector of the clusters at the i-th position, and u _lq is the cluster at the q-th position when the number of clusters is m to the power of 1 (m and l are natural numbers). Represents the mean vector. Hereinafter, as an example, a case where m = 2 is described. p is the maximum integer that satisfies the equation (4), and the order of p is O (log (N _c )). Moreover, q satisfies Formula (5). Note that u _lq is stored in the cluster management information 115.

This energy function E ₂ includes a term representing the sum of square distances of cluster averages at successive positions, and the past cluster average and current cluster obtained as a result of sequential clustering and stored in the cluster management information 115. It is defined as the sum of terms representing the sum of the mean square distances. Specifically, first, attention is paid to a past cluster arrangement at a certain point in time for past clusters stored. Next, the focused cluster number in the past cluster array is converted using Equation (5) so that the minimum value is 1 and the maximum value is N _c . Then, the cluster in the past cluster array having the cluster number of the cluster in the current cluster array and the cluster number after the closest conversion is obtained, and the square of the inter-vector distance of the cluster average of those clusters is added to the energy function. To do.

However, positions of both ends, i.e., the boundary condition for the first position and the N _c-th cluster defines one side only. Specifically, the distance between the cluster averages of the non-existing zeroth-position cluster and the first cluster is 0, and the non-existing _Nc + 1-th position cluster and the _Nc- th position cluster are clusters. The distance between the averages is also 0.

In the energy function E _2, the larger the similarity between i-th cluster and i-1-th cluster and i + 1-th cluster, the first half of the term is reduced, the i-th cluster corresponding to past cluster mean position The second half of the term becomes smaller.

  At this time, the amount of change in the energy function when the cluster at the i-th position and the cluster at the j-th position are exchanged is calculated by Expression (6). However, j> i.

  The position of the cluster in the adjacent cluster set that is the current update target is updated so that the energy function decreases based on the energy change amount. When the number of clusters is increased by the sequential clustering process, it is assumed that the new cluster is arranged at the end of the cluster arrangement, and the cluster arrangement is rearranged.

  Further, a cluster number that is a unique serial number is assigned to each updated cluster so as to correspond to the order of the cluster arrangement, and is managed by the cluster management information 112 together with the cluster average. The number assigned at this time is the cluster number of the cluster included in the set C * of the nearest clusters acquired in step 410. In addition, when a cluster is divided because the upper limit of the number of cluster members is exceeded, new cluster numbers are assigned by the increased number of clusters.

  Finally, when the number of clusters is a specific number, a set of cluster averages and cluster numbers of all clusters is stored as cluster information. In this embodiment, it is assumed that the number of clusters is 2 to the l-th power (where l is a natural number).

  Hereinafter, the reason why a term representing the sum of the square distances of the current cluster average and the past cluster average obtained as a result of sequential clustering is added to the energy function will be described in detail.

First, an ideal cluster where similar clusters gather in the neighborhood of the cluster array and dissimilar clusters are arranged in distant areas in the cluster array holds for all clusters. Consider a cluster arrangement. FIG. 5 shows the relationship between the distance between cluster average vectors in the vector space and the distance on the cluster array for each cluster included in the ideal cluster array. Here, the distance on the cluster array represents the difference between the array element numbers of the cluster array. Further, FIG. 6 shows a hierarchical structure of an ideal cluster arrangement in which the relationship shown in FIG. 5 is established. Here, as an example, the case where the number of clusters N _c is 12 is shown. The bottom rectangle (630) represents the cluster average of the cluster arrangement in the ideal state. The average value (610, 620) of the upper layer cluster average represents the average value of the cluster average included in each region when the lower layer cluster arrangement is divided into a plurality of regions. Here, the average values of the cluster averages of the respective layers have the relationship shown in FIG. 5 as in the lowermost cluster arrangement, and such a relationship is considered to be established hierarchically.

  Here, it is assumed that the average value of the cluster average obtained from the cluster arrangement in the ideal state is almost equal to the optimal clustering result obtained by dividing the entire image feature amount distribution into the same number as the average number of cluster averages. be able to. For example, if the average number of cluster averages is four, the average value of the cluster averages at that time is considered to be equal to the cluster average obtained from the clustering result obtained by dividing the entire vector space of the image feature vector into four. I can lose weight. This is because the average value of the cluster average in the ideal state appropriately divides the cluster arrangement according to the distribution of the image feature amount.

  Therefore, if the average value of the cluster average can be known in advance, a cluster having a cluster average whose distance is close to the average value of each cluster average is set as an area corresponding to the average value of each cluster average. Can be collected. This corresponds to adding an intervector distance between the cluster average and the average value of the cluster average to the energy function. By this operation, it is considered that the approximate position in the entire cluster arrangement can be determined when the cluster is actually arranged.

  However, since the structure shown in FIG. 6 is in an ideal state, the average value of the cluster average is essentially unknown. Therefore, it is necessary to obtain the average value of the cluster average using some means.

  Therefore, attention is focused on the sequential clustering process using the k-means method described above. In the sequential clustering process, the entire clustering is performed by repeatedly performing partial clustering when new data is sequentially registered. In addition, since the upper limit of the number of cluster members is determined in advance, the number of clusters similarly increases as the number of data increases.

  The cluster average series obtained here can be regarded as a fine approximation from a state in which the entire distribution of image feature values is roughly approximated as the number of registered data increases and the number of clusters increases. In other words, in the feature quantity distribution, there are many cluster averages in areas where there are many image feature quantities, and conversely there are only a few cluster means in areas where there are not many image feature quantities. I can say that. This can be considered to be equivalent to discretely representing the entire feature amount distribution by vector quantization in which the occurrence probability of the image feature amount in each divided region is constant. Therefore, assuming the randomness of the data to be registered, the cluster average obtained as a result of the sequential clustering can be considered to give an approximate value of the result of clustering all image feature amounts.

  Therefore, the cluster average obtained as a result of sequential clustering that divides the entire image feature distribution into the same number as the average number of cluster averages in the ideal state is approximated to the average value of the cluster average in the ideal state. Can be used as

  From the above, if a term representing the sum of the square distance of the cluster average and the past cluster average obtained as a result of the sequential clustering is added to the energy function, similar clusters are arranged in the neighborhood on the cluster array and similar. Non-clusters can be arranged in distant areas on the cluster arrangement.

  FIG. 7 is a flowchart showing cluster position rearrangement processing executed in the embodiment of the present invention. Here, processing executed by the energy function calculation unit 315 and the data storage unit 316 will be described in detail. The process shown in FIG. 7 is executed as part of a server program that implements the search server process 111. Therefore, the process shown in FIG.

  First, the CPU 152 searches for a set of clusters that maximizes the amount of energy reduction calculated by Expression (3) from the set C * of adjacent clusters that are currently updated (S710). Next, the CPU 152 determines whether or not a cluster set corresponding to the condition of step 710 has been found (S720). If no corresponding cluster set is found, the energy of the current cluster arrangement is the smallest and each cluster is considered to be optimally arranged, so the processing ends.

  On the other hand, if a corresponding cluster set is found, the CPU 152 updates the array by exchanging the position (S730), and returns to step 710 to search for the next cluster set. Finally, the cluster member feature vector is stored on the hard disk 154 in accordance with the position on the array thus obtained. Further, when the number of clusters is 2 to the power of 1 (l is a natural number) (S740), a set of cluster average and cluster position is stored as cluster information in the cluster management information 115 on the hard disk 154 (S750). The number of clusters may be stored when the number of clusters is not a power of 2 but another specific number.

  In order to clarify the difference between the cluster arrangements realized by the present invention and the technique disclosed in Patent Document 2, experiments were performed using 5 million vector data, and a cluster arrangement consisting of 12245 clusters was obtained. Similar to FIG. 5, for each cluster in the cluster array, the relationship between the distance between the cluster average vectors in the vector space and the distance on the cluster array is shown in FIG. 8, FIG. 9, FIG. This is a set of clusters having the same distance on the cluster array, a cluster average vector distance between clusters of each set is calculated, and the average value is plotted. The vertical axis represents the cluster average vector distance, and the horizontal axis represents the distance on the cluster array.

FIG. 8 shows the relationship between clusters realized by the method disclosed in Patent Document 2, and shows the case where the number of clusters N _c is 512, 2048, 8192, and 12245. FIG. 9 is an enlarged view showing only the relationship between clusters having a distance of 50 or less on the cluster arrangement.

  From the graphs of FIGS. 8 and 9, it can be seen that the average value of the distances between the cluster averages of the clusters arranged at a distance closer than about 10 on the cluster array is proportional to the distance on the cluster array. . Further, it can be seen that the average value of the distances between the cluster averages of the clusters arranged at a place where the distance on the cluster arrangement is more than about 10 is substantially constant. The fact that the average value of a plurality of data is constant can be considered that these data are distributed almost in a random state regardless of the distance on the cluster arrangement. Therefore, in the method disclosed in Patent Document 2, it is possible to arrange the clusters having high similarity so that they are adjacent to each other to some extent, but the similarity between clusters whose distances on the cluster array are more than a certain distance is It will be in the same state at random without depending on the distance. Furthermore, it can be seen from FIG. 9 that the range in which the average value of the distance between the cluster averages is proportional to the distance on the cluster arrangement is constant regardless of the total number of clusters. Therefore, it can be said that the number of clusters gathered in the vicinity decreases relatively as the total number of clusters increases.

FIG. 10 shows the relationship between clusters realized by the method shown in the present embodiment, and shows when the number of clusters N _c is 512, 2048, 8192, 12245. FIG. 11 is an enlarged view showing only the relationship between clusters having a distance of 50 or less on the cluster arrangement. This is similar to the experiment using the method of Patent Document 2, a set of clusters having the same distance on the cluster array is obtained, the distance between the cluster averages of the clusters of each set is calculated, and the average value is plotted. Is. The vertical axis represents the cluster average distance, and the horizontal axis represents the distance on the cluster array.

10 and 11, the range in which the average distance between the cluster averages is proportional to the distance on the cluster arrangement is about 100 when N _c is 512, and about 400 when N _c is 2048. It can be seen that when N _c is 8192, it is about 1600, and when N _c is 12245, it is about 2400. Therefore, it can be said that the number of clusters gathered in the vicinity is O (N) depending on the total number of clusters.

  In the above embodiment of the present invention, the case where the vector data to be searched is an image feature vector has been described as an example. However, the present invention can be applied to similar searches of any kind of vector data. Can do. An example of similar data search other than similar image search is an associative document search apparatus. The associative document search device performs a morphological analysis on a document group to be searched in advance, sets the meaning of each document as a unit vector, and expands the meaning of each document into a multidimensional vector as a sum of them, Similarity with other documents is determined by the distance of these multidimensional vectors. Therefore, it is possible to search for a document having a vector closest to the query sentence input by the user.

The block diagram which shows the structural example of the similar search system of this invention. The block diagram which shows the structural example of the similar data search process in the similar search system of this invention. The block diagram which shows the structural example of the new data registration process in the similar search system of this invention. The flowchart which shows the process performed at the time of data registration. The figure which shows the ideal relationship between the distance of cluster average vectors and the distance on a cluster arrangement | sequence about the arbitrary clusters contained in a cluster arrangement | sequence. The figure showing the structure which the cluster arrangement | sequence where the ideal relationship shown in FIG. 5 is materialized has. The flowchart which shows the process of rearrangement of a cluster position. The figure which shows the relationship between the distance of cluster average vectors and the distance on a cluster arrangement | sequence about the arbitrary clusters contained in the cluster arrangement | sequence obtained by a conventional method. The figure which expanded and showed the vicinity area | region of FIG. The figure which shows the relationship between the distance of cluster average vectors and the distance on a cluster arrangement | sequence about the arbitrary clusters contained in the cluster arrangement | sequence obtained by this invention. The figure which expanded and showed the vicinity area | region of FIG.

Explanation of symbols

110: Server computer 111: Search server process 112: Cluster management information 113: Cluster information 114: Cluster member feature data 115: Recorded cluster management information 120: Communication infrastructure 130: Client computers 131, 152: CPU
132, 153: Memory 133, 151: Interface 134: Input device 135: Output device 140: Image server 154: Hard disk 210: Image input unit 211: Image feature quantity extraction unit 212: Similar cluster search unit 213: Similar feature quantity search unit 214: Search result output unit 310: Image input unit 311: Image feature amount extraction unit 312: Similar cluster search unit 313: Clustering unit 314: Cluster division unit 315: Energy function calculation unit 316: Data storage units 610, 620: Ideal Average value of cluster averages 630 of ideal cluster arrays: Cluster average of ideal cluster arrays

Claims (4)

In a similar data search system that searches data similar to search query data from registered search target data,
A feature quantity extraction unit that extracts feature quantities from data and generates a feature quantity vector;
A clustering unit that classifies feature vectors of search target data into a plurality of clusters;
Manages the cluster number, which is a unique serial number assigned to each cluster, and the cluster representative value of each cluster, and stores the feature vector contained in the cluster in a continuous area of the storage medium, and the number of clusters is predetermined. A data storage unit that stores a cluster representative value and a cluster number of the cluster,
A similar cluster search unit that searches for clusters having cluster representative values that are close in distance in the vector space and feature vector extracted from given data;
A cluster dividing unit for dividing the cluster when the number of members of the cluster exceeds a preset upper limit;
The inter-vector distance 1 between the cluster representative value of the cluster having the specific number and the cluster representative value of the preceding and following clusters, and the vector between the cluster representative value of the cluster having the specific number and the cluster representative value of the stored cluster An energy function calculator for calculating the distance 2;
When additionally registering new data as a search target, the feature value vector of the new data extracted by the feature value extraction unit from the clusters stored in the storage medium has a cluster representative value that is close in distance in the vector space. A plurality of clusters are searched by searching for a predetermined number of clusters by the similar cluster search unit, and clustering by the clustering unit with respect to the feature amount vector of the new data and the feature amount vectors included in the searched predetermined number of clusters. The energy function calculation unit obtains the storage positions on the recording medium of the reclassified clusters so that the sum of the inter-vector distance 1 and the inter-vector distance 2 is reduced. A feature quantity vector included in the reclassified cluster is stored in a storage position. Similar data retrieval system. The similar data search system according to claim 1, further comprising a similar feature amount search unit and a search result output unit,
The similar cluster search unit searches the clusters stored in the storage medium for clusters having cluster representative values that are close in distance in the vector space with the feature amount vector extracted from the search query data,
The similar feature quantity search unit includes data having a feature quantity vector that is close in distance to a feature quantity vector extracted from the search query data from among the cluster members included in the cluster searched by the similar cluster search unit. Search for
The search result output unit outputs the data searched by the similar feature amount search unit. Data classified into a plurality of clusters assigned unique consecutive numbers as cluster numbers and their feature vectors, cluster number and cluster representative information of the current cluster, and clusters when the number of clusters is a predetermined number A storage unit that stores information on the cluster number and cluster representative value, a feature amount extraction unit, a similar cluster search unit, a clustering unit that classifies the feature amount vector of search target data into a plurality of clusters, a cluster division unit, The data storage unit, the inter-vector distance 1 between the cluster representative value of the cluster having the specific number and the cluster representative value of the preceding and following clusters, the cluster representative value of the cluster having the specific number, and the cluster representative of the stored cluster Using a similar data search system having an energy function calculation unit for calculating a distance 2 between vectors with a value, A similar data retrieval method for retrieving similar data with search query data from the registered search target data,
A step of extracting a feature quantity from the newly registered data by the feature quantity extraction unit to generate a feature quantity vector;
A predetermined number of clusters having cluster representative values that are close in distance in the vector space to the feature vector extracted from the newly registered data by the similar cluster search unit based on the current cluster information stored in the storage unit Searching for
Clustering by the clustering unit with respect to feature vectors of the newly registered data and feature vectors of cluster members included in the predetermined number of similar clusters, and classifying into a plurality of clusters;
Dividing the cluster by the cluster dividing unit when the number of members of the cluster exceeds the preset upper limit;
The feature vector included in the plurality of clusters classified by the clustering unit is stored on the storage medium so that the sum of the inter-vector distance 1 and the inter-vector distance 2 is reduced by the energy function calculation unit. Replacing the storage position;
The data storage unit stores the feature vector included in the cluster in a continuous area of the storage medium, and each time the number of clusters reaches a predetermined number, the cluster representative value and the cluster number of the cluster are stored in the storage unit. A method for retrieving similar data, comprising the step of storing. The similar data search method according to claim 3, wherein the similar data search system further includes a similar feature amount search unit and a search result output unit,
Searching for a cluster having a cluster representative value that is close in distance in the vector space with the feature vector extracted from the search query data from the current clusters stored in the storage medium by the similar cluster search unit;
A step of searching for data having a feature vector close to a distance in a vector space with a feature vector extracted from the search query data from among the cluster members included in the searched cluster by the similar feature search unit; ,
And a step of outputting the data searched by the similar feature amount search unit by the search result output unit.

Images