JP2011034254A - Similar search device, similar search system and similar search method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve high speed similar search by using a few hardware resources. <P>SOLUTION: A client terminal includes: a feature quantity for registration selection part for storing feature quantity data and representative data information showing which of the feature quantity data is representative data and a distance table showing similarity or distance between each representative data and each feature quantity data into a database, and for selecting one of the feature quantity data; a similarity calculation part for calculating the similarity or distance between the selected feature quantity data and the feature quantity for search; a determination part for determining whether or not the selected data and the extracted feature quantity are similar based on the calculated similarity or distance; and a processing part for performing processing corresponding to the determination result of the determination part. The feature quantity for registration selection part alternately selects the representative data and non-representative data from among the feature quantity data for non-registration by using the distance table. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for searching for data similar to multimedia data.

  Retrieving multimedia data similar to digital data such as images, moving images, music, and documents is called a similarity search. Similarity search generally extracts information called feature values used for calculating similarity or distance from multimedia data (hereinafter referred to as raw data) to be extracted, and shows the similarity between feature values. This is performed by regarding a thing with a large degree or a thing with a small distance indicating a degree of mismatch between feature quantities as similar.

  As a similar definition, depending on the use of similarity search, for example, among the multimedia data registered in a database or the like, the data having the largest feature amount with the feature amount of the multimedia data received as input is selected. There are cases where it is defined as similar, and cases where a plurality of pieces of data having a feature amount greater than a certain threshold are defined as similar.

  When the total number of feature quantities registered in the database or the like as registration targets (hereinafter referred to as registration feature quantities) is N, the similarity calculation is performed N times when the similarity is calculated for all registration feature quantities. is required. In general, since a large time cost is required for similarity calculation, if the number N of registration feature amounts increases, the search takes time in proportion thereto.

  On the other hand, a technique for reducing the number of times of calculating the similarity by calculating the similarity between the registration feature quantities in advance and efficiently searching for the registration feature quantity using the registration feature quantity has been proposed. (Patent Document 1).

  In paragraphs 0011 to 0017 of Patent Document 1, registration data is classified into a plurality of groups, and a matrix (hereinafter referred to as a matching degree table) in which the degree of matching (similarity) between registered data is described for each group is prepared. In addition, there is disclosed a technique for efficiently searching for registered data by using them at the time of search and reducing the number of verification degree calculations.

  Also, in Non-Patent Document 1, M representative registration feature amounts (hereinafter referred to as pivots) are selected from N registration feature amounts, and the features of each pivot and each registration feature amount are selected. A matrix describing the distance in the quantity space (hereinafter referred to as a distance table) is prepared, and using this, several feature values for registration are calculated while calculating the distance between the input data and each pivot. After the process of pruning to exclude from the selection target, obtaining the distance, or pruning is performed for all the pivots, the registration feature amount other than pivot (hereinafter referred to as non-pivot) is used. On the other hand, a technique called LAESA is described that reduces the number of distance calculations in the similarity search by obtaining the distance to the input data.

  As another similar search technique, an N × N matrix describing the distance between N pieces of registered data is prepared, and at the time of search, search and pruning of registered data are efficiently performed using them. A technique called AESA that reduces the number of times of distance calculation has been proposed in the past, and a survey has been conducted on these similar search techniques (Non-patent Document 2).

  Non-Patent Document 2 p. In 78-80, the number of times of calculating the distance in AESA is constant with respect to the total number N of registered data, that is, O (1), and the number of times of calculating the distance in LAESA described in Non-Patent Document 1. Is M + O (1).

JP 2007-286970 A

L. Mico et al. "An algorithm for finding nearest neighbors in constant average time with a linear space complexity," Proc. ICPR 1992, pp. 557-560 (1992) P. Zezula et al. , Similarity Search-The Metric Space Approach, Springer (2006)

In Patent Document 1, when the number of groups is G, the size of the matching degree table is O (N ² / G). Therefore, if the number of groups G is constant, the size of the collation degree table becomes O (N ² ), and the size of the collation degree table increases dramatically as the number of registered data N increases. There is a problem that hardware resources are required.

  On the other hand, when the number of groups G is increased in order to reduce the size of the matching level table, the number of times that the matching level is calculated increases in proportion thereto. If the number of groups G is 1 in Patent Document 1, this is equivalent to the case where no pruning process is performed in the above-described AESA. If the pruning process is not performed, the number of times of calculation may increase. Therefore, when the number of groups G is 1 in Patent Document 1, the number of times of calculating the similarity is O (1) or more.

  When the number of groups G is increased, the number of times of O (1) or more is calculated for each group search, so that the total number of times of calculation of O (G) or more, that is, a search time is required. Therefore, for example, when the group number G is increased in proportion to the increase in the registration feature quantity N, the size of the similarity table becomes O (N), which is the same order as the size of the distance table in AESA. There is a problem that the number of times of calculating the degree of similarity increases in proportion to the number N of registration feature amounts.

  In addition, since the number of times the distance is calculated in LAESA described in Non-Patent Document 1 is M + O (1), when the registration feature quantity N is large, the number of distance calculations in LAESA is the registration feature in Patent Document 1. This is far less than the number of similarities calculated when the group number G is increased in proportion to the quantity number N.

  However, in LAESA described in Non-Patent Document 1, a process of obtaining a distance or performing pruning is performed on all pivots, and then a distance from non-pivot is obtained. In this case, if the similar registration feature quantity is non-pivot, it is necessary to obtain the distance for all pivots or to perform pruning in order to obtain the distance to the non-pivot. There was room for improvement in reducing the number of times of calculating the distance.

  In a large-scale similar search system such as a similar image search service operated by a company, it is conceivable that the registration feature quantity N is enormous. For example, in biometric authentication for entrance / exit management, it is required to perform a similarity search in a short time using limited hardware resources.

  Therefore, it is desired to provide a similar search technique that reduces necessary hardware resources and shortens search time.

  The present invention is a similarity search device for searching for data similar to multimedia data, a storage unit for storing a plurality of registration feature data, means for acquiring feature data extracted from the multimedia data, Calculate the similarity or distance in the feature quantity space between the registration feature quantity selection unit for selecting one of the registration feature quantity data, the selected registration feature quantity data, and the extracted feature quantity A determination unit that determines whether the selected registration feature value data is similar to the extracted feature value based on the calculated similarity or distance, and a determination result by the determination unit A processing unit that performs processing.

  The storage unit further includes representative data information indicating which pivot (representative data) is among the plurality of registration feature value data, and the similarity in the feature value space between each pivot and each registration feature value data, or The registration feature value selection unit alternately stores pivot and non-pivot (non-representative data) for each predetermined number from the registration feature value data not yet selected. , If each voivot stored in the storage unit is selected using the similarity or distance in the feature amount space of each registration feature amount data, and the result of determination by the determination unit is that they are not similar, the determination unit Whether a similarity determination is made or not, a series of processes in which the feature amount selection unit selects registration feature amount data and the similarity calculation unit calculates similarity or distance and performs determination. And repeating until a determination with respect to the registration feature data all is done.

  According to the present invention, a high-speed similarity search is possible using a small number of hardware resources.

It is a block diagram which illustrates functional composition of a first embodiment of the present invention. It is a block diagram which illustrates the hardware constitutions of 1st, 2nd, 3rd embodiment of this invention. It is a flowchart which illustrates the registration process of 1st, 2nd embodiment of this invention. It is a flowchart which illustrates the pre-processing of 1st embodiment of this invention. It is a flowchart which illustrates the search process of 1st embodiment of this invention. It is the schematic which illustrates a distance table in case distance satisfy | fills symmetry. It is the schematic which illustrates a distance table in case distance does not satisfy symmetry. It is the schematic which illustrates the distance vector between qe, and the distance vector between pe. It is the schematic which illustrates the distance vector between qp, and the distance vector between np. It is a block diagram which illustrates functional composition of a second embodiment of the present invention. It is a flowchart which illustrates the pre-processing of 2nd, 3rd embodiment of this invention. It is a flowchart which illustrates the search process of 2nd embodiment of this invention. It is a block diagram which illustrates functional composition of a third embodiment of the present invention. It is a flowchart which illustrates the search process of 3rd embodiment of this invention.

  Hereinafter, embodiments will be described in detail with reference to the drawings.

  Hereinafter, the first embodiment will be described with reference to the drawings. The similarity search system according to the present embodiment allows a user who attempts authentication (hereinafter referred to as an “authentication user”) to input biometric information into the client terminal, and the system searches for similar biometric information data from a database in the client terminal. Is a biometric authentication system that identifies who is (or is not) any of users registered in the database (hereinafter, registered users) and performs authentication based on the result.

  For example, there is a method of using the feature amount similarity by multiplying the distance (dissimilarity) in the feature amount space by -1. Hereinafter, “distance” means a distance in the feature space. Further, as a criterion for determining whether two feature amounts are similar, the distance itself may be used instead of the similarity. In this case, the magnitude relationship is reversed from that in the case of using the similarity, and those having a small distance are regarded as similar.

  The client terminal includes pivot information indicating which pivot is the registration feature value data registered in the database, and a distance table indicating the distance between each pivot and each registration feature value data in the feature amount space. In addition, pivot and non-pivot are alternately selected for each predetermined number using a distance table, and data similar to the input biological information data is searched.

  As a result, even when the capacity of the memory or HDD is limited, a high-speed similarity search for a large number of registration feature data is possible.

  Also, the registration feature value data for calculating the distance from the feature value extracted from the input biometric information data (hereinafter referred to as the search feature value) is based on the magnitude of the correlation between the feature values calculated using the distance table. Select based on.

FIG. 1 shows a configuration example of the similarity search system of this embodiment. In the present embodiment, the raw data is digitized biological information.
This system includes a registration terminal 100 that transmits registration information acquired from a user to a server terminal, a server terminal 200 that stores registration information, and a client terminal 300 that performs a similarity search based on raw data for search input by the user. And a network 400. There may be one client terminal or a plurality of client terminals. The network 400 may use a network such as a WAN or a LAN, communication between devices using USB or IEEE1394, or wireless communication such as a mobile phone network or BlueTooth (registered trademark).

  For example, in the case of a company entrance / exit management system, the registration terminal 100 is installed in a registration center (such as a room reserved for registration) and a biometric information input sensor, and the client terminal 300 is installed near the door of the room. The door controller and the biometric information input sensor, the server terminal 200 may be a server placed in the server room, and the network 400 may be an in-house intranet. In the case of a company attendance management system, the registration terminal 100 is a PC and a biometric information input sensor placed in a registration center, the client terminal 300 is a PC and a biometric information input sensor installed in a room, and the server terminal 200 is a server. The server and the network 400 placed in the room may be configured as an in-house intranet.

  In these examples, a group ID 210 and a group ID 310, which will be described later, may be set to have different values for each room. By grouping in this way, the number N of registration feature amounts in one group can be reduced, and the number of times of calculating the distance can be reduced correspondingly, thereby enabling a faster search. Further, the group ID 210 and the group ID 310 may be omitted.

  The registration terminal 100 includes an ID acquisition unit 101 that acquires a group ID and a registration data ID, a raw data acquisition unit 102 that acquires raw data, and a feature amount extraction unit that extracts a registration feature amount from the registration raw data 103 and a communication I / F 104. The ID acquisition unit 101 is realized using a keyboard or the like. In this embodiment, the raw data acquisition unit 102 is realized using a biological information input sensor. Possible types of biometric information to be acquired include fingerprints, irises, voiceprints, veins, and the like. The feature amount extraction unit 103 may be on the server terminal 200 side. For example, in the case of a fingerprint, the feature amount to be extracted is extracted from a fingerprint image. In the case of an iris, an iris code (iris) created from an iris image. A bit string called “code” can be considered.

  The server terminal 200 includes a pivot determining unit 201 that determines M pivots from the N pieces of registration information 220 for each group ID 210, and a feature amount for each pivot and a feature amount for each registration feature amount data for each group ID 210. A distance table creation unit 202 that calculates a distance between the communication I / F 203 and a database 204 is provided. The registration feature quantity number N and the pivot number M may be different for each group ID 210. As the distance between two feature quantities, for example, when a minutia is used as a feature quantity, the distance obtained by the minutia matching method in which matching is performed using the position of each feature point and the direction of the ridge, the iris code is In the case of the feature amount, a Hamming distance between codes can be considered.

  The database 204 holds master data 205 for each group ID 210. The group ID 210 may be a method of setting the ID of a room where the user goes in and out. The master data 205 includes a group ID 210, registration information 220 for each registered user, and auxiliary information 230. The registration information 220 includes a registration data ID 221, raw data 222, and feature amount data 223. The registered data ID 221 may be a user employee number.

  Further, the raw data 222 may be omitted. However, if there is the raw data 222, when the feature quantity extraction unit 103 is upgraded, the latest feature quantity can be extracted from the raw data 222, so that there is an advantage that it is not necessary to re-register the raw data. The auxiliary information 230 includes the pivot information 231 that describes which of the feature amount data 223 of the registration information 220 is a pivot, and the distance that describes the distance between each pivot and the feature amount data 223 of each registration information 220. And a table 232.

  The client terminal 300 calculates a correlation between a raw data acquisition unit 301 that acquires raw data, a feature amount extraction unit 302 that extracts a search feature amount from the search raw data, and a search feature amount and pivot. The distance between the search feature quantity-pivot correlation calculation unit 303, the search feature quantity-non-pivot correlation calculation unit 304 that calculates the correlation between the search feature quantity and non-pivot, and the search feature quantity. Selection based on the calculated distance, a registration feature selection unit 305 that selects the registration information 320 to be calculated, a similarity calculation unit 306 that calculates the distance between the search feature and the feature data 323 of the registration information 320 The distance between the determination unit 307 that determines whether the feature amount data 323 of the registered information 320 is similar to the search feature amount, and the search feature amount have not yet been obtained, and pruning is also performed. Against non pivot and non-pivot, it comprises a pruning unit 307a that performs pruning process, a communication I / F308, a database 309.

  The correlation calculation by the correlation calculation units 303 and 304 is performed for the following distance vectors. A vector composed of the distance between the search feature quantity and the searched all registration feature quantity data is constituted by the distance between the “q-e distance vector” and pivot and the searched all registration feature quantity data. The vector consisting of the distance between the search feature value and the distances of all the searched pivots is the “q-p distance vector”, the non-pivot, A vector composed of a distance from pivot is defined as an “n-p distance vector”. You may use what was defined as a vector other than the above.

  The search feature quantity-pivot correlation calculation unit 303 calculates the correlation between the q-e distance vector and the p-e distance vector, and the search feature quantity-non-pivot correlation calculation unit 304 calculates the correlation between q-p. The correlation between the distance vector and the n-p distance vector is calculated. Details of the calculation of the correlation will be described later. The search feature quantity-pivot correlation calculation unit 303 and the pruning unit 307a may be omitted.

  The database 309 includes a group ID 310, registration information 320 for each registered user, and auxiliary information 330. The registration information 320 includes a registration data ID 321, raw data 322, and feature amount data 323. The raw data 322 may not be present. The auxiliary information 330 includes pivot information 331 that describes which of the feature quantity data 323 of the registration information 320 is a pivot, and a distance that describes the distance between each pivot and the feature quantity data 323 of each registration information 320. Table 332.

  The registration information 320 and the auxiliary information 330 are the same as the registration information 220 and the auxiliary information 230 of the master data 205 of the server terminal 200 having the same group ID. For example, in the entrance / exit management system, when the client terminal 300 is installed near the door of a room, the group ID 210 indicating the room is linked to the client terminal 300, and the same group ID is assigned. It is conceivable to obtain the registration information and auxiliary information possessed from the server terminal 200. By adopting such a configuration, users who are allowed to enter and leave the room can be managed for each room. In addition, each client terminal 300 stores only the data to be authenticated among the data registered in the system, and thus can be realized by an apparatus having a smaller capacity HDD and memory.

  The registration information 320 and the auxiliary information 330 are acquired, for example, when the client terminal 300 downloads from the server terminal 200 via the network by the communication I / F 308 at night when the system is not used by the user.

  In addition to the above-described functional units, a processing unit that performs processing according to the use of the system based on the determination result of the determination unit 307 is provided. For example, when the client terminal 300 includes a door controller, a processing unit that controls the door lock based on the determination result of the determination unit 307 is provided. Further, in the attendance management system, when the client terminal 300 includes a PC, the registration data ID associated with the feature amount data 323 determined to be similar is authenticated according to the determination result of the determination unit 307. A processing unit or the like is recorded as a log in the database along with the time when the operation is performed.

  FIG. 2 shows a hardware configuration of the registration terminal 100, the server terminal 200, and the client terminal 300 in the present embodiment. These terminals can include a CPU 500, a memory 501, an HDD 502, an input device 503, an output device 504, a communication device 505, and a portable storage medium 507 reading device 506.

  The input device 503 of the registration terminal 100 is, for example, a keyboard and a biological information input sensor, and the output device 504 is a display. The input device 503 of the server terminal 200 is a keyboard, for example, and the output device 504 is a display as well. The input device 503 of the client terminal 300 is, for example, a keyboard and a biological information input sensor, and the output device 504 is a display.

  Each functional unit included in the registration terminal 100, the server terminal 200, and the client terminal 300 described above is realized by each program that the CPU 500 reads out from the HDD 502 to the memory 501 and executes the program. Each program may be stored in advance in the HDD 502 in each terminal, or may be introduced into the HDD 502 from another device via a medium that can be used by each terminal as necessary. The medium refers to, for example, a storage medium 507 that can be attached to and detached from the reading device 506, or a communication medium such as a network, a carrier wave that propagates through the network, and a digital signal.

  FIG. 3 shows a registration processing procedure and data flow in this embodiment. The registration process described below is realized by a program that the CPU 500 of the registration terminal 100 reads out from the HDD 502 to the memory 501 and executes. And this program is comprised from the code | cord | chord for performing the various operation | movement demonstrated below.

First, the ID acquisition unit 101 of the registration terminal 100 receives a group ID and a registration data ID as input by a user operation (step S101).
The raw data acquisition unit 102 of the registration terminal 100 acquires raw data for registration by a user operation (step S102).
The feature extraction unit 103 of the registration terminal 100 extracts a registration feature from the registration raw data (step S103).
The registration terminal 100 transmits the group ID, the registration data ID, the raw data for registration, and the extracted feature data for registration to the server terminal 200 via the network 400 by the communication I / F 104 (step S104).
The server terminal 200 adds the received registration data ID, registration raw data, and registration feature value data as registration information 220 to the master data 205 having the same group ID 210 as the group ID received from the registration terminal 100. (Step S105).

  When the server terminal 200 includes the feature quantity extraction unit 103, in step S104, the client terminal 300 transmits only the registration ID and raw data, and the received feature quantity extraction unit 103 of the server terminal 200 receives the feature quantity extraction in step S103. What is necessary is just to process.

  FIG. 4 shows a processing procedure and data flow of pre-processing in the present embodiment. This process is performed between the registration process and the search process. For example, a method of performing at night on the day of registration can be considered. The pre-processing shown below is realized by a program that the CPU 500 of the server terminal 200 reads from the HDD 502 to the memory 501 and executes. And this program is comprised from the code | cord | chord for performing the various operation | movement demonstrated below.

  In the database 204 of the server terminal 200, if there is master data 205 to which the registration information 220 has been added for each group ID 210 since the previous pre-processing, the pivot determining unit 201 determines that N in the master data 205. A process of selecting M pieces as pivots from the feature amount data 223 of the pieces of registration information 220 is performed (step S201). The value of M is set in advance by, for example, an operation of a system administrator. The details of the pivot determination method will be described later.

  The distance table creation unit 202 of the server terminal 200 calculates the distance between each pivot determined by the pivot determination unit 201 and the feature amount data 223 of each registration information 220 in the same master data 205, and creates a distance table. (Step S202).

FIGS. 6A and 6B show examples of distance tables. Here, X ₁ , X ₂ ,..., X _M represent pivot, Y ₁ , Y ₂ ,..., Y _N-M represent non-pivot. For example, in FIG. 6 (b), a distance _d (Y _1, X 2) = 27 from the distance _{d (X} 2, Y 1) = 25, _{Y 1} from _{X 2} to _{Y 1} to _{X 2} . When the distance satisfies the symmetry (that is, when d (A, B) = d (B, A)), either one of the distance information may be stored as shown in FIG. Conversely, when the symmetry is not satisfied, both distance information is stored as shown in FIG. An example of the distance that satisfies the symmetry is a Hamming distance. Examples of distances that do not satisfy symmetry include distances obtained by the minutia matching method.

  In the similarity search system of this embodiment, the number of elements in the distance table is 2MN-M (M-1) when an asymmetric distance measure is used, and MN-M (M-M when a symmetric distance measure is used. 1) / 2, and in both cases, the size of the distance table is as small as O (N). Therefore, an increase in the size of the distance table accompanying an increase in the number N of registration feature quantities can be suppressed, and an effect of reducing the capacity of the database and the required free memory capacity can be obtained.

  For example, in the case of an entrance / exit management system or the like, since the door controller corresponding to the client terminal 300 is generally very limited in the capacity of the memory and HDD, the application of this embodiment is considered to be particularly useful.

  The pivot determining unit 201 and the distance table creating unit 202 of the server terminal 200 include the pivot information (which is the feature amount data 223 of the registration information 220 is pivot) about the M pivots determined in step S201, and step S202. The distance table obtained in step (1) is overwritten and stored in the auxiliary information 230 of the master data 205 (step S203).

  The client terminal 300 transmits the group ID 310 to the server terminal 200, and receives registration information 220 and auxiliary information 230 of the master data 205 having the same group ID 210 as the transmitted group ID from the server terminal 200. The received data is stored in the database 309 as registration information 320 and auxiliary information 330 (step S204).

FIG. 5 shows a search processing procedure and a data flow in this embodiment. The search processing shown below is realized by a program that the CPU 500 of the server terminal 200 reads from the HDD 502 to the memory 501 and executes. And this program is comprised from the code | cord | chord for performing the various operation | movement demonstrated below. The following is one embodiment, and the search processing procedure is not limited to this.
The CPU 500 of the client terminal 300 reads the registration information 320 and the auxiliary information 330 from the database 309 and develops them in the memory 501 (step S301).
The raw data acquisition unit 301 of the client terminal 300 acquires raw data for search from the user (step S302).
The feature quantity extraction unit 302 of the client terminal 300 extracts the search feature quantity from the search raw data (step S303).
The determination unit 307 of the client terminal 300 initializes the distance calculation count i to 0 and the number of pivots M ′ that have not been set to M (step S304). Here, the determination unit 307 proceeds to step S306 if the distance calculation count i is less than 2M ′, and proceeds to step S310 if it is 2M ′ or more (step S305).

In step S306, if the distance calculation count i is an even number, the process proceeds to step S307, and if it is an odd number, the process proceeds to step S309.
In step S307, in order to select one of the pivots, the search feature quantity-pivot correlation calculation unit of the client terminal 300 obtains the distance between the q-e distance vector and the search feature quantity. The correlation with the distance vector between pe in each pivot that is not pruned is calculated. Details of the method of calculating the correlation between the qe distance vector and the pe distance vector will be described later. Further, this step may not be executed.

  The registration feature quantity selection unit 305 of the client terminal 300 selects the data having the largest correlation calculated in step S307 as the target for calculating the distance among the pivots that have not yet been obtained and have not been pruned. (Step S308). When step S307 is not executed, pivot is selected according to a sequence determined in advance before the main search process. The pivot selection order may be determined randomly using, for example, a pseudo-random number generator. This order may be held in the master data 205 of the database 204, and the client terminal 300 may receive and store it together with other information in step S204, and read from the database 309 in step S301.

  On the other hand, in step S309, in order to select one of the non-pivots, the search feature amount-non-pivot correlation calculation unit of the client terminal 300 obtains the q-p distance vector and the distance yet. The correlation with the n-p distance vector in each non-pivot that has not been pruned is calculated. Details of a method of calculating the correlation between the qp distance vector and the np distance vector will be described later.

  The registration feature quantity selection unit 305 of the client terminal 300 calculates the distance from the non-pivot that has not yet obtained the distance and has not been pruned and has the largest correlation calculated in step S309. (Step S310).

The determination unit 307 of the client terminal 300 increases the distance calculation count i by 1 (step S311).
Similarity calculation unit 306 of the client terminal 300 calculates the feature quantity for search (and Q) characteristic amount data 323 of the registration information 320 and selected (the _{Z i)} the distance between d (Q, _{Z i)} (Step S312).
In step S313, the determination unit 307 of the client terminal 300 determines that they are similar if the distance d (Q, Z _i ) calculated in step S312 is smaller than the predetermined threshold τ, and the process proceeds to step S314. If it is more than threshold value (tau), it will determine with not being similar and will progress to step S315.

  If it is determined that they are similar, the client terminal 300 performs processing when the registration information 320 having the registration feature quantity data 323 similar to the search feature quantity is found (step S314).

  Depending on the use of biometric authentication, the registration information 320 may be used as a search result to end the search, or the process may return to step S305 to continue searching for other registration information 320. The former method is effective when it is desired to determine whether or not similar registration information 320 exists in as short a time as possible. The latter method is effective when it is desired to output all the registered information 320 determined to be similar or when it is desired to search for the most similar registered information 320.

  For example, in the case of an entrance / exit management system for checking whether or not an authenticated user is registered in the database, the search ends at the time of this step and the door of the locked room is unlocked. Processing can be considered. If the attendance management system is intended to ascertain as accurately as possible who the authenticated user is, the determination result is stored in the memory 501 and the process returns to step S305 to search for other registered information 320. A method of continuing can be considered.

  If the determination unit 307 determines in step S313 that they are not similar, the pruning unit 307a of the client terminal 300 has not yet obtained the distance, and the pruning process is performed on pivot and non-pivot that have not been pruned. Is performed (step S315). Details of the pruning process will be described later. Further, this step may not be executed.

The determination unit 307 of the client terminal 300 refers to the distance calculation count i and proceeds to step S317 if the distance calculation count i is the upper limit value i _max of the distance calculation count, and returns to step S305 if not i _max (step S316). ). In the present embodiment, the distance calculation number does not exceed the i _max, the worst value of the search time by setting the i _max (maximum value) is determined. Therefore, the system administrator sets a value of i _max in advance before the search so as to satisfy the condition of the time that can be searched according to the system.

  When the process proceeds to step S317 to end the search, the client terminal 300 performs processing at the end of the search. If the registration information 320 having the registration feature quantity data 323 similar to the search feature quantity is not found, the process at the time of the search failure is performed. For example, in the case of an entrance / exit management system, it is conceivable that the door is kept locked, assuming that the user is not allowed to enter / exit. In the case of the attendance management system, a message that prompts the user to re-enter biometric information may be output to the output device 504.

  When similar (one or more) registration information 320 is found, a plurality of registration information 320 may be used as a search result according to the use of the authentication system, or the registration information 320 that realizes the smallest distance among them. Only search results may be used. For example, in the case of the attendance management system, registration data IDs 321 of a plurality of registration information 320 determined to be similar are output to the output device 504, and the correct one is selected from among them, and the registration data ID 321 received as an input is a selection. Is stored in the database 309 (or 204) as a log together with the current time. Alternatively, the registration data ID 321 of the registration information 320 that realizes the smallest distance among the similar registration information 320 may be stored as a log together with the current time. The processing at the end of the search includes a processing unit that performs processing according to the use of the system.

  As described above, in the similar search system of this embodiment, as described in steps S305 to S310, first, pivot and non-pivot are alternately selected, and a distance is obtained for pivot or a branch is determined. After the process of cutting, the non-pivot distance that has not been pruned yet is obtained. By adopting such a method, it is not necessary to obtain distances for all pivots or to perform pruning in order to obtain non-pivot distances, thereby reducing the number of times of calculating distances. As a result, the effect of enabling a faster search is obtained.

  In this embodiment, pivot and non-pivot are alternately selected one by one. However, the selection rule is not limited to this, and for example, a predetermined number may be alternately selected. . For example, if two are to be selected alternately, in step S306, depending on the remainder of dividing i by 4, if 0 or 1 is selected, pivot is selected, if 2 or 3, non-pivot is selected. And so on. If two non-pivots are selected after selecting one pivot, it is determined in step S305 whether i <3M ′, and in step S306, the remainder of dividing the value of i by 3 is determined. Thus, a technique such as selecting pivot if it is 0, selecting non-pivot if it is 1 or 2, may be used as appropriate.

Hereinafter, the details of the method for determining pivot will be described. The determination of pivot follows the method described in Non-Patent Document 1. That is, pivot is determined as follows. The following processing is performed by the pivot determining unit of the server terminal 200. In other words, the CPU 500 of the server terminal 200 is realized by a program that is read from the HDD 502 to the memory 501 and executed. And this program is comprised from the code | cord | chord for performing the various operation | movement demonstrated below.
(1) Determination of First Pivot One is randomly selected from the feature amount data 223 of the N pieces of registration information 220, and is designated as the first pivot.
(2) a + 1-th _P 1 the feature amount data 223 of already determined registration information 220 selected as the pivot of the pivot of the (a> = 1), ··· , and _{P a,} characterized in the registration information 220 is not selected _{Let the} quantity data 223 be E ₁ ,..., E _N-a . At this time, _{Pa + 1} is determined as follows.

When an asymmetric distance measure is used, the following equation may be used instead of Equation 1.

  Hereinafter, a method for calculating the correlation between the q-e distance vector and the p-e distance vector by the search feature quantity-pivot correlation calculation unit 303 in step S307, and the search feature quantity -non- in step S309 The details of the calculation method of the correlation between the q-p distance vector and the n-p distance vector by the inter-pivot correlation calculation unit 304 will be described.

  FIG. 7 shows examples of q-e distance vectors and pe distance vectors, and FIG. 8 shows examples of q-p distance vectors and n-p distance vectors. The hatched portion of the distance table is registration data that has already been selected and the distance has been obtained or pruned.

Ｒ（ｑｅ，ｐｅ）＝Ｓｘｙ／（Ｓｘｘ＋Ｓｙｙ）
として求める。但し、

である。或いは、特許文献１に記載されているように、

として求めても良いし、

として求めても良い。但し、

である。数９はＤ_ｘｙとして市街地距離を用いたものと等価であるが、その他、ユークリッド距離、チェビシェフ距離、二次形式距離などの距離尺度を用いても良い。 The first distance vector is x = (x ₁ , x ₂ ,..., X _m ), and the second distance vector is y = (y ₁ , y ₂ ,..., Y _m ). . As a method of calculating these correlations, a method that follows the method described in Patent Document 1 can be considered. That is, the correlation R (x, y) is

R (qe, pe) = Sxy / (Sxx + Syy)
Asking. However,

It is. Or, as described in Patent Document 1,

You may ask as

You may ask as. However,

It is. _Equation 9 is equivalent to using city distance as D _xy , but other distance measures such as Euclidean distance, Chebyshev distance, and quadratic distance may be used.

  Further, instead of correlation between vectors as in Non-Patent Document 1, a distance between vectors may be obtained and used as it is as a selection criterion for the feature amount data 323 of the registration information 320. In this case, the selection criterion for pivot and non-pivot in step S308 or step S310 is not “the one with the largest correlation” but “the one with the smallest distance”. In Non-Patent Document 1, the Chebyshev distance is used as a distance scale, but other distance scales such as an urban area distance, an Euclidean distance, and a quadratic distance may be used.

  As the q-e distance vector, the p-e distance vector, the q-p distance vector, and the n-p distance vector, those defined as vectors other than those described above may be used. For example, as a q-e distance vector, a vector constituted by the distance between a search feature quantity and all searched non-pivots, and as a p-e distance vector, pivot and all searched You may use the vector comprised by the distance with non-pivot. In this case, since the distance between pivots is not used among the distance information stored in the distance table, the size of the distance table can be further reduced. When step S307 is not executed, the qe distance vector and the pe distance vector need not be used.

The details of the pruning process by the pruning unit 307a in step S315 will be described below. For the pruning process, a method that follows the technique described in Non-Patent Document 1 is considered. That is, the pruning process is performed as follows.
(1) Creation of Data Set for Performing Pruning Determination Create a set A ^* of feature amount data 323 of registration information 320 for determining whether or not to perform pruning. As a method of creating A ^* , for example, there are the following methods.
a) Let A ^* be the set of all non-pivots for which the distance has not yet been obtained and pruned.
b) Create A ^* in the same way as d) if distance is obtained for M / 2 or more pivots, otherwise use the same method as a).
c) Create A ^* in the same manner as d) if distance is obtained for M / 3 or more pivots, otherwise use the same method as a).
d) Let A ^* be the set of all pivots and non-pivots for which no distance has yet been obtained and pruned.
(2) One feature quantity data 323 (referred to as A) is selected from the selection A ^{* of} data to be subjected to pruning determination. Selected items are excluded from A ^* . However, if A ^* is empty, the pruning process is terminated.
(3) The distance d (Z _i , A) is obtained from the distance d ′ calculation distance table.
(4) Pruning determination _{| d (Q, Z i)} -d (Z i, A) |> = if tau, pruning and shall not selected for the similarity search for the feature amount data 323A To do. If the feature amount data 323 at this time is pivot, M ′ is set to M′−1. Return to (2).

  When the distance measure to be used satisfies the triangle inequality (such as the Hamming distance), it is guaranteed that the distance between the authentication feature quantity Q and the pruned feature quantity data 323 is τ or more. Therefore, the number of feature quantity data 323 to be distance-calculated can be reduced without accidentally pruning the feature quantity data 323 whose distance is less than τ. Thereby, the further shortening of authentication time can be implement | achieved and the effect which the convenience improves further as a result is acquired.

Although the method using the distance has been described above, the similarity search may be performed using the similarity instead of the distance. For example, instead of the distance table, a similarity table indicating the similarity in the feature space of each pivot and each registration feature value data is provided, and the correlation calculation units 303 and 304 may replace the feature vectors with each other. The correlation may be calculated using a similarity vector composed of similarities. Further, the similarity calculation unit 306 calculates not the distance but the similarity, or calculates the similarity by multiplying the calculated distance by −1, and the determination unit 307 determines the search feature amount and the registration feature. When the similarity to the quantity is greater than a predetermined threshold, it is determined that the two are similar, and the pruning unit 307a determines that the difference between the similarity between Q and Z _{i and} the similarity between Z _i and A is a predetermined threshold τ In the above case, pruning may be performed on A.

  The second embodiment will be described below with reference to the drawings. In the similarity search system of this embodiment, the authenticated user inputs biometric information, and the system searches for similar biometric information from the database in the server terminal, so that the authenticated user is someone among registered users (or who is not) ) And authenticates based on the result.

  FIG. 9 shows a configuration example of the similarity search system of the present embodiment. Here, differences from FIG. 1 will be described.

  This system includes a registration terminal 100 that transmits registration information acquired from a user to a server terminal, a server terminal 200 that stores the registration information and performs a similar search based on a search feature, and a search input that is input by a user. A client terminal 300 that transmits a feature amount extracted from raw data to the server terminal 200 and a network 400 are included. For example, in the case of a system (hereinafter referred to as an information access control system) in which information access control such as authentication at the time of PC login in a company or authentication at the time of accessing a business system is performed on the server side, the registration terminal 100 is a registration center (registration PC and biometric information input sensor placed in a room reserved for use), the client terminal 200 is a PC and biometric information input sensor in the room, the server terminal 300 is a server placed in the server room, and the network 400 is in-house An intranet configuration can be considered. In this example, the group ID 210 may be set to have a different value for each room, for each subnet, or for each client terminal 200.

  The search feature quantity-pivot correlation calculation unit 303, the search feature quantity-non-pivot correlation calculation unit 304, the registration feature quantity selection unit 305, the similarity calculation unit 306, the determination unit 307, and the pruning unit 307a are clients. The server terminal 200 is provided instead of the terminal 300. However, the pruning unit 307a may not be provided. The client terminal 300 does not have a database.

  Further, the feature amount extraction unit 103 and the feature amount extraction unit 302 may be on the server terminal 200 side as in the third embodiment. In this case, the registration terminal 100 and the client terminal 300 may transmit the raw data without transmitting the feature amount, and then extract the feature amount from the raw data received by the server terminal 200.

  The hardware configurations of the registration terminal 100, the server terminal 200, and the client terminal 300 in the present embodiment are the same as those in FIG.

  The registration processing procedure and data flow in this embodiment are the same as in FIG.

  FIG. 10 shows a processing procedure and data flow of pre-processing in the present embodiment. Steps S201 to S203 in FIG. 10 are the same as steps S201 to S203 in FIG.

FIG. 11 shows a search processing procedure and a data flow in this embodiment. Here, only differences from FIG. 5 will be described.
The CPU 500 of the server terminal 200 reads the master data 205 from the database 204 and expands it in the memory 501 (step S301). Alternatively, this processing is not performed, and instead, immediately after step S303a, the registration information 220 and auxiliary information 230 of the master data 205 having the received group ID may be read from the database 204 and expanded in the memory 501. The former has the advantage of being able to search in a shorter time, and the latter has the advantage of requiring less free memory space.

The client terminal 300 transmits the group ID and the search feature amount to the server terminal 200 (step S303a).
Steps S304 to S317 are performed not by the client terminal 300 but by the server terminal 200. The registration information 220 and the auxiliary information 230 used here are the master data 205 having the group ID received in step S303a.

  When the search is ended in step S314 and in step 317, the server terminal 200 transmits the search result to the client terminal 300. For example, in the case of authentication for the purpose of checking only whether an authenticated user is registered in the DB at the time of logging in to the client terminal 300, there is a method for transmitting access permission to the client terminal 300. Conceivable. Also, in the case of authentication for the purpose of ascertaining as accurately as possible who is the authenticated user when accessing the business system, the feature value for registration that has achieved the smallest distance that has been searched so far A method is conceivable in which a registration data ID 221 having data 223 is transmitted to the business system, and the business system transmits to the client terminal 300 that the access as the registration data ID 221 is permitted. The client terminal 300 performs processing according to the content received from the server terminal 300. For example, when it is received that the login to the client terminal 300 is permitted, the login is executed.

  Hereinafter, the third embodiment will be described with reference to the drawings. The similarity search system of this embodiment is a similar image search system in which a user inputs an image and the system searches for a similar image from a database in a server terminal.

  FIG. 12 shows a configuration example of the similarity search system of this embodiment. Here, differences from FIG. 9 will be described. In the present embodiment, the raw data is image data.

  In this system, registration information 220 is stored in a database 204, a server terminal 200 that performs a similar search based on a search feature quantity, and a client terminal that receives raw data for search from a user as input and transmits it to the server terminal 200 300 and the network 400 are comprised. For example, the server terminal 200 may be a server operated by a company, the client terminal 300 may be a home PC owned by the user, and the network 400 may be an in-house intranet.

  This system does not have the registration terminal 100 and the group ID 210. The feature amount extraction unit 302 is provided in the server terminal 200. As the feature quantity to be extracted, for example, a color histogram can be considered, and as the distance between two feature quantities, for example, a quadratic form distance can be considered.

  As the registration information 220, for example, the server terminal 200 stores an image collected from the Internet as raw data 222, assigns a registration data ID 221, and uses the feature amount extraction unit 302 to extract the feature amount data 223 from the raw data 222. A method such as extraction is conceivable. Alternatively, as in the first and second embodiments, the user may be able to register using a registration terminal.

  The hardware configurations of the registration terminal 100, the server terminal 200, and the client terminal 300 in the present embodiment are the same as those in FIG.

  The processing procedure and data flow of the pre-processing in the present embodiment are the same as those in FIG.

FIG. 13 shows a search processing procedure and a data flow in this embodiment. Here, only differences from FIG. 5 will be described.
The client terminal 300 transmits raw data for search to the server terminal 200 via the network 400 by the communication I / F 308 (step S302a).
The feature amount extraction process (step S303) is performed not by the client terminal 300 but by the feature amount extraction unit 302 of the server terminal 200.

  When the search is ended in step S314 and in step 317, the server terminal 200 transmits the search result to the client terminal 300, and the client terminal 300 outputs the search result to the output device 504. At this time, there is a method of using one or more registration raw data 222 that realizes a distance smaller than a threshold as a search result, and a method of using raw data 222 that realizes the smallest distance as a search result. . In the former case, the raw data 222 may be ranked in ascending order of distance, the raw data 222 may be transmitted together with the rank, and the client terminal 300 may display it on the output device 504.

  In addition, this invention is not limited to said each embodiment, It can change suitably in the range which does not deviate from a summary.

  The present invention can be applied to any application that performs a similar search of multimedia data such as images, moving images, music, and documents. For example, the present invention can be applied to a similar image search system, a similar moving image search system, a similar music search system, a similar document search system, an entrance / exit management system by biometric authentication, an attendance management system, a PC login system, and the like.

100 registered terminal 101 ID acquisition unit 102 raw data acquisition unit 103 feature quantity extraction unit 200 server terminal 201 pivot determination unit 202 distance table creation unit 205 master data 210 group ID
220 Registration information 221 Registration data ID
222 Raw data 223 Feature quantity 231 Pivot information 232 Distance table 300 Client terminal 303 Search feature quantity-pivot correlation calculation section 304 Search feature quantity-non-pivot correlation calculation section 305 Registration feature quantity selection section 306 Similarity calculation Part 307 determination part 307a pruning part

Claims (16)

A similar retrieval device for retrieving data similar to multimedia data,
A storage unit for storing a plurality of registration feature data;
Means for obtaining a search feature extracted from the multimedia data;
A registration feature quantity selection unit that selects one of the plurality of registration feature quantity data;
A similarity calculation unit for calculating a similarity or distance in the feature amount space between the selected feature data for registration and the feature for search;
A determination unit that determines whether the selected registration feature data and the search feature data are similar based on the calculated similarity or distance;
A processing unit that performs processing according to a determination result by the determination unit;
With
The storage unit further includes representative data information indicating which representative data is selected from the plurality of registration feature data, and similarity in the feature data space between each representative data and each registration feature data Stores degrees or distances,
The registration feature quantity selection unit is the representative data and the registration feature quantity data other than the representative data from among the unselected registration feature quantity data which is registration feature quantity data that has not yet been selected. The non-representative data is alternately selected by a predetermined number, using the similarity or distance in the feature amount space of each representative data and each registration feature amount data stored in the storage unit,
If the result of the determination is that they are not similar, the determination unit causes the registration feature amount selection unit to select registration feature amount data, causes the similarity calculation unit to calculate the similarity or distance, and A similarity search apparatus characterized by repeating a series of processes of making a determination until a similarity determination is made or a determination is made for all the registration feature data. The similarity search device according to claim 1,
Using the similarity or distance in the feature amount space of each representative data and each registration feature amount data stored in the storage unit, the correlation between the search feature amount and the non-representative data is a first correlation. A correlation calculation unit that calculates as
The similarity search device, wherein the registration feature value selection unit selects the non-representative data based on the calculated first correlation. The similarity search device according to claim 2,
The correlation calculation unit
The first correlation is calculated using a distance vector representing a similarity or distance between the search feature and the representative data and a distance vector representing a similarity or distance between the non-representative data and the representative data. Similarity search apparatus characterized by performing. The similarity search device according to claim 2 or 3, wherein:
The correlation calculation unit calculates a correlation between the search feature quantity and the representative data as a second correlation,
The similarity search device, wherein the registration feature value selection unit selects the representative data based on the calculated second correlation. The similarity search device according to claim 4,
The correlation calculation unit
A distance vector representing a distance between the search feature quantity and the registration feature quantity data already selected by the registration feature quantity selection unit; the representative data; and the already selected registration feature quantity data. The similarity search apparatus, wherein the second correlation is calculated using a distance vector representing a distance of The similarity search device according to claim 4,
The correlation calculation unit
A distance vector representing the distance between the search feature quantity and the non-representative data already selected by the registration feature quantity selection unit, and a distance between the representative data and the already selected non-representative data. A similarity search apparatus that calculates the second correlation using a distance vector. The similarity search device according to claim 2 or 3, wherein:
The similarity search device, wherein the registration feature quantity selection unit selects the representative data in a predetermined order. The similarity search device according to any one of claims 1 to 7,
The determination unit is
When the similarity in the feature space between the selected feature data for registration and the feature for search calculated by the similarity calculation unit is greater than a predetermined threshold, or when the distance is smaller than a predetermined threshold And a similarity search device characterized in that it is determined to be similar. The similarity search device according to claim 8,
For the unselected registration feature quantity data, the similarity or distance in the feature quantity space with the registration feature quantity data selected by the registration feature quantity selection unit, and
The difference between the similarity or distance in the feature amount space between the selected feature data for registration and the feature amount for search, calculated by the similarity calculation unit,
If it is greater than a predetermined threshold,
The similarity search apparatus characterized by including the pruning part which determines that the said feature-value selection part does not select the said feature-value data for unselected registration as a target of a similar search. The similarity search device according to any one of claims 1 to 9,
The similarity search device, wherein the multimedia data is biometric information of a user to be authenticated. The similarity search device according to any one of claims 1 to 10, wherein:
As means for acquiring the search feature amount,
A raw data acquisition unit that receives the multimedia data as an input;
A feature quantity extraction unit for extracting feature quantities from the multimedia data received as input;
A similarity search apparatus comprising: The similarity search device according to any one of claims 1 to 10, wherein:
Connected to a terminal device via a network,
The terminal device is
A raw data acquisition unit that receives the multimedia data as an input;
A feature quantity extraction unit for extracting the search feature quantity from the multimedia data received as input, and a means for transmitting the extracted feature quantity to the device;
With
The similarity search apparatus, wherein the similarity search apparatus acquires the feature amount by receiving it from the terminal device via a network. A similarity search system comprising the similarity search device according to any one of claims 1 to 10, a registration terminal device, and a server device including a storage unit,
The registered terminal device
Means for receiving the multimedia data as input;
Means for extracting feature data for registration from the acquired multimedia data;
Means for transmitting the extracted feature data for registration to the server;
With
The server device is
Means for selecting a plurality of representative data from the plurality of registration feature data received;
Means for transmitting representative data information indicating which of the plurality of representative data is selected and the plurality of registration feature data to the similarity search device;
A similarity search system comprising: The similarity search system according to claim 13,
The server device is
Means for calculating a similarity or distance in the feature amount space between each representative data and each registration feature amount data;
A similarity search system comprising: means for transmitting the calculated similarity or distance to the similarity search device. A similar search method for searching data similar to multimedia data,
Obtaining a search feature extracted from the multimedia data;
A registration feature quantity selection step of selecting one of the plurality of registration feature quantity data stored in the storage unit;
A similarity calculation step for calculating a similarity or distance in the feature amount space between the selected feature data for registration and the feature for search;
A determination step of determining whether or not the selected registration feature value data and the search feature value are similar based on the calculated similarity or distance;
A processing step for performing processing according to a determination result by the determination unit;
With
In the registration feature quantity selection step, representative data information indicating which representative data is selected from the plurality of registration feature quantity data further stored in the storage unit, each representative data, and each Using the similarity or distance in the feature quantity space of the registration feature quantity data, the representative data and the representative data are selected from the unselected registration feature quantity data that is not yet selected registration feature quantity data. The non-representative data that is the registration feature value data other than is alternately selected for each predetermined number,
As a result of the determination in the determination step, if the similarity is not similar, the registration feature amount selection step, the similarity calculation step, and the determination step are performed by performing similarity determination or all the registration feature amount data. A similarity search method characterized by repeating until a determination is made.   A program that causes a computer to function as each unit of the apparatus according to any one of claims 1 to 14.

Images