JP2018022390A - Verification device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable collation between images with high accuracy and with high efficiency.SOLUTION: A feature quantity coding part 11 calculates a feature quantity code obtained by quantizing a feature quantity vector associated with each partial area specified in a reference image to one or more discrete values. A corresponding candidate calculation part 12 calculates distance between a certain calculated feature quantity code and a feature quantity vector associated with each partial area specified in a query image, and calculates a corresponding candidate between the partial area specified in the reference image corresponding to the feature quantity code and the partial area specified in the query image on the basis of the distance. A determination part 13 determines whether the corresponding candidate is appropriate on the basis of a geometric relation between geometric information associated with the partial area specified in the reference image being a corresponding candidate and geometric information associated with the partial area specified in the query image, and determines the corresponding candidate to be corresponding when the corresponding candidate is appropriate.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a verification apparatus, method, and program, and more particularly, to a verification apparatus, method, and program for verifying the suitability of correspondence between partial areas defined in each of two images.

  In recent years, progress in image recognition technology has been remarkable. Up to now, the use areas where the recognition target and environment are limited such as face / fingerprint authentication and factory automation have been the focus. Recently, with the spread of small imaging devices such as smartphones, there is an increasing industrial demand for recognition of free shot images in which a general user has shot an arbitrary object in a free place or environment. Expectations are particularly high for O2O services that connect products in the real world and the web world, and information guidance / navigation services that recognize various landmarks in the real environment and provide information.

  There can be several forms of image recognition technology for such new applications, but one of the typical ones is image recognition based on object retrieval. The typical procedure is outlined according to the technique described in Non-Patent Document 1. First, by analyzing the luminance values of each image, a large number of characteristic partial areas are extracted, and the characteristics of each partial area are expressed as feature quantity vectors consisting of real values and integer values (generally, local feature quantities etc. be called). Next, by measuring the Euclidean distance between feature quantity vectors for partial areas included in two different images, the correspondence between the partial areas between different images is taken, and image pairs having a large number of correspondences have the same object. It is assumed that is reflected.

  In image recognition based on object retrieval, a database of images (reference images) obtained by photographing an object to be recognized in advance is constructed. By searching for a reference image in the database that contains the same object as the captured query image, the object existing in the query image can be specified by object search.

  One of the greatest features of object search is to express one image as a set of one or more partial regions (and feature vector describing them). Even if it is an image that shows the same object in a single word, it does not appear in the same position, orientation (partial area angle), and size in every image, but it is taken in various ways depending on the image. It is normal. In addition, it is almost impossible to know in advance how to capture an object in an image that a general user freely shoots. However, it is desirable that the feature vector describing the image has invariance independent of the position, orientation, and size.

  It is difficult to obtain the desired invariance with a global feature amount in which an entire image is represented by a single vector. For example, the color (RGB value) of each pixel arranged in a vector is not invariant with respect to position, orientation, and size. On the other hand, an abstraction of some information, such as a color histogram, can have invariance to position and orientation, but is not invariant to size. In addition, the accuracy is easily lowered, for example, the object is weak even when a part of the object is missing.

  On the other hand, in object search, an image is expressed by a set of partial areas. In this case, the partial areas are the same as the partial areas wherever they exist in the image. ing. In addition, a feature vector that describes a partial region has been invented that has invariance with respect to posture and size. For example, Scale Invariant Feature Transform (SIFT) described in Non-Patent Document 1 is a representative example.

  As described above, according to a typical procedure for object search, an image is represented by a set of one or more partial regions, so that an image including the same object can be robustly used regardless of position, posture, and size. You can search.

  However, the object search procedure has a serious problem. The correspondence between the partial areas is determined based on the distance between the feature vectors of the partial areas extracted from two different images, but in reality, even if they are different objects, they are very close features. In many cases, a quantity vector is obtained. As a result, an erroneous correspondence between partial areas that should not be handled originally occurs, and a different object may be searched.

  In view of such a problem, a verification technique for verifying the suitability of correspondence between partial areas has been invented for the purpose of preventing different objects from being searched.

  Non-Patent Document 1 discloses a verification method based on a generalized Hough transform. For partial regions obtained from the same object, verification is performed based on the assumption that changes in position, posture, and size between corresponding partial regions on the object are consistent depending on the shooting viewpoint. It is a method to do. First, between two different images, a real value distance between feature vectors is calculated to obtain a correspondence candidate between the partial areas, and then the position, posture, and size of the partial areas that have become correspondence candidates are expressed as “ Find the "shift". These deviations are obtained as four-dimensional real values, that is, deviations of four amounts of the position of the partial area (x and y coordinates on the image), the angle, and the size (scale) of the partial area. If a four-dimensional histogram is constructed based on this four-dimensional displacement, the set of corresponding partial areas obtained from the same object will be consistently displaced with respect to the position, orientation, and size of the partial areas. As these should be, they are assumed to be concentrated in a very small number of bins in the histogram. Accordingly, only the correspondence candidates distributed in the bins with high frequency are regarded as truly effective correspondences, and the other candidates are deleted as not being effective correspondences.

  Patent Document 1 discloses a technique obtained by improving Non-Patent Document 1. Although it is the same to determine the suitability of the correspondence based on the position, orientation, and size deviation of the partial area that is a candidate for correspondence, the affine-invariant local feature quantity is used as the feature quantity vector, so that the orientation is three-dimensionally rotated. Thinking about the angle. As a result, a finer verification than that of Non-Patent Document 1 is possible.

  Non-Patent Document 2 discloses a verification method based on a different approach. After all, it is the same as the prior art until the corresponding candidates for the partial areas between different images are obtained, but with this method, the position shift of the partial areas when a plurality of corresponding candidates are viewed as a set is specified. By considering only correspondence candidates that are constrained by the linear transformation to be valid correspondences, correspondence candidates that are not valid are deleted.

JP-A-2015-95156

D.G.Lowe, “Distinctive Image Features from Scale-Invariant Keypoints”, International Journal of Computer Vision, pp.91-110, 2004 J. Philbin, O. Chum, M. Isard, Josef Sivic and Andrew Zisserman.Object retrieval with large vocabularies and fast spatial matching 1470-1477, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2007.

  From a global perspective, the existing invention first obtains correspondence candidates based on the calculation of high-dimensional real / integer value distances between feature vectors, and then analyzes the position shift of the partial region precisely. Thus, the suitability of the response candidate is verified. Although verification by such a method is highly accurate, there is a problem that it takes a huge amount of time and space.

  That is, in order to obtain a correspondence candidate, it is necessary to obtain all high-dimensional real / integer distances between all feature vectors contained in one image and those contained in another image. Requires a lot of calculation and requires much processing time. In addition, for all images existing in the reference image database, it is necessary to store a high-dimensional real value / integer value feature vector extracted from each real value / integer value. / The amount of memory will be enormous.

  In addition, since the position (or posture / size) of the partial area is usually expressed by a real value, a large number of real value operations are required to obtain these precise deviations. A considerable amount of memory is also required for storage.

  As described above, up to the present, no verification technique has been invented that can verify the suitability of the correspondence efficiently in terms of time and space calculations while being highly accurate.

  The present invention has been made to solve the above-described problems. By compressing a real-value / integer-value feature vector into a low-capacity code, collation between images can be performed with high accuracy and high efficiency. An object of the present invention is to provide a verification apparatus, a method, and a program that enable the verification.

  In order to achieve the above object, the verification device according to the first aspect of the invention relates to geometric information including at least the size of the partial area associated with each of the one or more partial areas defined in the first image. And the feature quantity vector and the geometric information and the feature quantity vector associated with each of the one or more partial areas defined in the second image are specified in the first image. A verification apparatus for obtaining a correspondence between the partial area and the partial area defined in the second image, wherein the feature vector associated with each partial area defined in the first image A feature amount encoding unit for obtaining a feature amount code quantized into one or more discrete values, the feature amount code obtained for the first image, and each of the feature amount codes defined for the second image Associated with the subregion Further, a distance from the feature quantity vector is obtained, and based on the distance, the partial area defined in the first image corresponding to the feature quantity code and the partial area defined in the second image A correspondence candidate calculation unit that obtains a correspondence candidate with: the geometric information associated with the partial region defined in the first image that is the correspondence candidate; and the partial region defined in the second image A determination unit that determines whether the corresponding candidate is appropriate based on a geometric relationship with the geometric information associated with the information, and if it is appropriate, a determination unit that corresponds to the determination. ing.

  In the verification device according to the second aspect of the present invention, the verification device further includes a geometric information encoding unit that quantizes the geometric information into one or more discrete values and obtains a geometric information code, and the determination unit is the correspondence candidate Obtained from the geometric information code obtained from the geometric information associated with the partial area defined in the first image and the geometric information associated with the partial area defined in the second image Based on the geometric relationship with the geometric information code, it may be determined whether or not the corresponding candidate is appropriate, and if it is appropriate, it may be made to correspond.

  Further, in the verification device according to the third invention, the feature amount encoding unit is based on the sign of each element of the feature amount vector associated with each of the partial areas defined in the first image. Alternatively, a binary vector obtained by binary quantization may be obtained as a feature amount code.

  In the verification device according to the fourth invention, the determination unit obtains a histogram based on the geometric relationship for each correspondence candidate, and the correspondence candidate corresponding to a bin having a high frequency of the histogram is appropriate. You may make it determine.

  A verification method according to a fifth aspect of the present invention includes a geometric information and a feature quantity vector including at least the size of the partial area associated with each of the one or more partial areas defined in the first image; When the geometric information and the feature vector associated with each of the one or more partial regions defined in the image of the first image are input, the partial region defined in the first image, and the second A verification method in a verification device for obtaining a correspondence with the partial area defined in the image of the feature, wherein a feature amount encoding unit is associated with each partial area defined in the first image. A step of obtaining a feature amount code obtained by quantizing a quantity vector into one or more discrete values, a certain feature amount code obtained by the correspondence candidate calculation unit for the first image, and the second image Each said A distance from the feature vector associated with the partial area is obtained, and based on the distance, the partial area defined in the first image corresponding to the feature code and the second image are defined. A step of obtaining a correspondence candidate with the partial area, and a determination unit that adds the geometric information associated with the partial area defined in the first image that is the correspondence candidate to the second image. Determining whether or not the corresponding candidate is appropriate based on the geometric relationship with the geometric information associated with the defined partial region, and if appropriate, corresponding to the step; It is characterized by including.

  Further, in the verification device according to the sixth invention, the geometric information encoding unit further includes a step of quantizing the geometric information into one or more discrete values to obtain a geometric information code, wherein the determination unit determines Is associated with the geometric information code obtained from the geometric information associated with the partial area defined in the first image as the correspondence candidate and the partial area defined in the second image. Further, based on the geometric relationship with the geometric information code obtained from the geometric information, it is determined whether or not the corresponding candidate is appropriate. .

  Further, in the verification device according to the seventh invention, the step of encoding by the feature amount encoding unit includes each element of the feature amount vector associated with each of the partial regions defined in the first image. A binary vector obtained by performing binary quantization on the basis of the sign may be obtained as a feature amount code.

  A program according to an eighth invention is a program for causing a computer to function as each part of the verification device according to the first to fourth inventions.

  According to the verification apparatus, method, and program of the present invention, a feature amount code obtained by quantizing a feature amount vector associated with each partial region defined in the first image into one or more discrete values is obtained. The distance between the obtained feature quantity code and the feature quantity vector associated with each partial area defined in the second image is obtained, and the first image corresponding to the feature quantity code is obtained based on the distance. A candidate for correspondence between the partial region defined in the second image and the partial region defined in the second image, the geometric information associated with the partial region defined in the first image as the correspondence candidate, and the second Based on the geometric relationship with the geometric information associated with the partial area defined in the image, it is determined whether or not the corresponding candidate is appropriate. Collation between images is possible with high accuracy and high efficiency To, the effect is obtained that.

It is a block diagram which shows the structure of the verification apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the verification process routine in the verification apparatus which concerns on the 1st Embodiment of this invention. The figure which shows the geometric relationship of the geometric information of the partial area | region of two images. The figure which shows the table which contrasted the geometric relationship of the geometric information of the partial area | region of two images. It is a block diagram which shows the structure of the verification apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the verification process routine in the verification apparatus which concerns on the 2nd Embodiment of this invention.

<Outline according to Embodiment of the Present Invention>

  First, an outline of the embodiment of the present invention will be described.

  The verification apparatus according to the present embodiment is defined in the query image and the geometric information and feature quantity vector including at least the size of the partial area, which are associated with each of the one or more partial areas defined in the reference image. When the geometric information and the feature vector associated with each of the one or more partial regions are input, the correspondence between the partial region defined in the reference image and the partial region defined in the query image is obtained. It is a verification device. Note that the first image corresponds to the reference image, and the second image corresponds to the query image.

  In the embodiment of the present invention, a high-dimensional real value / integer value distance is calculated by quantizing a feature vector when obtaining correspondence candidates between partial areas obtained from two images. Without having to do so. In addition, more efficient verification can be performed by quantizing the geometric information. In addition, with respect to the reference image, verification can be executed if only the quantized low-capacity feature vector is stored instead of the original high-dimensional real-value / integer-value feature vector, thus reducing the capacity. Can do. In addition, the geometric information of the partial area used in the embodiment of the present invention only needs to hold at least the size of the partial area, and multidimensional geometric information such as position, posture, and size is integrated. It is not always necessary to analyze accurately and precisely. As a result, it is possible to perform geometric verification with high efficiency in terms of both time and space.

  Furthermore, when quantizing based on whether each element of the feature vector is positive or negative, the binary vector, which is the resulting code, is an important element for measuring the distance of the feature vector. The feature vector information can be stored with high accuracy. For query images that have little effect on the time / space calculation efficiency, the original feature vector is used as it is to minimize the information to be quantized. As a result, high-precision verification is possible without reducing accuracy.

<Verification apparatus according to the first embodiment of the present invention>

  Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.

<< Overall Configuration of First Embodiment >>

  FIG. 1 is a block diagram showing an example of the configuration of the verification apparatus 1 according to the first embodiment of the present invention. The verification device 1 illustrated in FIG. 1 includes a feature amount encoding unit 11, a correspondence candidate calculation unit 12, and a determination unit 13.

  The verification device 1 is connected to the database 2 via a communication unit and communicates information with each other, so that information on an arbitrary image can be registered in the database 2 or read out.

  The image information here includes at least the geometric information and the feature amount vector of the partial area of each image, and these are associated with each other. The partial area of the image may be determined in any way as long as it is a partial area of the image. Preferably, it is defined by a known partial region extraction method such as SIFT described in Non-Patent Document 1.

  Further, the geometric information of the partial area includes information such as the position, posture, and size of the partial area, but it is sufficient that at least the size is included in the example of the first embodiment of the present invention. Preferably, it further includes posture and position. The size of the partial area may be determined in any way, but for example, it is defined based on the radius when the partial area is considered to be a circle, the long side when the partial area is assumed to be a square, etc. That's fine. Not only the size but also the posture and position can be obtained by selecting a suitable known method such as the method described in Non-Patent Document 1, for example.

  The database 2 can be configured by, for example, a file system implemented on a general general-purpose computer. Each image is given an identifier (for example, an ID by a serial number or a unique image file name) that can uniquely identify each image, and further, a partial region defined in the image, a feature vector, or a feature code It is assumed that the file describing the file is also stored in association with the identifier of the image. Alternatively, it may be similarly implemented and configured by RDBMS (Relational Database Management System) or the like. In addition, the metadata may include, for example, data representing the content of the image (image title, summary text, or keyword), data related to the image format (image data amount, thumbnail size, etc.), and the like. Although not required, it is not essential in the practice of the present invention.

  The database 2 may be inside or outside the verification apparatus 1, and any known communication means can be used. In the present embodiment, the communication means is assumed to be outside. Are connected to communicate via a network (not shown) such as the Internet or TCP / IP.

  Further, each unit and database 2 provided in the verification device 1 may be configured by a computer or a server provided with an arithmetic processing device, a storage device, or the like, and the processing of each unit may be executed by a program. This program is stored in a storage device included in the verification apparatus 1, and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or provided through a network. Of course, any other components need not be realized by a single computer or server, but may be realized by being distributed to a plurality of computers connected by a network.

  The image information itself does not necessarily have to be stored in the database 2, and may be configured to be directly input from the outside as appropriate, for example. In such a configuration, for example, when the embodiment of the present invention is used for object search, a reference image is subjected to necessary processing for reference image information in advance and stored in the database 2 to obtain a query image. Is suitable for the purpose of receiving and processing input of query image information from the outside at the timing of inquiry as appropriate. As a specific example, in the example of the configuration of the verification apparatus illustrated in FIG. 1, the reference image information 3 that is image information related to one or more reference images is stored in the database 2 in advance. It is connected to the verification apparatus 1 in a form that can be read / registered mutually. In addition, the configuration is such that the query image information 4 input as an inquiry can be received from the outside.

  Thereafter, in the example of the first embodiment of the present invention, when verifying the correspondence between two images, one of the reference images stored as the reference image information 3 especially for the purpose of object search. An example will be described in which the correspondence between the image and one of the query images input as the query image information 4 is verified. In the case of verifying the correspondence between a plurality of sets of images, the processing described below may be repeated as many times as the number of sets to be verified.

  << Processing Unit of First Embodiment >>

  Each processing unit of the verification apparatus 1 in the present embodiment will be described.

  The feature amount encoding unit 11 obtains a feature amount code obtained by quantizing a feature amount vector associated with each partial region defined in the reference image into one or more discrete values. Here, after receiving the reference image information 3 from the database 2, a feature amount code obtained by quantizing the feature amount vector associated with each partial region of the reference image information 3 into one or more discrete values is obtained.

  The obtained feature amount code may be stored in the database 2 in association with the reference image information 3 as necessary, or may be output to the correspondence candidate calculation unit 12. In the example of the first embodiment of the present invention, the case where the present process is applied only to the reference image side will be described. As an effect of applying this processing only to the reference image side without applying this processing to the query image side, the following can be mentioned.

(1) Normally, unlike the case of a reference image that is always stored, it is not always necessary to store a query image. Therefore, even if the query image is encoded at a low capacity, the influence on the temporal and spatial calculation amount is not affected. rare.
(2) By performing the encoding, the information of the original feature vector is somewhat lost, so that it is possible to maintain the accuracy by avoiding unnecessary encoding.

  Of course, the same processing can be applied to the query image side, and it is needless to say that this processing can be applied only to the query image side.

  The correspondence candidate calculation unit 12 includes a certain feature amount code obtained for the reference image by the feature amount encoding unit 11 and a feature amount associated with each partial region defined in the query image in the query image information 4 received from the outside. A distance to the vector is obtained, and based on the distance, a correspondence candidate between the partial area defined in the reference image corresponding to the feature amount code and the partial area defined in the query image is obtained. The query image information 4 includes information on a query image and a set of partial regions and feature quantity vectors of the query image. Note that it is not always necessary to obtain a partial area of a query image that is a candidate for correspondence with respect to all partial areas of the reference image.

  The determination unit 13 performs, for each set of geometric information associated with the partial area defined in the reference image that is a correspondence candidate, and geometric information associated with the partial area defined in the query image, the geometric information of the set. And determines whether the corresponding candidate for the set is appropriate based on the determined geometric relationship, and if it is appropriate, determines that this is a corresponding candidate and outputs determination result 5 To do. Here, the determination unit 13 obtains a histogram based on the geometric relationship for each corresponding candidate, and determines that a corresponding candidate corresponding to a bin having a high histogram frequency is appropriate.

<< Process overview >>

  Next, the process of the verification apparatus 1 in this Embodiment is demonstrated. FIG. 2 is a flowchart showing the flow of processing.

  First, in step S301, the feature amount encoding unit 11 obtains a feature amount code obtained by quantizing a feature amount vector associated with each partial region defined in the reference image into one or more discrete values. The obtained feature amount code may be stored in the database 2 or may be transmitted to the correspondence candidate calculation unit 12. In addition, this process may be performed in advance before receiving a query image.

  Subsequently, in step S302, when the correspondence candidate calculation unit 12 accepts a set of partial areas and feature amount vectors in the query image information 4 as input, the correspondence candidate calculation unit 12 associates the partial areas defined in the query image information 4 with each partial area. The distance between the feature vector and the feature code of each partial area of the reference image is obtained, and based on this distance, the partial area defined in the reference image corresponding to the feature code and the query image It is obtained as a candidate for correspondence with the partial area, and is transmitted to the determination unit 13.

  Subsequently, in step S303, the determination unit 13 sets a set of geometric information associated with the partial area defined in the reference image that is a correspondence candidate and geometric information associated with the partial area defined in the query image. For each, the geometric relationship of the set is obtained, and based on the obtained geometric relationship, it is determined whether or not the corresponding candidate for the set is appropriate. Then, the determination result 5 is output. Here, the determination unit 13 obtains a histogram based on the geometric relationship for each corresponding candidate, and determines that a corresponding candidate corresponding to a bin having a high histogram frequency is appropriate.

<< Processing Details of Each Process in First Embodiment >>

  Hereinafter, an example of the detailed processing of each processing will be described in the present embodiment.

[Feature encoding]

  First, a description will be given of processing in the feature amount encoding unit 11 to obtain a feature amount code by quantizing a feature amount vector of a partial region having a reference image.

  The problem of quantizing a high-dimensional feature quantity vector into one or more discrete values can be regarded as a problem of vector quantization, and any vector quantization method can be applied. For example, there is a quantization method based on the K-average method (K is a natural number), which is one of the typical vector quantization methods.

  The K-average method (or many other quantization methods not limited to this) needs to learn a quantizer in advance. For example, in the case of the K-average method, the K-average method is applied in advance using a set of a certain number of feature quantity vectors as learning data to obtain K cluster centers. When a certain feature quantity vector is actually quantized, the cluster center is most determined from the feature quantity vector, and the cluster center (vector) of the cluster may be used as the feature quantity code.

  For encoding, binary quantization is preferably used. In binary quantization, an original real value or integer value feature vector is quantized into a binary vector. A binary vector is a vector in which each element of the vector takes only a binary value. The value of the element is not particularly limited, but may be {0, 1}, {1, -1}, for example. Preferably, in order to simplify the distance calculation later, it is better to set {1, -1}, which will be described below. If the dimension of the binary vector is B (B is a natural number), the number of possible binary vectors is 2B, that is, it is equivalent to quantization to 2B discrete values.

  An example of a process of quantizing a real value / integer value feature quantity vector into a binary vector will be described. For convenience, if the original real value / integer value feature quantity vector is represented by x and the binary vector is represented by b, a binary vector can be obtained by the following equation (1), for example.

... (1)

  Here, sign is a function that takes a feature quantity code for each element. More strictly, among elements of the feature quantity vector x, 1 is returned for an element that takes 0 or more, and -1 is returned for an element that is less than 0. Suppose that it is a function. By using the above equation (1), it is possible to obtain a binary vector having the same dimension as that of the original feature vector.

  On the other hand, many of the feature quantities used for object search are originally composed of non-negative real / integer values. In this case, the binary vector according to the above equation (1) always has 1 for all elements. There is a problem of becoming. In order to avoid this problem, a bias component μ is preferably introduced.

... (2)

  μ is a vector having the same dimension as x. Although the value of the element may be freely determined, it is preferable to use an average value, median value, average vector, median value vector, or the like of the feature vector group.

  Furthermore, there is a problem that the information loss is large simply by taking the element code as in the above formulas (1) and (2), and the accuracy of the distance calculation when determining the correspondence candidates is greatly deteriorated. In order to execute binary quantization that does not impair information as much as possible, preferably binary quantization according to the following equation (3) is used.

... (3)

  Here, A is a matrix having the same size as the number of dimensions of x, and A ′ represents the transpose of A. This matrix A can be formed by any known matrix construction method, but is preferred because it can be obtained by a principal component analysis because a matrix that optimally stores the information amount of the feature vector group can be obtained. . Since principal component analysis also has an aspect as a dimension reduction method, there is an advantage in that it is possible to obtain a lighter feature quantity code by reducing the dimension of a binary vector.

  The binary vector obtained by these methods well preserves the angle component formed by the original feature vector. The correspondence candidate is calculated based on the distance between the feature quantity vectors, but it is known that the influence of the angle component is large in obtaining the distance. Therefore, by preserving the angle component well, there is an advantage that information can be compressed without degrading the accuracy of distance calculation.

[Correspondence candidate calculation processing]

  Next, a process for obtaining correspondence candidates between partial areas defined between two different images in the correspondence candidate calculation unit 12 will be described. In an example of the first embodiment of the present invention, the process is used to determine a corresponding partial region between a query image and each reference image.

A partial area extracted from the query image is represented as Q _i , and a partial area extracted from the reference image is represented as R _j . Hereinafter, taking the partial areas Q _i and R _j as an example, a process for determining whether or not these are correspondence candidates will be described.

Each partial region is associated with a feature vector expressing the partial region or a feature code obtained by quantizing the feature vector. It is assumed that the feature code describing the partial region Q _i is represented as q and the feature code describing R _j is represented as r. At this time, a distance dist (Q _i , R _j ) between the partial areas is obtained by the following equation.

... (4)

  Here, according to an example of the first embodiment of the present invention, r is a feature value code, and particularly when this is expressed by a binary vector of {1, -1}, the above equation (4) is It can be converted as follows.

... (5)

Further, q _m and r _m are the m-th elements of vectors q and r, respectively. Here, since r _m is {1, -1} only take one of the values, after all the (5) is evaluable only addition and subtraction of the elements of q, to increase efficiency of the distance calculation be able to.

  In practice, this distance is calculated for all partial region combinations.

Subsequently, based on the obtained distance between the partial areas, it is determined whether or not each set of partial areas is a correspondence candidate. When focusing on the partial region R _j of the reference image, it is assumed that the partial region of the query image closest to this is Q _i and the next partial region thereof is Q _k . At this time, when the condition of the following expression (6) is satisfied, it is determined that R _j and Q _i are correspondence candidates.

... (6)

  Here, T is a parameter determined in advance, and may take an arbitrary value of 0 <T ≦ 1. For example, T = 0.8 may be set.

  By performing the above calculation for all sets of partial areas in each of the reference images for all query images, it is possible to obtain partial areas that are candidates for correspondence.

Note that the correspondence candidates obtained in this way allow duplication. That is, for the partial area of a reference image R _j, a plurality of partial regions of the query image is likely to become the corresponding candidates. Usually, even though the objects are the same, a plurality of partial areas do not often correspond to a certain partial area of the object. Therefore, post-processing may be introduced so as not to allow duplication of correspondence candidates. For example, after all the correspondences are obtained by the above method, candidate regions on the query image side corresponding to a plurality of candidate regions are listed. Subsequently, among the partial regions of the reference image corresponding to the partial region on the query image side, only the closest region is determined to be an effective corresponding candidate, and the corresponding candidate is rejected for the other pairs To do. By introducing the processing as described above, it is possible to constrain all the partial regions to always correspond one-to-one.

[Determination process]

  Next, a process of determining whether or not the corresponding partial area correspondence candidate is appropriate based on the geometric information of the partial area in the determination unit 13 will be described. Here, among the correspondence candidates obtained by the correspondence candidate calculation unit 12, a pair that is considered not to be an effective correspondence (that is, not a correspondence between partial areas extracted from an object) is deleted.

  Suppose that the query image and the reference image include the same object. If the object is a rigid body, the object in the query image and the object in the reference image are only taken from different viewpoints. Under realistic assumptions, this viewpoint variation is consistent with the appearance of the subregion. give. In other words, if the partial areas that are candidates for correspondence are pairs of partial areas that are correctly present on the same object, the geometric information of the partial area on the query side and the reference image that is a candidate for correspondence The geometric relationship (displacement) of the geometric information of the side area is consistent. Therefore, it is only necessary to consider a pair of correspondence candidates that are consistent in this way of deviation as an effective correspondence and reject those that are not.

  This will be described in an easy-to-understand manner with reference to FIGS. FIG. 3 shows two images A and B including the same object. Each of the three types of patterns 51A, 51B, 52A, 52B, 53A, and 53B surrounded by a broken line is defined as a partial area, and a set of partial areas (for example, 51A and 51B) represented by the same numbers. Suppose that it is determined as a corresponding candidate. The purpose is to determine only a pair of partial areas (in this case, 51A and 51B and 52A and 52B) existing on the same object as an effective correspondence.

  As can be seen from FIG. 3, the positions, postures, and sizes of 51A and 51B and 52A and 52B on the same object change in the same manner depending only on the shooting viewpoint of the camera. Recognize. FIG. 6 shows the result of quantifying the x-coordinate, y-coordinate, and posture where the position of the partial region on the image is quantified, and specifically obtaining the geometric relationship. Since 51A and 51B and 52A and 52B are close to each other in the amount of deviation, if a histogram is constructed for this amount of deviation, the corresponding candidates of 51A and 51B and 52A and 52B are concentrated and distributed in the same bin on the histogram. . Therefore, as a result, if it is determined that only the correspondence candidates of the partial areas in the bins of the histogram with high frequency are effective correspondences, it is possible to delete a pair that is not an effective correspondence.

  In the example of FIG. 4, the position / posture / size is used as the geometric information. However, as described above, in the embodiment of the present invention, it is sufficient that at least the size is defined as the geometric information. In this case, since the histogram is simplified to one dimension, efficient processing is possible.

  Further, the determination is not limited to such a histogram, and other known geometric relationship verification methods may be used. For example, it is known that the geometric relationship between partial areas on the same object is constrained by linear transformation under appropriate conditions. There are known effective methods such as the RANSAC algorithm described in Reference 1 and the LO-RANSAC algorithm described in Reference 2 as a method for obtaining such a linear transformation and a pair of correspondence candidates having a geometric relationship according to the linear transformation. Therefore, these may be used.

[Reference 1] MA Fischler and RC Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Comm. ACM, vol. 24, no. 6, pp. 381-395, 1981 .

[Reference 2] O. Chum, J. Matas, and S. Obdrzalek, “Enhancing RANSAC by generalized model optimization,” Proceedings of Asian Conference on Computer Vision, pp. 812-817, 2004.

  With the above procedure, it can be determined whether or not the query image and the reference image include the same object.

  As described above, according to the verification apparatus according to the first embodiment of the present invention, the feature vector associated with each partial region defined in the reference image is quantized into one or more discrete values. The feature value code is obtained, the distance between the obtained feature value code and the feature value vector associated with each partial area specified in the query image is obtained, and the feature value code is supported based on the distance. To obtain a candidate for correspondence between the partial area specified in the reference image and the partial area specified in the query image, the geometric information associated with the partial area specified in the reference image that is a correspondence candidate, and the query image It is possible to determine whether the corresponding candidate is appropriate based on the geometric relationship with the geometric information associated with the specified partial area, and to determine whether the corresponding candidate is appropriate. High efficiency To enable the collation between.

<Verification Device According to Second Embodiment of the Present Invention>

  Hereinafter, a second embodiment of the present invention will be described in detail with reference to the drawings. In addition, the same code | symbol is attached | subjected about the location similar to 1st Embodiment, and description is abbreviate | omitted.

  The second embodiment is different from the first embodiment in that the geometric information of the partial region is encoded.

<< Overall Configuration of Second Embodiment >>

  FIG. 5 is a block diagram showing an example of the configuration of the verification apparatus 101 according to the second embodiment of the present invention. The verification apparatus 101 illustrated in FIG. 1 includes a feature amount encoding unit 11, a correspondence candidate calculation unit 12, a geometric information encoding unit 14, and a determination unit 13.

  << Processing Unit of Second Embodiment >>

  The geometric information encoding unit 14 performs one or more of each of the geometric information associated with the partial area defined in the reference image that is a correspondence candidate and the geometric information associated with the partial area defined in the query image. Quantize to discrete values to obtain geometric information code. Note that the geometric information code may be obtained for only one of the reference image and the query image.

  The determination unit 13 uses the geometric information code obtained from the geometric information associated with the partial area defined in the reference image that is a correspondence candidate and the geometric information obtained from the geometric information associated with the partial area defined in the query image. For each pair of codes, the geometric relationship of the pair is obtained, and based on the obtained geometric relationship, it is determined whether or not the corresponding candidate for the pair is appropriate. Is determined to correspond, and a determination result 5 is output.

<< Process overview >>

  Next, the process of the verification apparatus 101 in this Embodiment is demonstrated. FIG. 6 is a flowchart showing the flow of processing.

  In step S303B, the geometric information encoding unit 14 performs each of the geometric information associated with the partial region defined in the reference image that is a correspondence candidate and the geometric information associated with the partial region defined in the query image. Quantize to one or more discrete values to obtain a geometric information code.

  In step S304B, the determination unit 13 obtains the geometric relationship for each set of the geometric information code of the partial area on the reference image side and the geometric information code of the partial area on the query image side, which is a correspondence candidate. Based on this, it is determined whether or not the corresponding candidate is appropriate. If it is appropriate, it is determined that the corresponding candidate is appropriate, and a determination result 5 is output.

  Since other configurations and operations of the second embodiment are the same as those of the first embodiment, description thereof is omitted.

<< Processing Details of Each Process in Second Embodiment >>

  The geometric information encoding process in the geometric information encoding unit 14 will be described. In the determination process described in the first embodiment, the determination process is performed on the assumption that the geometric information is a real value, but the geometric information may be encoded.

  The geometric information can be encoded based on the quantization as in the case of the feature vector encoding. For example, taking the size as an example, the size range is divided into several sections, and encoding is performed based on which section the size of a partial area belongs to. More specifically, suppose that the size is divided into four categories of 0 or more, less than 2, 2 or more, less than 4, 4 or more, less than 6, or 6 or more, and “1”, “2”, “ The geometric information code is defined as “3” and “4”. At this time, if the size of a partial area is 2.5, the size of the partial area belongs to 2 sections, so the geometric information code of the size is 2.

  By performing the encoding as described above, the calculation of the geometric relationship can be simplified. For example, if a geometric information code of one size is 2 and a geometric information code of the other size is 3 for a set of partial areas that are certain correspondence candidates, the deviation is simply 3-2 = 1. Can be evaluated.

  Although the case of size has been described as an example here, it is needless to say that processing can be similarly performed when other types of geometric information (position, posture, etc.) are used.

  Thus, by compressing the geometric information of the partial region into a low-capacity code, it is possible to perform more efficient verification.

  As described above, according to the verification apparatus according to the second embodiment of the present invention, the feature vector associated with each partial region defined in the reference image is quantized into one or more discrete values. The feature value code is obtained, the distance between the obtained feature value code and the feature value vector associated with each partial area specified in the query image is obtained, and the feature value code is supported based on the distance. The candidate corresponding to the partial area specified in the reference image and the partial area specified in the query image is obtained, the geometric information is quantized into one or more discrete values, the geometric information code is obtained, and the reference that is the corresponding candidate Based on the geometric relationship between the geometric information code obtained from the geometric information associated with the partial area defined in the image and the geometric information code obtained from the geometric information associated with the partial area defined in the query image, Concerned Response candidate is equal to or appropriate, by the corresponding this where appropriate, to allow matching between images with high accuracy and high efficiency.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1,101 Verification apparatus 2 Database 3 Reference image information 4 Query image information 5 Determination result 11 Feature-value encoding part 12 Corresponding candidate calculation part 13 Determination part 14 Geometric information encoding part

Claims (8)

Geometric information and feature quantity vector including at least the size of the partial area associated with each of the one or more partial areas defined in the first image, and one or more defined in the second image When the geometric information and the feature vector associated with each of the partial areas are input, the partial area defined in the first image, and the partial area defined in the second image A verification device that requests
A feature amount encoding unit for obtaining a feature amount code obtained by quantizing the feature amount vector associated with each of the partial regions defined in the first image into one or more discrete values;
A distance between the certain feature amount code obtained for the first image and the feature amount vector associated with each of the partial regions defined in the second image is obtained, and based on the distance, A correspondence candidate calculation unit for obtaining a correspondence candidate between the partial region defined in the first image corresponding to the feature amount code and the partial region defined in the second image;
Geometric relationship between the geometric information associated with the partial area defined in the first image as the correspondence candidate and the geometric information associated with the partial area defined in the second image On the basis of whether or not the corresponding candidate is appropriate, and if it is appropriate, a determination unit corresponding to this,
Verification device including A geometric information encoding unit that quantizes the geometric information into one or more discrete values and obtains a geometric information code;
The determination unit includes the geometric information code obtained from the geometric information associated with the partial area defined in the first image that is the correspondence candidate, and the partial area defined in the second image. A determination is made as to whether or not the corresponding candidate is appropriate based on a geometric relationship with the geometric information code obtained from the geometric information associated with the item, and if it is appropriate, it is determined as a corresponding item. The verification apparatus according to 1.   The feature amount encoding unit obtains a binary value obtained by performing binary quantization on each element of the feature amount vector associated with each of the partial areas defined in the first image based on the positive / negative. 3. The verification apparatus according to claim 1, wherein the vector is obtained as a feature amount code.   The determination unit obtains a histogram based on the geometric relationship for each correspondence candidate, and determines that the correspondence candidate corresponding to a bin having a high frequency in the histogram is appropriate. The verification apparatus according to item 1. Geometric information and feature quantity vector including at least the size of the partial area associated with each of the one or more partial areas defined in the first image, and one or more defined in the second image When the geometric information and the feature vector associated with each of the partial areas are input, the partial area defined in the first image, and the partial area defined in the second image A verification method in a verification device that seeks the correspondence of
A feature amount encoding unit obtaining a feature amount code obtained by quantizing the feature amount vector associated with each of the partial areas defined in the first image into one or more discrete values;
The correspondence candidate calculation unit obtains a distance between the certain feature amount code obtained for the first image and the feature amount vector associated with each partial region defined in the second image, and Obtaining a candidate for correspondence between the partial area defined in the first image corresponding to the feature code and the partial area defined in the second image based on a distance;
The determination unit includes the geometric information associated with the partial area defined in the first image as the correspondence candidate, and the geometric information associated with the partial area defined in the second image. Determining whether or not the corresponding candidate is appropriate based on the geometric relationship of
Verification method including A geometric information encoding unit further comprising: quantizing the geometric information into one or more discrete values to obtain a geometric information code;
The step of determining by the determining unit is defined in the geometric information code obtained from the geometric information associated with the partial area defined in the first image that is the correspondence candidate, and in the second image. Based on the geometric relationship with the geometric information code obtained from the geometric information associated with the partial area, it is determined whether or not the corresponding candidate is appropriate. The verification method according to claim 5.   The step of encoding by the feature amount encoding unit performs binary quantization on each element of the feature amount vector associated with each of the partial areas defined in the first image based on the positive / negative. The verification method according to claim 5 or 6, wherein the binary vector obtained in step (5) is obtained as a feature amount code.   The program for functioning a computer as each part of the verification apparatus of any one of Claims 1-4.