JP6014008B2 - Geometric verification apparatus, geometric verification method and program - Google Patents

Description

  The present invention relates to a geometric verification apparatus, a geometric verification method, and a program for verifying the consistency of corresponding points indicating the same point in space from two or more images obtained by photographing the same subject.

  The technique of detecting corresponding points indicating the same point in the space in at least two or more images input by a camera that captures a real space enables three-dimensional reconstruction of the subject and object recognition.

  For example, in the method described in Non-Patent Document 1, a key point that is a characteristic point such as a corner is determined from each image, a feature amount is extracted using a neighborhood region of the feature point, and between the two images. A method of performing object recognition using feature points having the most similar feature amounts as corresponding points is described. However, both images are affected by a background different from that of the subject, the presence of other objects, camera thermal noise, noise due to image compression error, and the like, resulting in erroneous correspondence.

  Therefore, in addition to point-to-point correspondence, it is common to remove erroneous correspondence by geometric verification that verifies whether the point group-to-point group correspondence is geometrically consistent. In the method described in Non-Patent Document 1, the amount of change between key points in both images is determined using attribute information including the x-coordinate and y-coordinate of the key point, the scale, and the rotation of the local region. Voting on a four-dimensional space defined by the amount of change in coordinates, y-coordinate, scale, and rotation amount of the local area, and adopting only the corresponding points that have obtained a certain number of votes, the main two-dimensional between images It implements geometric verification that calculates the affine transformation and eliminates miscorrespondence.

  FIG. 12 is a diagram showing an outline of processing for removing a false correspondence from a key point detected between two images. FIG. 12A is a diagram illustrating a correspondence example of key points detected in two images (image 1 and image 2). FIG. 12A shows an example in which the key points detected in the image 1 are erroneously associated with the key points detected in the image 2. FIG. 12B shows the amount of change between key points in both images (Δx: amount of change in x-coordinate, Δy: amount of change in y-coordinate, Δs: amount of change in scale of the local region including key points, Δr: It is a figure which shows an example of the four-dimensional space defined by the variation | change_quantity of rotation of a local region.

  FIG. 12C is a diagram showing an outline of two-dimensional affine transformation obtained based on the amount of change between key points in both images. FIG. 12C shows a case where a two-dimensional affine transformation is obtained for a heart-shaped figure based on the amount of change between key points in both images. FIG. 12D shows a result of removing the false correspondence by the two-dimensional affine transformation obtained in FIG. As illustrated in FIG. 12, in the method described in Non-Patent Document 1, a main two-dimensional affine transformation is calculated to eliminate an erroneous correspondence.

David G. Lowe, "Distinctive Image Features from Scale-Invariant Key points", International Journal of Computer Vision, Vol.60, Issue 2, pp.91-110, November 2004

  In the above-described method, the x-coordinate, y-coordinate, scale, and scale between the subject corresponding portion (key point) of one image and the subject corresponding portion of the other image are assumed on the assumption that the subject is a two-dimensional object. Geometric verification is realized using the fact that the amount of change in the rotation amount of the local region is constant between key points. Therefore, when the subject is a three-dimensional object, three-dimensional distortion occurs on the image, so that the amount of change in the x-coordinate, y-coordinate, scale, and rotation amount of the local region is not constant between key points. There is a problem in that it is impossible to verify the integrity of the image.

  The present invention has been made to solve the above-described problem. Even when the subject is a three-dimensional object, the geometry that can verify the consistency of corresponding points obtained from a plurality of images obtained by photographing the same subject. An object is to provide a verification apparatus, a geometric verification method, and a program.

  One embodiment of the present invention inputs a plurality of images obtained by photographing the same subject, determines an observation direction of affine-invariant key points extracted in each of the input images based on the images, and An affine transformation keypoint extraction / corresponding point determination unit that associates affine invariant keypoints as corresponding points, and an observation direction that estimates a difference in observation direction of the affine invariant keypoints at each of the corresponding points as an observation direction change amount Changes in the observation direction included in the cell are calculated as the number of changes in the observation direction included in the cell obtained by dividing the voting space corresponding to the number of dimensions in the observation direction and the change amount estimation unit with a predetermined size. An observation space voting unit for associating corresponding points corresponding to the quantity with the cell, and vote counting for outputting corresponding points associated with the cell whose number of votes is larger than a predetermined threshold The geometric verification device, characterized in that it comprises and.

  Further, according to one aspect of the present invention, in the geometric verification device described above, the affine transformation key point extraction / corresponding point determination unit is a camera posture when the image is captured, and is based on an affine invariant key point. The observed three-dimensional camera posture is determined as the observation direction, and the observation space voting unit performs voting on each cell in the three-dimensional voting space based on the amount of change in the observation direction.

  Further, according to one aspect of the present invention, in the geometric verification device described above, the affine transformation key point extraction / corresponding point determination unit is a camera posture when the image is captured, and is based on an affine invariant key point. A three-dimensional camera posture and a scale determined according to the distance between the camera and the affine invariant key point when the image is taken as the observation direction, and the observation space voting unit Voting based on the amount of change in the observation direction is performed for each cell.

  Further, according to one aspect of the present invention, in the geometric verification device described above, the affine transformation key point extraction / corresponding point determination unit is a camera posture when the image is captured, and is based on an affine invariant key point. A three-dimensional camera posture and a scale determined according to the distance between the camera and the affine invariant key point when the image is taken as the observation direction, and the observation direction change amount estimation unit is affine invariant The difference in the observation direction of the key point and the difference in the coordinates of the affine invariant key point in the image are estimated as the amount of change in the observation direction, and the observation space voting unit observes each cell in the six-dimensional space. Voting is performed based on the amount of change in direction.

  Another embodiment of the present invention is a geometric verification method performed by a geometric verification apparatus that inputs a plurality of images obtained by photographing the same subject, and observes affine invariant key points extracted in each of the input images. An affine transformation key point extraction / corresponding point determination step that determines a direction based on the image and associates an affine invariant key point between the images to be a corresponding point, and an observation direction of the affine invariant key point at each of the corresponding points The observation direction change estimation step that estimates the difference between the observation directions as the amount of change in the observation direction, and the number of changes in the observation direction included in the cell obtained by dividing the voting space corresponding to the number of dimensions in the observation direction by a predetermined size An observation space voting step that calculates the number of votes and associates a corresponding point corresponding to the amount of change in the observation direction included in the cell with the cell, and a vote There is a geometric verification method characterized by having a vote counting step of outputting the corresponding points associated with larger cell than a predetermined threshold.

  One embodiment of the present invention is a program for causing a computer to function as the geometric verification apparatus described above.

  According to the present invention, the difference in the observation direction with respect to the affine invariant key point when an image is captured in the three-dimensional space in which the subject exists is estimated as the amount of change, and the amount of change is determined in the voting space determined according to the observation direction. By using the voting used, it is possible to remove corresponding points whose change amount is different from other change amounts, and even when the subject is a three-dimensional object, the corresponding points obtained from a plurality of images obtained by photographing the same subject The geometric consistency can be verified.

It is a block diagram which shows the structural example of the geometric verification apparatus 1 in this embodiment. It is a flowchart which shows the process which the affine transformation key point extraction and corresponding point determination part 11 in this embodiment performs. It is a figure which shows the coordinate system which determines the observation direction in this embodiment. It is a figure which shows the example of the corresponding point attribute table which the corresponding point attribute information storage part 12 in this embodiment memorize | stores. It is a flowchart which shows the process which the observation direction variation | change_quantity estimation part 21 in this embodiment performs. It is a figure which shows the outline | summary of the variation | change_quantity of the observation direction which the observation direction variation | change_quantity estimation part 21 in this embodiment calculates. It is a figure which shows the example of the observation direction change amount table which the observation direction change amount memory | storage part 22 in this embodiment memorize | stores. It is a flowchart which shows the process which the observation space vote part 31 in this embodiment performs. It is a figure which shows an example of the voting space which the observation space voting part 31 in this embodiment produces | generates. It is a flowchart which shows the process which the vote counting part 41 in this embodiment performs. It is a figure which shows an example of the process by the geometric verification apparatus 1 in this embodiment. It is a figure which shows the outline | summary of the process which removes a miscorrespondence from the key point detected between two images.

  Hereinafter, a geometric verification apparatus, a geometric verification method, and a program according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of the geometric verification apparatus 1 according to the present embodiment. As shown in the figure, the geometric verification device 1 includes an affine transformation key point extraction / corresponding point determination unit 11, a corresponding point attribute information storage unit 12, an observation direction change amount estimation unit 21, an observation direction change amount storage unit 22, an observation. A space voting unit 31, a voting space information storage unit 32, a vote counting unit 41, and a vote counting result storage unit 42 are provided.

  The affine transformation key point extraction / corresponding point determination unit 11 receives at least two images. The affine transformation key point extraction / corresponding point determination unit 11 detects a key point in each input image using an affine transformation key point detector, and extracts a feature amount of the detected key point. The affine transformation key point extraction / corresponding point determination unit 11 determines attribute information including the observation direction (θ, φ, ψ) for each detected key point.

  Also, the affine transformation key point extraction / corresponding point determination unit 11 determines the corresponding points between the images using the feature quantities of the key points, and associates the corresponding point IDs for identifying the corresponding points with the attribute information of the corresponding points. And stored in the corresponding point attribute information storage unit 12. The affine transformation key point extraction / corresponding point determination unit 11 observes the corresponding point ID and the corresponding point attribute information stored in the corresponding point attribute information storage unit 12 after associating each key point between images. It outputs to the direction change amount estimation part 21. FIG.

  The corresponding point attribute information storage unit 12 stores the corresponding point ID of the corresponding point determined by the affine transformation key point extraction / corresponding point determination unit 11 and the attribute information of the corresponding point in association with each other.

  The observation direction change amount estimation unit 21 inputs the corresponding point ID and the attribute information of the corresponding point from the affine transformation key point extraction / corresponding point determination unit 11. The observation direction change amount estimation unit 21 calculates the change amount of the key point observation direction for each corresponding point from the attribute information including the observation directions (θ, φ, ψ) of three degrees of freedom between the key points forming the corresponding point. The corresponding point ID, the attribute information of the corresponding point, and the change amount of the observation direction are associated with each other and stored in the observation direction change amount storage unit 22. When the observation direction change amount estimation unit 21 calculates the change amount of the observation direction with respect to all the input corresponding point IDs, the observation direction change amount estimation unit 21 associates the corresponding point ID with the attribute information and the change amount of the observation direction and outputs them to the observation space voting unit 31.

  The observation direction change amount storage unit 22 stores the observation direction change amount calculated by the observation direction change amount estimation unit 21, the corresponding point ID, and the corresponding point attribute information in association with each other.

  The observation space voting unit 31 inputs the attribute information associated with the corresponding point ID and the change amount of the observation direction from the observation direction change amount estimation unit 21. The observation space voting unit 31 generates voting space information indicating the voting space based on the observation direction in the voting space information storage unit 32, and performs voting based on the change amount of the observation direction in the voting space to obtain the voting space information. Update. The observation space voting unit 31 associates the corresponding point ID associated with the change amount with a cell in the voting space when voting based on the change amount of the observation direction. The observation space voting unit 31 outputs the voting space information to the vote counting unit 41 when voting is performed using all the input change amounts of the observation direction.

  The voting space information storage unit 32 stores voting space information generated by the observation space voting unit 31. The voting space information is configured, for example, as an array in which an area for storing the number of votes and the corresponding point ID is assigned to each cell obtained with a fixed size in a space defined by a dimension indicating the observation direction.

The vote counting unit 41 inputs the voting space information from the observation space voting unit 31. The vote counting unit 41 reads the number of votes associated with each cell in the input voting space information. When the number of votes read out is larger than a predetermined threshold value, the vote counting unit 41 associates the corresponding point ID associated with the corresponding cell with the number of read votes and stores it in the vote counting result storage unit 42. After reading the number of votes of all cells, the vote-tick unit 41 outputs the corresponding points associated with the corresponding point IDs stored in the vote-tick result storage unit 42 as verified corresponding points, and Output the number of votes.
The vote counting result storage unit 42 stores the corresponding point ID associated with the cell in which more votes than the threshold have been made and the number of votes of the cell in association with each other.

  Hereinafter, details of processing in each unit included in the geometric verification apparatus 1 will be described. FIG. 2 is a flowchart showing processing performed by the affine transformation key point extraction / corresponding point determination unit 11 in the present embodiment. When the processing is started, the affine transformation key point extraction / corresponding point determination unit 11 inputs at least two images (step S11).

The affine transformation key point extraction / corresponding point determination unit 11 extracts affine invariant key points and feature amounts from the input images (step S12). Here, the affine invariant key points are local features included in the image and have invariance to the affine transformation. The affine invariant key points can be extracted from the image by using the method described in Reference Document 1 below, for example. Moreover, you may extract an affine invariant key point from an image using well-known methods, such as MSER, Harris-Affine, and Hessian-Affine, instead of the method described in the reference literature.
[Reference 1] Jean-Michel Morel, Guoshen Yu, "ASIFT: A New Framework for Fully Affine Invariant Image Comparison", Journal SIAM Journal on Imaging Sciences, Volume 2, Issue 2, pp.438-469, April 2009

  The affine transformation key point extraction / corresponding point determination unit 11 determines at least an observation direction (θ, φ, ψ) for each extracted affine invariant key point, and includes attribute information including the determined observation direction (θ, φ, ψ). Is associated with the key point and stored in the corresponding point attribute information storage unit 12 (step S13). FIG. 3 is a diagram showing a coordinate system for determining the observation direction in the present embodiment. As shown in the figure, when an arbitrary point (for example, an affine invariant key point) is set as the origin, the direction in which the origin is observed becomes the observation direction (θ, φ, ψ). Assuming a spherical surface centered on the origin, when the origin is observed from the observation point, the longitude with respect to the position where the straight line connecting the observation point and the origin intersects the spherical surface is θ, and the latitude is φ. ψ is the amount of rotation about a straight line connecting the observation point and the origin. The observation direction (θ, φ, ψ) is a camera posture based on affine invariant key points (feature points). Note that a coordinate system other than the coordinate system shown in FIG. 3 may be used as long as the coordinate system can specify the camera posture based on the affine invariant key point when the image is captured.

  The observation direction (θ, φ, ψ) is obtained by decomposing the transformation matrix A of the following equation (1) indicating the (x, y) coordinate transformation of the affine invariant key points between the images, so that the camera located at the observation point is analyzed. The motion can be obtained as θ, φ, and ψ in the following equation (2). Note that s in Expression (2) is a determinant of the transformation matrix A.

  The determination of θ, φ, and ψ generates an image that simulates θ and φ by rotating the original image or reducing it in the vertical or horizontal direction, as described in Reference 1 in determining affine invariant key points. When extracting local features such as SIFT and SURF that are invariant to rotation around the scale s and ψ from the generated image, the key points including attribute information around the extracted ψ are θ and This can be realized by associating φ. In addition, when using an affine invariant key point detector of a type that detects an affine region, such as MSER, Harris-Affine, Hessian-Affine, and Salient Region, the major axis and minor axis of the detected affine region (elliptical region) This can be realized by calculating θ, φ, and ψ from the ratio of the direction and length. The scale s is a value corresponding to the distance between the affine invariant key point and the camera when the image is captured.

  When the affine transformation key point extraction / corresponding point determination unit 11 associates the attribute information with the key point and stores it in the corresponding point attribute information storage unit 12, the scale s calculated when the affine invariant key point is detected is also included in the attribute information. It may be included. Hereinafter, a case where the scale s is also included in the attribute information will be described.

  The affine transformation key point extraction / corresponding point determination unit 11 determines corresponding points (affine invariant key points combined between images) using the feature quantities of the affine invariant key points between the input images, and corresponding point attribute information. It memorize | stores in the memory | storage part 12 (step S14). The process of determining the corresponding points can be performed using a known method such as the method described in Non-Patent Document 1 or the method described in Reference Document 1.

  The affine transformation key point extraction / corresponding point determination unit 11 outputs the attribute information of the corresponding point associated with the corresponding point ID stored in the corresponding point attribute information storage unit 12 to the observation direction change amount estimation unit 21 (step S15), the process is terminated.

  FIG. 4 is a diagram illustrating an example of a corresponding point attribute table stored in the corresponding point attribute information storage unit 12 according to the present embodiment. The figure shows an example in which two images are input to the affine transformation key point extraction / corresponding point determination unit 11. The corresponding point attribute table has columns of items of corresponding point IDs and attribute information including the coordinates (x, y), s, θ, φ, and ψ of the corresponding points. The coordinates (x, y) of the corresponding points are the coordinates of the affine invariant key points (corresponding points) on the image. s is the above-described scale s, and θ, φ, and ψ are observation directions (θ, φ, ψ). A corresponding point attribute table row exists for each corresponding point, and the attribute information of the corresponding point of one of the two input images and the attribute information of the corresponding point of the other image form one corresponding point ID. Associated. For example, the corresponding point ID “2” includes attribute information (x, y, s, θ, φ, ψ) = (40, 40, 2, 40, 70, 30) of one image and the other. Is associated with attribute information (x ′, y ′, s, θ, φ, ψ) = (15, 20, 3, 60, 100, 90) of the corresponding image.

  FIG. 5 is a flowchart showing processing performed by the observation direction change amount estimation unit 21 in the present embodiment. The observation direction change amount estimation unit 21 inputs the corresponding point ID of the corresponding point detected by the affine transformation key point extraction / corresponding point determination unit 11 and the attribute information of each corresponding point (step S21), and the corresponding point ID and each corresponding point. The point attribute information is associated and stored in the observation direction change amount storage unit 22.

The observation direction change amount estimation unit 21 determines whether or not the process of calculating the observation direction change amount has been completed for all the corresponding point IDs stored in the observation direction change amount storage unit 22 (step S22).
If the processing has not been completed for all corresponding point IDs (step S22: NO), the observation direction change amount estimation unit 21 selects a corresponding point for which the observation direction change amount is not calculated (step S23).
The observation direction change amount estimation unit 21 calculates an observation direction change amount including at least a difference of three degrees of freedom of the observation directions (θ, φ, ψ) from the attribute information of the corresponding points selected in step S23 (step S24). ).

FIG. 6 is a diagram showing an overview of the change amount in the observation direction calculated by the observation direction change amount estimation unit 21 in the present embodiment. Here, it is assumed that an affine invariant key point and its attribute information have already been extracted between image 1 and image 2 obtained by photographing the same subject from different directions. First, consider a point a on a subject in a three-dimensional space from which affine transformation key points are extracted. Among the corresponding points corresponding to the point a, the attribute information of the point a on the image 1 is (θ _1a , φ _1a , ψ _1a ), and the attribute information of the point a on the image 2 is (θ _2a , φ _2a , Ψ _2a ), the observation direction # 1a and the observation direction # 2a correspond to the image 1 and the image 2 on the observation sphere having three degrees of freedom for observing the subject shown in FIG.

  Note that a point b different from the point a on the subject in the three-dimensional space is affected by the projection transformation distortion from the three-dimensional space to the image (two-dimensional space), and the observation direction # 1a and the observation direction # 2a Obtain different observation directions # 1b and # 2b. However, the movement of the camera that captured the image 1 and the image 2 is unique, and therefore the amount of change in the observation direction of the affine transformation key point is constant. That is, if the correspondence between two images is accurate, the amount of change in the observation direction calculated from the attribute information of the corresponding point is always the same value. Conversely, if the correspondence is inaccurate, the amount of change will be different.

The amount of change in viewing direction (Δθ, Δφ, Δψ), the case where the image 2 as a reference can be calculated as _{(Δθ = θ 2a -θ 1a,} Δφ = φ 2a -φ 1a, Δψ = ψ 2a -ψ 1a) . If the image 1 as a reference can be calculated as _{(Δθ = θ 1a -θ 2a,} Δφ = φ 1a -φ 2a, Δψ = ψ 1a -ψ 2a). In order to perform geometric verification in consideration of the change amount of the scale, the change amount Δs = s ₁ −s ₂ or Δs = s ₂ −s ₁ of the scale is calculated and included in the change amount in the observation direction. Good. Further, the change amount (Δx, Δy) of the coordinates (x, y) of the corresponding point on the image may be calculated in the same manner and included in the change amount in the observation direction.

  The observation direction change amount estimation unit 21 stores the calculated observation direction change amount in the observation direction change amount storage unit 22 in association with the corresponding point ID, and sets a processed flag for the corresponding point ID (step S21). S25), the process returns to step S22.

  When it is determined in step S22 that all the corresponding point IDs have been processed (step S22: YES), the observation direction change amount estimation unit 21 stores the corresponding point IDs stored in the observation direction change amount storage unit 22. All of them are associated with their attribute information and the amount of change in the observation direction and output to the observation space voting unit 31 (step S26), and the process is terminated.

  FIG. 7 is a diagram illustrating an example of an observation direction change amount table stored in the observation direction change amount storage unit 22 according to the present embodiment. The observation direction change amount table includes the corresponding point ID, the attribute information including the coordinates (x, y) of the corresponding point, s, θ, φ, and ψ, and the change amount of the observation direction including Δs, Δθ, Δφ, and Δψ. It has a column of each item. A row of the observation direction change amount table exists for each corresponding point ID. The corresponding point ID is associated with attribute information of each affine invariant key point that forms the corresponding point and the amount of change in the observation direction. For example, the corresponding point ID “2” includes attribute information (x, y, s, θ, φ, ψ) = (40, 40, 2, 40, 70, 30) of one key point forming the corresponding point. , Attribute information (x ′, y ′, s, θ, φ, ψ) of the other key point = (15, 20, 3, 60, 100, 90) and the change amount (Δs, Δθ, Δφ in the observation direction) , Δψ) = (1, 20, 30, 60).

FIG. 8 is a flowchart showing processing performed by the observation space voting unit 31 in the present embodiment. The observation space voting unit 31 inputs the attribute information associated with each corresponding point ID and the amount of change in the observation direction from the observation direction change amount estimation unit 21 (step S31).
The observation space voting unit 31 generates, in the voting space information storage unit 32, voting space information indicating a voting space including at least the amount of change (Δθ, Δφ, Δψ) in the observation direction, and information corresponding to each cell in the voting space. Is initialized (step S32).

  FIG. 9 is a diagram illustrating an example of a voting space generated by the observation space voting unit 31 in the present embodiment. The voting space shown in the figure is a three-dimensional space composed of observation directions with three degrees of freedom. The observation space voting unit 31 secures a storage area corresponding to each of a plurality of cells obtained by dividing the three-dimensional space of Δθ, Δφ, and Δψ into a certain size in the voting space information storage unit 32 and voting space information. Is generated. In other words, the voting space information is information that data structures an array determined according to the number of dimensions of the voting space. The observation space voting unit 31 may generate a voting space in a four-dimensional space to which Δs is further added. Further, the voting space may be generated in a six-dimensional space to which Δx and Δy are further added. Alternatively, the observation space voting unit 31 may generate a voting space in a two-dimensional space of Δθ and Δφ. The initialization of the information corresponding to each cell is, for example, to set the number of votes stored in the storage area corresponding to each cell to 0 (zero).

The observation space voting unit 31 determines whether or not the processing has been completed using the amount of change in the observation direction associated with all the input corresponding points (step S33).
When processing has not been completed for all corresponding points (step S33: NO), the observation space voting unit 31 selects a corresponding point that has not been processed (step S34), and is associated with the selected corresponding point. “1” is added to the number of votes of the cell corresponding to the amount of change in the observation direction (step S35).

The observation space voting unit 31 associates the corresponding point ID of the corresponding point selected in step S34 with the cell to which the vote has been added in step S35 (step S36). Specifically, the observation space voting unit 31 stores the corresponding point ID in the storage area corresponding to the cell to which the vote is added.
The observation space voting unit 31 sets a processed flag for the selected corresponding point (step S37), and returns the process to step S33.

  In the determination in step S33, when the processing has been completed for all corresponding points (step S33: YES), the observation space voting unit 31 includes the voting space information stored in the voting space information storage unit 32, The attribute information associated with the corresponding point ID is output to the vote counting unit 41 (step S38), and the process ends.

FIG. 10 is a flowchart showing the processing performed by the vote-tapping unit 41 in the present embodiment. The vote counting unit 41 inputs the voting space information and the attribute information associated with each corresponding point ID from the observation space voting unit 31 (step S41).
The vote-voting unit 41 determines whether or not the processing has been completed for all cells that are included in the input voting space information and that have a vote (step S42).

If the processing has not been completed for all the cells (step S42: NO), the vote-opening unit 41 selects a cell that has not been processed as a processing target and reads out the number of votes from the storage area corresponding to the cell (step S43). .
The vote opening part 41 determines whether or not the read number of votes is larger than a predetermined threshold (step S44), and if the read number of votes is equal to or less than the threshold (step S44: NO), the process proceeds to step S47. The threshold value may be determined according to the number of affine invariant key points detected by the affine transformation key point extraction / corresponding point determination unit 11, for example. Specifically, half of the detected number of corresponding points or a value corresponding to 80% may be used as the threshold value.

If it is determined in step S44 that the number of votes read is larger than the threshold (step S44: YES), the vote-voting unit 41 reads all the corresponding point IDs associated with the cell selected as the processing target from the voting space information. (Step S45).
For each corresponding point ID that has been read, the vote-tick unit 41 associates the number of votes of the selected cell with the processing target and stores it in the vote-tick result storage unit 42 (step S46).
The vote-tick part 41 sets a processed flag for the cell selected as the processing target (step S47), and returns the process to step S42.

  If it is determined in step S42 that the processing has been completed for all the cells (step S42: YES), the vote-voting unit 41 has attribute information of the corresponding points associated with the corresponding point IDs stored in the vote-voting result storage unit 42. Are output as verified corresponding points, and the number of votes associated with the corresponding point ID is output (step S48), and the process is terminated.

  By performing the above-described processing by the vote counting unit 41, a combination of coordinates on the image of the affine invariant key points detected in each of the plurality of images input to the geometric verification device 1 can be obtained as verified corresponding points. .

  According to the geometric verification device 1 of the present embodiment, the change in the observation direction is changed from the change in the other observation direction by voting based on the change in the observation direction and selecting the change in the observation direction based on the voting result. Different, i.e. inaccurate combinations of corresponding points can be excluded.

  FIG. 11 is a diagram illustrating an example of processing performed by the geometric verification apparatus 1 according to the present embodiment. The figure shows (A) an example of corresponding points before geometric verification, that is, the processing result of the affine transformation key point extraction / corresponding point determination unit 11, and (B) an example of corresponding points after geometric verification, that is, the vote counting unit 41. The processing result is shown. As shown in the figure, the change in the observation direction associated with the corresponding point is different from the change in the other observation direction, and only an appropriate corresponding point is obtained as the output of the geometric verification device 1. I understand.

  You may make it implement | achieve the geometric verification apparatus 1 in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” is a program that dynamically holds a program for a short time, like a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  The present invention has been specifically described above based on the example of the embodiment. However, the above description of the embodiment is for explaining the present invention, and limits the invention described in the claims. It should not be construed as reducing the scope. Moreover, each means structure of this invention is not restricted to the above-mentioned embodiment, Of course, various deformation | transformation are possible within the technical scope as described in a claim.

  Although it is possible to verify the consistency of corresponding points obtained from a plurality of images obtained by photographing the same subject by image processing or the like, it can also be applied to indispensable uses.

DESCRIPTION OF SYMBOLS 1 ... Geometric verification apparatus 11 ... Affine transformation key point extraction and corresponding point determination part 12 ... Corresponding point attribute information storage part 21 ... Observation direction change amount estimation part 22 ... Observation direction change amount storage part 31 ... Observation space voting part 32 ... Vote Spatial information storage unit 41 ... voting unit 42 ... voting result storage unit

Claims (6)

Input multiple images obtained by photographing the same subject, determine the observation direction of affine invariant key points extracted in each of the input images based on the images, and correspond affine invariant key points between the images Affine transformation key point extraction / corresponding point determination unit to be used as corresponding points,
An observation direction change amount estimation unit that estimates a difference in observation direction of an affine invariant keypoint at each of the corresponding points as a change amount in the observation direction;
The number of changes in the observation direction contained in the cell obtained by dividing the voting space corresponding to the number of dimensions in the observation direction by a predetermined size is calculated as the number of votes, and the corresponding points corresponding to the change in the observation direction contained in the cell An observation space voting unit that associates
A geometric verification device comprising: a vote-opening unit that outputs a corresponding point associated with a cell whose number of votes is greater than a predetermined threshold. The geometric verification device according to claim 1,
The affine transformation key point extraction / corresponding point determination unit
Determining the three-dimensional camera posture as the observation direction, which is a camera posture when the image is taken and is based on an affine invariant keypoint;
The observation space voting unit
A geometric verification device that performs voting on each cell in a three-dimensional voting space based on the amount of change in observation direction. The geometric verification device according to claim 1,
The affine transformation key point extraction / corresponding point determination unit
A camera posture when the image is taken and a three-dimensional camera posture based on an affine invariant key point, and a scale determined according to a distance between the camera and the affine invariant key point when the image is taken Is determined as the observation direction,
The observation space voting unit
A geometric verification device that performs voting on each cell in a four-dimensional space based on the amount of change in observation direction. The geometric verification device according to claim 1,
The affine transformation key point extraction / corresponding point determination unit
A camera posture when the image is taken and a three-dimensional camera posture based on an affine invariant key point, and a scale determined according to a distance between the camera and the affine invariant key point when the image is taken Is determined as the observation direction,
The observation direction change amount estimation unit
Estimating the difference in the observation direction of the affine invariant key point and the difference in the coordinates of the affine invariant key point in the image as the amount of change in the observation direction,
The observation space voting unit
A geometric verification device that performs voting on each cell in a six-dimensional space based on the amount of change in observation direction. A geometric verification method performed by a geometric verification apparatus that inputs a plurality of images obtained by photographing the same subject,
An affine transformation key point extraction / corresponding point determination step that determines an observation direction of an affine invariant key point extracted in each of the input images based on the image, and associates an affine invariant key point between the images as a corresponding point. When,
An observation direction change amount estimation step for estimating a difference in observation direction of an affine invariant key point at each of the corresponding points as a change amount in the observation direction;
The number of changes in the observation direction contained in the cell obtained by dividing the voting space corresponding to the number of dimensions in the observation direction by a predetermined size is calculated as the number of votes, and the corresponding points corresponding to the change in the observation direction contained in the cell An observation space voting step that associates
And a vote-voting step for outputting corresponding points associated with cells whose number of votes is greater than a predetermined threshold.   The program for functioning a computer as a geometric verification apparatus as described in any one of Claims 1-4.

Images