JP2017138742A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to accurately compare a comparison source image with a comparison destination image.SOLUTION: The present invention extracts local feature points from a comparison source image and a comparison destination image, and calculates their local features. The present invention then determines the local feature points in corresponding relationship with each other on the basis of the calculated local features, and estimates a common area between the comparison source image and comparison destination image.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technique for comparing a comparison source image and a comparison destination image.

  There has been proposed a method of searching for a similar image using a local feature amount (local feature amount) of an image. In this method, first, characteristic points (local feature points) are extracted from an image (Non-Patent Document 1). Next, a feature amount (local feature amount) corresponding to the local feature point is calculated based on the local feature point and surrounding image information (Non-Patent Document 2).

  In the method using the local feature amount, the local feature amount is defined as information including a plurality of elements that are rotation invariant and enlargement / reduction invariant. Thereby, even when the image is rotated, enlarged or reduced, the search can be performed. The local feature amount is generally expressed as a vector. However, it is a theoretical story that the local feature is invariant to rotation and enlargement / reduction. In an actual digital image, the local feature before image rotation and enlargement / reduction processing corresponds to that after processing. Some variation occurs between the local feature amount.

  In order to extract the rotation-invariant local feature value, for example, in Non-Patent Document 2, the main direction is calculated from the pixel pattern of the local region around the local feature point, and the local region is rotated based on the main direction when calculating the local feature value. To normalize the direction. Further, in order to calculate the local feature quantity that does not change in size, the image of different scales is generated internally, and local feature points are extracted from the images of the respective scales and the local feature quantities are calculated. Here, a series of image sets of different scales generated internally is generally called a scale space.

  With the above-described method, a plurality of local feature points are extracted from one image. In image retrieval using local feature amounts, matching is performed by comparing local feature amounts calculated from respective local feature points. A widely used voting method (Patent Document 1) finds a feature point that is similar to a local feature amount of each feature point extracted from a search source image by a nearest threshold process. ”Is voted, and the higher the number of votes, the more similar.

  Furthermore, there is also a RANSAC process that obtains a correspondence relationship between feature points that are similar to each other by a predetermined threshold or more and verifies whether the position information of the pair satisfies the same geometric transformation. In this process, two pairs are randomly selected from pairs of feature points that are similar to each other by a predetermined threshold or more to obtain an affine transformation matrix. Next, it is a method of verifying whether or not the position information of the remaining feature point pairs that are more than a predetermined threshold satisfies the affine transformation matrix, and determining that they match if a predetermined number of threshold pairs are satisfied. (Non Patent Literature 3)

JP 2009-284084 A

C. Harris and M.M. J. et al. Stephens, “A combined corner and edge detector,” In Album Vision Conference, pages 147-152, 1988. David G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, 60, 2 (2004), pp. 199 91-110. M.M. A. Fischler and R.M. C. Bolles, "Random sample consensus: A paradigm formatfitting with applications to imaging analysis," Commun. ACM, no. 24, vol. 6, pp. 381-395, June 1981.

  In the method described in Non-Patent Document 3, when comparing the similarity between the comparison source image and the comparison destination image, the comparison target image and the comparison source image have similar characteristics without specifying a target area. We asked for the correspondence of points. Therefore, the similarity between the comparison source image and the comparison destination image cannot be obtained with high accuracy. Accordingly, an object of the present invention is to make it possible to accurately compare a comparison source image and a comparison destination image.

  In order to solve the above problems, the present invention includes an input unit that inputs a comparison source image and a comparison destination image, an extraction unit that extracts local feature points from the input comparison source image and the comparison destination image, A calculating means for calculating a local feature for the extracted local feature points, and determining the local feature points in a correspondence relationship between the comparison source image and the comparison destination image based on the calculated local features And a first estimation unit that estimates a common area between the comparison source image and the comparison destination image based on the determined local feature points in the determined correspondence relationship. To do.

  According to the above configuration, the present invention can accurately compare the comparison source image and the comparison destination image.

1 is a block diagram showing a hardware configuration of an image processing apparatus according to a first embodiment. FIG. 2 is a block diagram showing a software configuration of the image processing apparatus according to the first embodiment. 5 is a flowchart of comparison processing in the image processing apparatus according to the first embodiment. The figure explaining the outline | summary of the comparison process which concerns on 1st Embodiment. The flowchart of the geometric relationship calculation using the RANSAC process in 1st Embodiment. 10 is a flowchart of comparison processing in the image processing apparatus according to the second embodiment. 10 is a flowchart of comparison processing in the image processing apparatus according to the third embodiment.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. First, the hardware configuration of the image processing apparatus according to the present embodiment will be described with reference to the block diagram of FIG. The image processing apparatus according to the present embodiment includes a server apparatus and a client apparatus. Each of the server device and the client device may be realized by a single computer device, or may be realized by distributing each function to a plurality of computer devices as necessary. When configured by a plurality of computer devices, they are connected by a local area network (LAN) or the like so that they can communicate with each other. The computer device can be realized by an information processing device such as a personal computer (PC) or a workstation (WS).

  In FIG. 1, a CPU 101 is a central processing unit that controls the entire computer apparatus 100. The ROM 102 is a Read Only Memory that stores programs and parameters that do not need to be changed. The RAM 103 is a Random Access Memory that temporarily stores programs and data supplied from an external device. The external storage device 104 is a storage device such as a hard disk or a memory card that is fixedly installed in the computer device 100. The external storage device 104 may include an optical disk such as a flexible disk (FD) and a Compact Disk (CD) that can be detached from the computer apparatus 100, a magnetic or optical card, an IC card, a memory card, and the like. The input device interface 105 is an interface with an input device 109 such as a pointing device or a keyboard that receives data from a user and inputs data.

  The output device interface 106 is an interface with a monitor 110 for displaying data held by the computer apparatus 100 and supplied data. The communication interface 107 is a communication interface for connecting to a network line 111 such as the Internet, a digital camera 112, a digital video camera 113, a smartphone 114, and the like. The system bus 108 is a transmission path that connects the units 101 to 107 so that they can communicate with each other. The processing of each flowchart of the present embodiment, which will be described later, is executed by the CPU 101 executing a program stored in a computer-readable storage medium such as the ROM 102.

  FIG. 2 is a block diagram showing a software configuration of the image processing apparatus according to the present embodiment. In the figure, an image input unit 201 inputs a comparison source image (search source image) and a comparison destination image (registered image). The image feature amount group extraction unit 202 extracts an image feature amount group of the comparison source image and the comparison destination image input from the image input unit 201. The image feature quantity group extraction unit 202 has a function as a feature point extraction unit that extracts local feature points from each image and a feature calculation unit (first calculation unit) that calculates local features for the extracted local feature points. .

  The image feature amount comparison unit 203 is a process of performing RANSAC processing, further comparing feature point coordinates, and confirming whether the search result is appropriate. Specifically, the image feature quantity comparison unit 203 determines corresponding local feature points that are in a correspondence relationship based on the similarity between the local feature quantity of the comparison source image and the local feature quantity of the comparison destination image ( A first determination means). Also, a function as a vote counting means that counts the frequency of feature points that match the converted feature point coordinates by performing affine transformation and the frequency that the coordinates do not match, retry when the number of votes exceeds a predetermined threshold It has the function as a stop means to stop. Further, the image feature amount comparison unit 203 has a function as a common region estimation unit (first estimation unit) that estimates a common region of the comparison source image and the comparison destination image from the voting result. In addition, in the common area, a feature point that has no correspondence based on the similarity between the local feature amount and the local feature amount of the comparison target image and a feature point that performs affine transformation and does not match the feature point coordinates after transformation are determined. It has a function as corresponding point extraction means (second determination means). Also, using these results, a different area estimation means (second estimation means) for obtaining an area having a difference between the comparison source image and the comparison target image, and a similarity score calculation means (a second score estimation means) for performing processing for reflecting the result on the similarity score It also has a function as a second calculation means).

  The comparison result output unit 204 outputs the comparison result of the image feature amount comparison unit 203. The storage unit 205 is a memory, HDD, or the like that stores data being processed. Note that each of these components is centrally controlled by the CPU 101.

  Next, details of the comparison process between the comparison source image and the comparison destination image in the present embodiment will be described. When one or more images similar to the comparison source image (search source image) are searched from a plurality of comparison destination images (registered images), the following comparison processing is performed on each comparison destination image and is calculated. A similar image may be determined from the overall score.

  FIG. 3 is a flowchart illustrating an example of a comparison processing procedure in the image processing apparatus according to the present embodiment. In the figure, first in step S301, the image input unit 201 reads a comparison source image and a comparison destination image. FIG. 4 is a diagram for explaining the outline of the comparison processing according to this embodiment. In this S301, it is assumed that the comparison source image and the comparison destination image as shown in FIG. Note that the comparison destination image may be read in advance before this processing flow, and the information may be registered in the storage unit 205.

  In step S302, the image feature quantity group extraction unit 202 calculates local features for both the comparison source image and the comparison target image. First, the image feature quantity group extraction unit 202 extracts local feature points (local feature points) that are robustly extracted even when the image is rotated. As a method for extracting local feature points, a Harris operator is used here (see C. Harris and MJ Stephens, “A combined corner and edge detector,” In Alvery Vision Conference, pages 147-152, 1988.).

  Specifically, with respect to the pixel on the output image H obtained by applying the Harris operator, the pixel values of the pixel and pixels in the vicinity of the pixel (eight pixels in total) (total nine pixels) are examined. Then, a point at which the pixel becomes a local maximum (a pixel value of the pixel becomes the maximum among the nine pixels) is extracted as a local feature point. Here, even when the pixel reaches a local maximum, it is not extracted as a local feature point if the value of the pixel is less than or equal to the threshold value. Note that any feature point extraction method is applicable as long as it is a method that can extract local feature points, not limited to the above-described feature point extraction method using the Harris operator.

  Then, the image feature quantity group extraction unit 202 calculates the feature quantity (local feature) of the extracted local feature points. As local features, known techniques such as SIFT, SURF, FAST, etc. can be used. In addition, when the comparison destination image is registered in the storage unit 205 in advance, local features may be obtained in advance and stored in the storage unit 205.

  In step S303, the image feature amount comparison unit 203 performs a voting process on the comparison target image from the comparison source image to obtain a pair of the number of votes and corresponding points. Here, first, the local feature of the comparison target image is Vq, the local feature point associated with the local feature is Q, and the coordinates are Q (x ′, y ′). Further, a local feature existing in the comparison source image is Vs, a local feature point associated with the local feature is S, and a coordinate is S (x, y). The image feature quantity comparison unit 203 calculates vector vector feature distances between Vq and Vs for all combinations, and creates a shortest distance corresponding point list. Then, a corresponding point pair of Vq and Vs is extracted such that the calculated distance between the vector features is equal to or less than the threshold value Tv and is the shortest distance.

  In step S304, the image feature amount comparison unit 203 determines whether the number of votes is equal to or greater than a threshold value. If the number of votes is less than the threshold value, the process proceeds to step S305, and the process ends, assuming that the comparison source image and the comparison destination image are not similar.

  On the other hand, if the number of votes is equal to or greater than the threshold, the process proceeds to step S306. In step S306, RANSAC is used for the extracted corresponding point pair to obtain the geometric relationship between the feature point positions of the comparison source image and the comparison destination image, and the correct geometric relationship, that is, the positive corresponding point is determined. Then, the image feature amount comparison unit 203 obtains the number of positive corresponding points and uses it as a pairing score. At the same time, non-positive corresponding points that are feature points that do not satisfy the geometric relationship in the corresponding point pairs are determined, and the number thereof is obtained. The image feature amount comparison unit 203 stores the non-positive corresponding points of the comparison source image and the comparison destination image in the storage unit 205. Details of the process in step S306 will be described later.

  In step S307, the image feature amount comparison unit 203 specifies (estimates) the common area from the positive corresponding points on the comparison target image. Specifically, based on the distribution of the positive corresponding points in the comparison target image, an area including the positive corresponding points is obtained and specified as a common area. As a method for obtaining the region, it is only necessary to simply obtain the circumscribed rectangle including the positive corresponding point. Note that the shape of the region is not limited to a rectangle, but may be another shape such as a polygon or an indefinite shape. In the example shown in FIG. 4, since the comparison source image is a part of a certain document image, the comparison destination image is the entire document image. Therefore, here, a common area is specified on the comparison destination image. That is, it may be determined in advance whether the comparison source image or the comparison destination image specifies the common region, but preferably, the inclusion relationship is estimated between the comparison source image and the comparison destination image, and more regions are determined. It is better to select an image that covers and identify a common area on the image.

  Here, FIG. 4B shows an outline of the processing in steps S302 to S307. In the figure, the points indicated by ○ and × are feature points for obtaining local features. In FIG. 4B, a local feature that is similar in the RANSAC process and satisfies the geometric relationship is indicated by a circle as a positive corresponding point. On the other hand, a point that does not satisfy the geometric relationship even if the local feature is not similar or the local feature is similar is indicated by x as a non-positive corresponding point. The dotted line in the figure represents the correspondence. Further, the common area obtained based on the distribution of the positive corresponding points ◯ by the processing up to step S307 is indicated by a black rectangle on the comparison target image in FIG.

  In the present embodiment, since the common area between the comparison source image and the comparison destination image is determined in this way, the similarity (comprehensive score described later) between the comparison source image and the comparison destination image is calculated. The degree of similarity can be obtained with high accuracy.

  Subsequently, in step S308, the image feature amount comparison unit 203 obtains a feature point group of non-positive corresponding points in the common area specified on the comparison destination image. First, the image feature amount comparison unit 203 obtains a feature point (a feature point having no similar corresponding point) in the common area of the comparison destination image that has not been paired with a non-positive corresponding point of the comparison source image. Furthermore, the geometric transformation obtained in step S306 is performed on the feature points of the comparison source image that did not form a pair of corresponding points with the non-positive corresponding point of the comparison destination image, and projected into the common area of the comparison destination image. Select and ask only for things.

  By calculating the logical sum of these results, a feature point group in which the comparison source image and the comparison destination image do not match in the common region of the comparison source image and the comparison destination image is obtained. FIG. 4C shows the processing result of step S308, and shows a state in which both of the two types of feature points are obtained in the common area indicated by a black frame.

  In the present embodiment, by calculating the logical sum of these two types of feature points, the above feature points are obtained individually for the comparison source image and the comparison destination image, or they do not become non-positive corresponding points or corresponding point pairs. The difference area can be expanded and specified as compared with the case where the obtained feature points are obtained individually. In particular, when the entire image is similar between the comparison source image and the comparison destination image and there is a difference in a part of the area, for example, a case where the background image is the same and only the character portion is different is considered. . In such a case, local feature points cannot be taken so much in characters, and a large difference area may not be obtained, but the difference area is expanded by taking the logical sum of two types of feature points as described above. I can take it.

  The method of determining the difference area based on the two types of non-corresponding points may be determined by applying an image area separation method in OCR, for example. In the image area separation, a projection histogram of black pixels of a binary image in the x direction and the y direction is generated and a region of a character or a figure is estimated. In this embodiment, however, the x direction and y direction in the comparison destination image are not positive. The difference area is determined from the histogram of the corresponding points. For simplicity, it is determined as a circumscribed rectangle including a section in which the value of the histogram is a predetermined value or more in each of the x direction and the y direction. In the present embodiment, for example, as shown in FIG. 4D, one or more region groups are determined as different regions from the histogram of non-positive corresponding points in the x direction and the y direction.

  In step S309, the image feature amount comparison unit 203 obtains the number of non-positive corresponding points in each difference area determined on the comparison target image. Then, an area where the non-positive corresponding point is less than the predetermined threshold is excluded from the difference area. This is because, for example, the influence of the feature points caused by irregularity due to paper stains of the document document is eliminated, and the difference area is limited to an area having a certain size such as characters. Here, the number of non-corresponding corresponding points is obtained in each different area, but the number of non-positive corresponding points (density) for each different area is calculated to exclude those having a density lower than a predetermined value. Good.

  In step S310, the image feature quantity comparison unit 203 calculates the sum of the number of feature points that did not form a pair of non-positive corresponding points and corresponding point pairs from each of the region groups specified in step S309. In step S <b> 311, the image feature quantity comparison unit 203 calculates a total score by subtracting a penalty score corresponding to the total number of non-positive corresponding points from the number of positive corresponding points, that is, the pairing score. When strongly reflecting the difference area in the comparison result, the total number of the non-correct corresponding points should be subtracted from the number of positive corresponding points. When the difference area is weakly reflected, the non-correct corresponding points What is necessary is just to subtract the thing which multiplied the count of less than 1 to the sum total of the number of positive correspondence points. Then, the comparison result output unit 204 outputs the calculated total score as a comparison result between the comparison source image and the comparison destination image. For example, when searching for a predetermined number of images similar to the comparison source image by calculating a total score of a plurality of comparison destination images with respect to the comparison source image, the comparison result output unit 204 calculates with this flow. The total score thus output is output to be temporarily stored in the storage unit 205.

  Through the above comparison processing, in this embodiment, it is possible to calculate a total score that more sensitively reflects the difference in the common area than the pairing score based on the number of positive corresponding points using simple RANSAC. Therefore, even when comparing images that are generally similar and partially different, for example, when comparing images that have the same background pattern and different character strings. Thus, it is possible to obtain a total score reflecting the difference in the character string that is the difference portion. Thereby, the accuracy of the comparison result between the comparison source image and the comparison destination image can be increased.

  Next, details of the processing in step S306 described above will be described with reference to FIG. FIG. 5 is a flowchart for calculating the geometric relationship between the feature point positions of the comparison source image and the comparison destination image using the RANSAC process.

  First, symbols are defined. It is assumed that the local feature of the comparison target image is Vq, the local feature point associated with the local feature is Q, and the coordinate is Q (x ′, y ′). Further, a local feature existing in the comparison source image is Vs, a local feature point associated with the local feature is S, and a coordinate is S (x, y).

  In step S501, a variable VoteMax representing the final number of votes is initialized to zero. Next, in step S502, the shortest distance corresponding point list which is the processing result of S303 is read. The content is repeated, but is a corresponding point pair of Vq and Vs such that the calculated distance between the vector features is equal to or less than the threshold value Tv and the shortest distance.

Thereafter, for the kth corresponding point registered in the shortest distance corresponding point list, the local features of the corresponding point are described as Vq (k) and Vs (k), respectively, and the distance between the vector features is equal to or less than the threshold Tv. Let the number of features be Nq (k) and Ns (k), respectively. Further, local feature points associated with Vq (k) and Vs (k) are subscripts such as Qk and Sk, coordinates are Qk (x ′ _k , y ′ _k ), Sk (x _k , y _k ), and the like. Are described together. Further, the number of pairs of corresponding points registered in the shortest distance corresponding point list created in step S303 is m.

  In step S503, a variable Count indicating the number of iterations of the similarity calculation process is initialized to zero. In step S504, it is determined whether the iteration count number Count has not exceeded a predetermined maximum number of iterations Rn. Here, when it exceeds, it progresses to step S521, the final vote number VoteMax is output, and this process is complete | finished.

  If it is determined in step S504 that the iteration count number Count does not exceed the maximum number of iterations Rn, the process proceeds to step S505, and a variable Vote indicating the number of votes is initialized to zero. Next, in step S506, two sets of coordinates of corresponding point pairs are randomly extracted from the shortest distance corresponding point list.

Here, these coordinates are described as Q1 (x ′ ₁ , y ′ ₁ ), S1 (x ₁ , y ₁ ), Q2 (x ′ ₂ , y ′ ₂ ), and S2 (x ₂ , y ₂ ). Next, in step S507, the extracted Q1 (x ′ ₁ , y ′ ₁ ), S1 (x ₁ , y ₁ ), Q2 (x ′ ₂ , y ′ ₂ ), and S2 (x ₂ , y ₂ ) Assuming that the conversion shown in (1) is satisfied, f is obtained from the variable a in (Formula 1). However, in step S507 shown in FIG. 5, a matrix composed of variables a to d is denoted by M, and a matrix composed of variables e to f is denoted by T.

ここで、第１の実施形態では、簡略化のため、相似変換だけを考える。このとき、上記（数式１）は以下の（数式２）のように書き換えられる。

Here, in the first embodiment, only the similarity transformation is considered for simplification. At this time, the above (Formula 1) is rewritten as the following (Formula 2).

このとき、変数ａ、ｂ、ｅ、ｆはｘ’_１、ｙ’_１、ｘ_１、ｙ_１、ｘ’_２、ｙ’_２、ｘ_２、ｙ_２を使って式（３）から式（６）で表される。

At this time, the variables a, b, e, and f are changed from Equation (3) to Equation (6) using x ′ ₁ , y ′ ₁ , x ₁ , y ₁ , x ′ ₂ , y ′ ₂ , x ₂ , y _2. ).

次に、投票数集計手段の説明をステップＳ５０８からＳ５１７までの処理を用いて行う。

Next, the vote counting means will be described using the processing from step S508 to S517.

  In step S508, the corresponding point selection variable k is initialized to 3 in order to select a point other than the two sets of points selected from the shortest distance corresponding point list in step S505 described above. In step S509, it is determined whether or not the corresponding point selection variable k exceeds the number m of corresponding points registered in the shortest distance corresponding point list. Here, if it exceeds, the process moves to step S518, which will be described later. If it is determined in step S509 that the corresponding point selection variable k does not exceed the number m of corresponding points registered in the shortest distance corresponding point list, the process proceeds to step S510.

In this step S510, points other than the two sets of points S1 (x ₁ , y ₁ ) and S2 (x ₂ , y ₂ ) randomly extracted from the shortest distance corresponding point list in step S505 described above correspond to the shortest distance. Extract from the point list. In the first embodiment, the extracted point is described as Sk (x _k , y _k ).

Next, in step S511, coordinates Sk ′ (x ′ _k , y ′ _k ) to which Sk (x _k , y _k ) is transferred using the equation (2) are _obtained .

Thereafter, in step S512, the geometric distance between the coordinates Sk ′ (x ′ _k , y ′ _k ) and the coordinates Qk (x ′ _k , y ′ _k ) is calculated as the Euclidean distance, and the Euclidean distance is calculated from the threshold Td. A comparison is made to determine whether it is below the threshold. If the Euclidean distance is less than or equal to the threshold value Td, the process proceeds to step S513, the vote number Vote is incremented, and the coordinates of S′k are stored as positive corresponding points in step S514. Then, the process proceeds to step S515.

  If the Euclidean distance is greater than the threshold value Td, NG_Vote, which is the number of feature points that did not satisfy the geometrical relationship in step S516, is incremented. Remember as. Then, the process proceeds to step S515.

  In step S515, the corresponding point selection variable k is incremented, and the process returns to step S509 to extract an unprocessed pair on the shortest distance corresponding point pair list, and the corresponding point selection variable k is registered in the shortest distance corresponding point list. The above-described processing is repeated until the number m of corresponding points is exceeded.

  Next, step S518, which is a process when the corresponding point selection variable k exceeds the number m of corresponding points registered in the shortest distance corresponding point list in step S509, will be described. In step S518, the value of the vote number Vote and the value of the final vote number VoteMax are compared. If the value of the vote number Vote is larger than the value of the final vote number VoteMax, the process proceeds to step S519.

  In step S519, after the value of the final vote number VoteMax is replaced with the value of the vote number Vote, the iteration count number Count is incremented in step S520, and the process returns to step S504 described above.

  If the value of the vote number Vote is equal to or smaller than the value of the final vote number VoteMax in step S518, the process proceeds to step S520, the repeat count number Count is incremented, and the process returns to step S504 described above.

  In the description of the score calculation method according to the first embodiment, only the similarity transformation has been considered. However, for other geometric transformations such as affine transformation, a transformation matrix corresponding to each is obtained in step S507. It is possible to respond. For example, in the case of affine transformation, first, in step S506, the number of coordinates of the set of corresponding points is set to 3. Next, in step S507, equation (1) is used instead of equation (2), and the three sets of corresponding points selected in step S506 (6 points in total) may be used to obtain f from variable a.

  As described above, according to the present embodiment, the common region is determined based on the correspondence relationship between the local feature points of the comparison source image and the comparison target image, and the comparison source image and the comparison source image are determined based on the correspondence relationship of the local feature points in the common region. The degree of similarity of the comparison target images is obtained and compared. As a result, it is possible to accurately obtain a comparison result between the comparison source image and the comparison destination image.

  Further, in the present embodiment, a difference area that is a difference between the comparison source image and the comparison destination image is obtained from the common area, and the non-corresponding relationship in the difference area is reflected in the comparison process. The accuracy of the comparison process can be further increased.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. The second embodiment is characterized in that a score corresponding to the sum of the areas of the different area groups with respect to the area of the common area is used as a penalty score and is subtracted from the total score. Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. In addition, about the structure already demonstrated in 1st Embodiment, the description is abbreviate | omitted and the same code | symbol is attached | subjected.

  FIG. 6 is a flowchart of the comparison process according to the present embodiment. In FIG. 6, the processes in steps S601 to S609 are the same as the processes in steps S301 to S309 in FIG.

  In step S610, the image feature amount comparison unit 203 calculates the area of each different area specified by the processing up to step S609. Here, the total number of pixels included in all the different regions is calculated using the number of pixels constituting the region as the area of the region. In addition, the number of pixels is similarly obtained for the area of the common region. Then, the total score is obtained by subtracting the total number of pixels in the different area from the number of pixels in the common area. When the difference area is strongly reflected in the result of the comparison process, the total number of pixels in all the difference areas is subtracted from the number of pixels in the common area, and when the difference area is weakly reflected, the total number of pixels in all the difference areas is 1 What is necessary is just to subtract the number multiplied by less than the number of pixels in the common area.

  As described above, according to the present embodiment, the area in the difference area is used as the penalty score, not the number of non-positive corresponding points in the difference area. Therefore, a result reflecting the non-corresponding relationship in the different area by the comparison process can be obtained as compared with the first embodiment.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. In the third embodiment, the size of the common area of the comparison source image and the comparison destination image is matched, the histogram of each RGB color channel of the pixels in the common area is smoothed, and then the difference between the two at the pixel level Is calculated. The third embodiment of the present invention will be described below with reference to the drawings. In addition, about the structure already demonstrated by 1st, 2nd embodiment, the description is abbreviate | omitted and the same code | symbol is attached | subjected.

  FIG. 7 is a flowchart of the comparison process according to the present embodiment. In FIG. 7, the processing of steps S701 to S707 is the same as the processing of steps S301 to S307 in FIG.

  In step S708, the image feature amount comparison unit 203 normalizes the size based on the common area obtained in step S707 so that the common area of the comparison source image and the common area of the comparison destination image have the same size. As described above, the image feature amount comparison unit 203 according to the present embodiment has a function as a normalization unit that normalizes the size of the common area.

  In step S709, the image feature amount comparison unit 203 performs preprocessing on the common area of the comparison source image and the comparison target image, specifically, smoothes the histogram in each RGB color channel. For the smoothing of the color histogram, a known technique may be used. For example, the histogram can be smoothed based on Equation 7 below. As described above, the image feature amount comparison unit 203 according to the present embodiment has a function as a preprocessing unit that performs preprocessing (pixel value smoothing) on a common area whose size is normalized.

上記の数式７は、階級値ｄが０から２５５とした場合の平滑化後の階級値ｄ１を表す。ここで、ＳＩＺＥは総画素数、Ｐ（ｉ）は階級値ｉの画素頻度である。比較元画像の共通領域と比較先画像の共通領域の双方に対してｄ＝０から２５５まで変化させた時のｄ１の値を求めておき、ｄとｄ１の対応をルックアップテーブルとして作成しておく。図７に示す比較処理を実行する際には、記憶部２０５に予め作成しておいたルックアップテーブルを参照して画素値を変換（平滑化）することで、処理の高速化を図ることが可能となる。

The above formula 7 represents the class value d1 after smoothing when the class value d is 0 to 255. Here, SIZE is the total number of pixels, and P (i) is the pixel frequency of the class value i. The value of d1 when d = 0 to 255 is obtained for both the common area of the comparison source image and the common area of the comparison destination image, and the correspondence between d and d1 is created as a lookup table. deep. When the comparison process shown in FIG. 7 is executed, it is possible to increase the processing speed by converting (smoothing) pixel values with reference to a lookup table created in advance in the storage unit 205. It becomes possible.

  In step S710, the image feature amount comparison unit 203 calculates the absolute value of the difference in each RGB color channel with respect to the pixel at the corresponding position in the comparison source image and the comparison destination image, and accumulates this to obtain the similarity distance. Since this value is more similar as it is closer to 0, in this embodiment, the value corresponding to the calculated similarity distance is used as the total score.

  As described above, according to the present embodiment, a common area is obtained between the comparison source image and the comparison destination image, the normalization process is performed so that the size of the common area is the same, and the normalization process is performed and then the pixel value is smoothed. I am trying to process it. With this configuration, for example, a comparison result can be obtained with high accuracy even in comparison between images having different conditions, such as an image in which the comparison source image is a scan image and the comparison destination image is rasterized original electronic data.

[Other Embodiments]
In each of the above-described embodiments, SIFT or SURF obtained from a luminance image is given as an example of the feature amount as a local feature. However, a local feature in a color difference signal or an RGB color channel can also be used. In each of the above embodiments, an example in which the correspondence between local feature points is obtained by RANSAC processing has been described. However, if the correspondence between local feature points can be obtained, another conventional technique may be used. .

  In the first and second embodiments, the number of feature points having a pair of the comparison source image and the comparison destination image in the common region and the number of feature points having no pair are simply determined without specifying the difference region. It is also conceivable to calculate the total score from these values. For example, the total score may be determined using a value obtained by subtracting the number of feature points having no pair from the number of feature points having a pair.

  In the description of the third embodiment, an example is given in which histogram smoothing is performed for each color channel of RGB, but histogram smoothing is performed only with luminance information, and absolute values of pixel differences are accumulated. Also good. Furthermore, binarization processing may be performed on the luminance distribution of the common area with a predetermined threshold value to obtain a difference in the binary image.

  In addition, the present invention supplies software (program) for realizing the functions of the above-described embodiments to a system or apparatus via a network or various storage media, and the computer of the system or apparatus (or CPU, MPU, etc.) programs Is read and executed. Further, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. The present invention is not limited to the above embodiments, and various modifications (including organic combinations of the embodiments) are possible based on the spirit of the present invention, and these are excluded from the scope of the present invention. is not. That is, the present invention includes all the combinations of the above-described embodiments and modifications thereof.

DESCRIPTION OF SYMBOLS 201 Image input part 202 Image feature-value group extraction part 203 Image feature-value comparison part 204 Comparison result output part 205 Storage part

Claims (11)

An input means for inputting a comparison source image and a comparison destination image;
Extraction means for extracting local feature points from the input comparison source image and comparison destination image;
Calculating means for calculating local features for the extracted local feature points;
First determination means for determining the local feature point in a correspondence relationship between the comparison source image and the comparison target image based on the calculated local feature;
First estimation means for estimating a common area between the comparison source image and the comparison destination image based on the determined local feature points in the correspondence relationship;
An image processing apparatus comprising: Second determining means for determining local feature points having no correspondence relationship between the comparison source image and the comparison destination image in the estimated common region;
Second estimation means for estimating a difference area between the comparison source image and the comparison destination image based on the determined local feature points having no corresponding relationship;
The image processing apparatus according to claim 1, further comprising:   The image processing apparatus according to claim 2, wherein the second estimation unit estimates a rectangle including the determined local feature point having no corresponding relationship as the difference area.   First, a score related to the similarity between the input comparison source image and the comparison destination image is calculated based on the determined local feature point in the correspondence relationship and the determined local feature point without the correspondence relationship. The image processing apparatus according to claim 2, further comprising: 2 calculating means.   The second calculating means calculates the score based on the number of local feature points in the determined correspondence relationship and the number of local feature points not in the determined correspondence relationship. The image processing apparatus according to claim 4.   The second calculating means calculates the score based on an area of the local feature points in the determined correspondence relationship and an area of the local feature points not in the determined correspondence relationship. The image processing apparatus according to claim 4. Normalizing means for normalizing the size of the common area of the estimated comparison source image and the comparison destination image;
The image processing apparatus according to claim 1, further comprising preprocessing means for performing preprocessing on the normalized comparison source image and comparison destination image.   The image processing apparatus according to claim 7, wherein the preprocessing unit performs pixel value smoothing as the preprocessing.   The apparatus further comprises second calculating means for calculating a score related to the similarity between the input comparison source image and the comparison target image based on the absolute value of the smoothed pixel value. Item 9. The image processing apparatus according to Item 8. Enter the comparison source image and the comparison destination image,
Extracting local feature points from the input comparison source image and the comparison destination image,
Calculating a local feature for the extracted local feature points;
Based on the calculated local feature, determine the local feature point in a correspondence relationship between the comparison source image and the comparison destination image,
Estimating a common area between the comparison source image and the comparison destination image based on the local feature points in the determined correspondence relationship;
An image processing method.   A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 9.

Images