JP2017138743A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to accurately compare a comparison source image with a comparison destination image.SOLUTION: The present invention estimates a difference area between a comparison source image and a comparison destination image, and executes OCR processing on the difference area.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technique for comparing a comparison source image and a comparison destination image.

  There has been proposed a method of searching for a similar image using a local feature amount (local feature amount) of an image. In this method, first, characteristic points (local feature points) are extracted from an image (Non-Patent Document 1). Next, a feature amount (local feature amount) corresponding to the local feature point is calculated based on the local feature point and surrounding image information (Non-Patent Document 2).

  In the method using the local feature amount, the local feature amount is defined as information including a plurality of elements that are rotation invariant and enlargement / reduction invariant. Thereby, even when the image is rotated, enlarged or reduced, the search can be performed. The local feature amount is generally expressed as a vector. However, it is a theoretical story that the local feature is invariant to rotation and enlargement / reduction. In an actual digital image, the local feature before image rotation and enlargement / reduction processing corresponds to that after processing. Some variation occurs between the local feature amount.

  In order to extract the rotation-invariant local feature value, for example, in Non-Patent Document 2, the main direction is calculated from the pixel pattern of the local region around the local feature point, and the local region is rotated based on the main direction when calculating the local feature value. To normalize the direction. Further, in order to calculate the local feature quantity that does not change in size, the image of different scales is generated internally, and local feature points are extracted from the images of the respective scales and the local feature quantities are calculated. Here, a series of image sets of different scales generated internally is generally called a scale space.

  With the above-described method, a plurality of local feature points are extracted from one image. In image retrieval using local feature amounts, matching is performed by comparing local feature amounts calculated from respective local feature points. A widely used voting method (Patent Document 1) finds a feature point that is similar to a local feature amount of each feature point extracted from a search source image by a nearest threshold process. ”Is voted, and the higher the number of votes, the more similar.

  Furthermore, there is also a RANSAC process that obtains a correspondence relationship between feature points that are similar to each other by a predetermined threshold or more and verifies whether the position information of the pair satisfies the same geometric transformation. In this process, two pairs are randomly selected from pairs of feature points that are similar to each other by a predetermined threshold or more to obtain an affine transformation matrix. Next, it is a method of verifying whether or not the position information of the remaining feature point pairs that are more than a predetermined threshold satisfies the affine transformation matrix, and determining that they match if a predetermined number of threshold pairs are satisfied. (Non Patent Literature 3)

JP 2009-284084 A

C. Harris and M.M. J. et al. Stephens, “A combined corner and edge detector,” In Album Vision Conference, pages 147-152, 1988. David G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, 60, 2 (2004), pp. 199 91-110. M.M. A. Fischler and R.M. C. Bolles, "Random sample consensus: A paradigm formatfitting with applications to imaging analysis," Commun. ACM, no. 24, vol. 6, pp. 381-395, June 1981.

  In the method described in Patent Document 1, the degree of similarity is compared and evaluated based on corresponding feature points between images of a comparison source image (search source image) and a comparison destination image (registered image). Therefore, for example, when targeting an image including characters, if the number of feature points extracted from the character portion is relatively smaller than the feature points included in the natural image, background image, and background pattern in the image, Differences in character parts are absorbed. Specifically, in image comparisons such as flyers, pamphlets, and guides, where only the characters in the same background are different, the difference in the feature points of the character part is not easily reflected and is similar to the comparison source image. There is a possibility that a plurality of images may be searched as images to be performed. Therefore, an object of the present invention is to make it possible to accurately compare a comparison source image and a comparison target image even in comparison between images in which most of the images are similar and partially different.

  In order to solve the above-described problem, the present invention provides an input unit that inputs a comparison source image and a comparison destination image, an estimation unit that estimates a difference area between the input comparison source image and the comparison destination image, And an OCR processing unit that performs an OCR process on the estimated difference area.

  According to the above configuration, the present invention can accurately compare the comparison source image and the comparison destination image.

1 is a block diagram showing a hardware configuration of an image processing apparatus according to a first embodiment. FIG. 2 is a block diagram showing a software configuration of the image processing apparatus according to the first embodiment. 5 is a flowchart of comparison processing in the image processing apparatus according to the first embodiment. The figure explaining the outline | summary of the comparison process which concerns on 1st Embodiment. The flowchart of the geometric relationship calculation using the RANSAC process in 1st Embodiment.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. First, the hardware configuration of the image processing apparatus according to the present embodiment will be described with reference to the block diagram of FIG. The image processing apparatus according to the present embodiment includes a server apparatus and a client apparatus. Each of the server device and the client device may be realized by a single computer device, or may be realized by distributing each function to a plurality of computer devices as necessary. When configured by a plurality of computer devices, they are connected by a local area network (LAN) or the like so that they can communicate with each other. The computer device can be realized by an information processing device such as a personal computer (PC) or a workstation (WS).

  In FIG. 1, a CPU 101 is a central processing unit that controls the entire computer apparatus 100. The ROM 102 is a Read Only Memory that stores programs and parameters that do not need to be changed. The RAM 103 is a Random Access Memory that temporarily stores programs and data supplied from an external device. The external storage device 104 is a storage device such as a hard disk or a memory card that is fixedly installed in the computer device 100. The external storage device 104 may include an optical disk such as a flexible disk (FD) and a Compact Disk (CD) that can be detached from the computer apparatus 100, a magnetic or optical card, an IC card, a memory card, and the like. The input device interface 105 is an interface with an input device 109 such as a pointing device or a keyboard that receives data from a user and inputs data.

  The output device interface 106 is an interface with a monitor 110 for displaying data held by the computer apparatus 100 and supplied data. The communication interface 107 is a communication interface for connecting to a network line 111 such as the Internet, a digital camera 112, a digital video camera 113, a smartphone 114, and the like. The system bus 108 is a transmission path that connects the units 101 to 107 so that they can communicate with each other. The processing of each flowchart of the present embodiment, which will be described later, is executed by the CPU 101 executing a program stored in a computer-readable storage medium such as the ROM 102.

  FIG. 2 is a block diagram showing a software configuration of the image processing apparatus according to the present embodiment. In the figure, an image input unit 201 inputs a comparison source image (search source image) and a comparison destination image (registered image). The image feature amount group extraction unit 202 extracts an image feature amount group of the comparison source image and the comparison destination image input from the image input unit 201. That is, the image feature quantity group extraction unit 202 performs feature point extraction on the comparison source image and the comparison destination image, and performs extraction processing of rotation / scaling invariant feature quantities such as SIFT from pixels near the feature point. The image feature quantity group extraction unit 202 has a function as a feature point extraction unit that extracts local feature points from each image and a feature calculation unit (first calculation unit) that calculates local features for the extracted local feature points. .

  In the present embodiment, a case where two images of a comparison source image and a comparison destination image are compared will be described. However, the present invention is not limited to this, and there are a plurality of comparison destination images. By calculating feature amounts in advance for the comparison destination image set, the feature amount group of the comparison source images and the calculated features You may make it compare with a quantity group. At that time, for each feature quantity of the comparison source image, a set of nearest neighbor points existing within a predetermined threshold is obtained, and when the nearest neighbor point includes many feature quantities derived from the same comparison destination image, the comparison destination image As a part of the similar image search process that regards as a similar image candidate, a process described later may be applied.

  The image feature amount comparison unit 203 is a process of performing RANSAC processing, further comparing feature point coordinates, and confirming whether the search result is appropriate. Specifically, the image feature quantity comparison unit 203 determines corresponding local feature points that are in a correspondence relationship based on the similarity between the local feature quantity of the comparison source image and the local feature quantity of the comparison destination image ( A first determination means). Also, a function as a vote counting means that counts the frequency of feature points that match the converted feature point coordinates by performing affine transformation and the frequency that the coordinates do not match, retry when the number of votes exceeds a predetermined threshold It has the function as a stop means to stop. Further, the image feature amount comparison unit 203 has a function as a common region estimation unit (first estimation unit) that estimates a common region of the comparison source image and the comparison destination image from the voting result. In addition, in the common area, a feature point that has no correspondence based on the similarity between the local feature amount and the local feature amount of the comparison target image and a feature point that performs affine transformation and does not match the feature point coordinates after transformation are determined. It has a function as corresponding point extraction means (second determination means). Moreover, it also has a function as a different area estimation means (second estimation means) for obtaining an area having a difference between the comparison source image and the comparison destination image using these results.

  The image erecting processing unit 204 performs a conversion process such as rotation / enlargement / reduction on the comparison source image using a geometric conversion coefficient such as an affine transformation obtained by the image feature amount comparison unit 203, and aligns with the comparison target image. Do. In this embodiment, a use case of scanned image data as a comparison source image and rasterized document image data as a comparison destination image is considered, and the comparison destination image is upright as an original image of a printed material Can be assumed. In such a case, by aligning with the comparison destination image, it is possible to erect even if the comparison source image is rotated. Thereby, the OCR processing performed by the OCR processing unit 205 described later can be performed with high accuracy.

  The OCR processing unit 205 extracts a partial image from the comparison destination image for the difference area obtained by the image feature amount comparison unit 203. At the same time, a partial image is extracted for the same region from the converted comparison source image that has been aligned by the image erecting processing unit 204. Then, OCR processing is performed on each partial image to obtain character recognition information of the partial image.

  The OCR result comparison unit 206 compares the character recognition information of the difference part between the comparison source image and the comparison target image obtained by the OCR processing unit 205, and the difference in character information in the region where there is a difference in the feature amount comparison. Scrutinize. Then, the output unit 207 outputs the comparison result of the OCR result comparison unit 206. The storage unit 208 is a memory, HDD, or the like that stores data being processed. Note that each of these components is centrally controlled by the CPU 101.

  Next, details of the comparison process between the comparison source image and the comparison destination image in the present embodiment will be described. FIG. 3 is a flowchart illustrating an example of a comparison processing procedure in the image processing apparatus according to the present embodiment. In the figure, first in step S301, the image input unit 201 reads a comparison source image and a comparison destination image. FIG. 4 is a diagram for explaining the outline of the comparison processing according to this embodiment. In this S301, it is assumed that the comparison source image and the comparison destination image as shown in FIG. Note that the comparison target image may be read in advance before this processing flow, and the information may be registered in the storage unit 208.

  In step S302, the image feature quantity group extraction unit 202 calculates local features for both the comparison source image and the comparison target image. First, the image feature quantity group extraction unit 202 extracts local feature points (local feature points) that are robustly extracted even when the image is rotated. As a method for extracting local feature points, a Harris operator is used here (see C. Harris and MJ Stephens, “A combined corner and edge detector,” In Alvery Vision Conference, pages 147-152, 1988.).

  Specifically, with respect to the pixel on the output image H obtained by applying the Harris operator, the pixel values of the pixel and pixels in the vicinity of the pixel (eight pixels in total) (total nine pixels) are examined. Then, a point at which the pixel becomes a local maximum (a pixel value of the pixel becomes the maximum among the nine pixels) is extracted as a local feature point. Here, even when the pixel reaches a local maximum, it is not extracted as a local feature point if the value of the pixel is less than or equal to the threshold value. Note that any feature point extraction method is applicable as long as it is a method that can extract local feature points, not limited to the above-described feature point extraction method using the Harris operator.

  Then, the image feature quantity group extraction unit 202 calculates the feature quantity (local feature) of the extracted local feature points. As local features, known techniques such as SIFT, SURF, FAST, etc. can be used. When the comparison destination image is registered in the storage unit 208 in advance, local features may be obtained in advance and stored in the storage unit 208.

  In step S303, the image feature amount comparison unit 203 performs a voting process on the comparison target image from the comparison source image to obtain a pair of the number of votes and corresponding points. Here, first, the local feature of the comparison target image is Vq, the local feature point associated with the local feature is Q, and the coordinates are Q (x ′, y ′). Further, a local feature existing in the comparison source image is Vs, a local feature point associated with the local feature is S, and a coordinate is S (x, y). The image feature quantity comparison unit 203 calculates vector vector feature distances between Vq and Vs for all combinations, and creates a shortest distance corresponding point list. Then, a corresponding point pair of Vq and Vs is extracted such that the calculated distance between the vector features is equal to or less than the threshold value Tv and is the shortest distance.

  In step S304, the image feature amount comparison unit 203 determines whether the number of votes is equal to or greater than a threshold value. If the number of votes is less than the threshold value, the process proceeds to step S305, and the process ends, assuming that the comparison source image and the comparison destination image are not similar.

  On the other hand, if the number of votes is equal to or greater than the threshold, the process proceeds to step S306. In step S306, RANSAC is used for the extracted corresponding point pair to obtain the geometric relationship between the feature point positions of the comparison source image and the comparison destination image, and the correct geometric relationship, that is, the positive corresponding point is determined. Then, the image feature amount comparison unit 203 obtains the number of positive corresponding points and uses it as a pairing score. At the same time, non-positive corresponding points that are feature points that do not satisfy the geometric relationship in the corresponding point pairs are determined, and the number thereof is obtained. The image feature amount comparison unit 203 stores the non-positive corresponding points of the comparison source image and the comparison destination image in the storage unit 208. Details of the process in step S306 will be described later.

  In step S <b> 307, the image feature amount comparison unit 203 specifies a common area from the positive corresponding points on the comparison target image. Specifically, based on the distribution of the positive corresponding points in the comparison target image, an area including the positive corresponding points is obtained and specified as a common area. As a method for obtaining the region, it is only necessary to simply obtain the circumscribed rectangle including the positive corresponding point. Note that the shape of the region is not limited to a rectangle, but may be another shape such as a polygon or an indefinite shape. In the example shown in FIG. 4, since the comparison source image is a part of a certain document image, the comparison destination image is the entire document image. Therefore, here, a common area is specified on the comparison destination image. That is, it may be determined in advance whether the comparison source image or the comparison destination image specifies the common region, but preferably, the inclusion relationship is estimated between the comparison source image and the comparison destination image, and more regions are determined. It is better to select an image that covers and identify a common area on the image.

  Here, FIG. 4B shows an outline of the processing in steps S302 to S307. In the figure, the points indicated by ○ and × are feature points for obtaining local features. In FIG. 4B, a local feature that is similar in the RANSAC process and satisfies the geometric relationship is indicated by a circle as a positive corresponding point. On the other hand, a point that does not satisfy the geometric relationship even if the local feature is not similar or the local feature is similar is indicated by x as a non-positive corresponding point. The dotted line in the figure represents the correspondence. Further, the common area obtained based on the distribution of the positive corresponding points ◯ by the processing up to step S307 is indicated by a black rectangle on the comparison target image in FIG.

  Subsequently, in step S308, the image feature amount comparison unit 203 obtains a feature point group of non-positive corresponding points in the common area specified on the comparison destination image. First, the image feature amount comparison unit 203 obtains a feature point (a feature point having no similar corresponding point) in the common area of the comparison destination image that has not been paired with a non-positive corresponding point of the comparison source image. Furthermore, the geometric transformation obtained in step S306 is performed on the feature points of the comparison source image that did not form a pair of corresponding points with the non-positive corresponding point of the comparison destination image, and projected into the common area of the comparison destination image. Select and ask only for things. In the present embodiment, feature points that did not become a corresponding point pair, that is, feature points that did not have a corresponding point within the threshold Tvec are included in the non-positive corresponding points.

  By calculating the logical sum of these results, a feature point group in which the comparison source image and the comparison destination image do not match in the common region of the comparison source image and the comparison destination image is obtained. FIG. 4C shows the processing result of step S308, and shows a state in which both of the two types of feature points are obtained in the common area indicated by a black frame.

  In the present embodiment, by calculating the logical sum of these two types of feature points, the above feature points are obtained individually for the comparison source image and the comparison destination image, or they do not become non-positive corresponding points or corresponding point pairs. The difference area can be expanded and specified as compared with the case where the obtained feature points are obtained individually. In particular, when the entire image is similar between the comparison source image and the comparison destination image and there is a difference in a part of the area, for example, a case where the background image is the same and only the character portion is different is considered. . In such a case, local feature points cannot be taken so much in characters, and a large difference area may not be obtained, but the difference area is expanded by taking the logical sum of two types of feature points as described above. I can take it.

  The method of determining the difference area based on the two types of non-corresponding points may be determined by applying an image area separation method in OCR, for example. In the image area separation, a projection histogram of a black pixel of a binary image with respect to the x direction and the y direction is generated and a region of a character or a figure is estimated, but in this embodiment, non-positive correspondence in the x direction and the y direction in the comparison target image. A difference area is determined from the histogram of the points. For simplicity, it is determined as a circumscribed rectangle including a section in which the value of the histogram is a predetermined value or more in each of the x direction and the y direction. In the present embodiment, for example, as shown in FIGS. 4D and 4E, one or more area groups are determined as different areas from the histogram of non-positive corresponding points in the x direction and the y direction.

  In step S309, the image feature amount comparison unit 203 obtains the number of non-positive corresponding points in each difference area determined on the comparison target image. Then, an area where the non-positive corresponding point is less than the predetermined threshold is excluded from the difference area. This is because, for example, the influence of the feature points caused by irregularity due to paper stains of the document document is eliminated, and the difference area is limited to an area having a certain size such as characters. Here, the number of non-corresponding corresponding points is obtained in each different area, but the number of non-positive corresponding points (density) for each different area is calculated to exclude those having a density lower than a predetermined value. Good.

  In step S310, the image feature amount comparison unit 203 extracts (cuts out) the difference area specified in step S309. In this embodiment, since the comparison source image may be rotated / scaled, the comparison source image is first converted into an image aligned with the common area with the comparison destination image, and then the comparison source image Get the difference area at. Specifically, the geometric transformation of each pixel of the comparison source image is performed using the geometric transformation coefficients (transformation matrices M and T in FIG. 5 described later) obtained by the RANSAC process in step S306. Then, a different area is cut out from each of the converted comparison source image and comparison destination image. FIG. 4F shows a state in which the comparison source image is rotated so as to fit the common area with the comparison destination image by the process of step S310. FIG. 4G is a diagram illustrating a state in which a different area is cut out from each of the converted comparison source image and the comparison destination image.

  In step S311, the OCR processing unit 205 performs OCR processing on each difference area of the comparison source image and the comparison destination image obtained in step S310, and acquires character recognition information. In this embodiment, the recognition result of the character recognition information is “character” or “character string” information obtained as a result of the processing. The OCR process may use a generally known technique.

  In the present embodiment, when performing the OCR process on each difference area of the comparison source image and the comparison target image, a character binary image extraction process for the OCR process is performed for each difference area. In general OCR processing, binarization processing based on density is performed on an entire image including characters in order to separate the character image from other images. For example, if the character is black and the background is a light pattern image, the frequency distribution of pixel density is roughly divided into two peaks (the surrounding density and the surrounding density). Part with a high frequency distribution). Therefore, by setting a threshold value for the density value that separates these two peaks, it is possible to separate the character and the pixels of the other portions, so that the threshold value can be determined relatively easily. A binary image of a character is obtained by regarding a pixel having a density equal to or higher than the threshold as a character pixel.

  However, in the case of printed materials such as flyers, brochures, and invitations, the background may be a complex pattern or a natural image, and the character color may differ depending on the location. Even if similar binarization processing is performed in such a situation, it is difficult to obtain an appropriate threshold value, and it is difficult to extract a character image with high accuracy. On the other hand, in the OCR process in step S311 of the present embodiment, it is highly possible that the character and other pixels can be separated by performing the binarization process only on the partial images of the different areas. That is, even if it is a printed image, if it is limited to a portion where characters are described, it is often the case that a contrast that humans can visually recognize is added, and there is a high possibility that the threshold value can be determined well.

  In the present embodiment, as described above, the difference area between the comparison source image and the comparison target image is obtained. However, such a difference area is highly likely to be a character portion in an image of a printed matter such as a flyer, and is accurate. It is expected to be able to recognize characters well. FIG. 4H schematically shows how character recognition information is acquired from each difference region of the comparison source image and the comparison destination image by the process of step S311.

  In step S312, the OCR result comparison unit 206 compares “characters” or “character strings” obtained from corresponding differences between the comparison source image and the comparison target image. In step S313, the output unit 207 outputs information on the difference of the compared character recognition information. The present embodiment is not limited by the method of outputting the difference information, but here, the recognized character string difference is output. If the comparison source image and the comparison destination image in FIG. 4 are compared, information indicating that there is a difference between “6.8” and “12.31” is output as shown in FIG. The information is displayed, for example, by a display unit provided in the image processing apparatus. Further, for example, the comparison result of the OCR result comparison unit 206 is scored and subtracted from the score obtained from the voting based on the local feature amount, so that it is used for image search processing based on the similarity score, and other configurations are adopted. Also good.

  Next, details of the processing in step S306 described above will be described with reference to FIG. FIG. 5 is a flowchart for calculating the geometric relationship between the feature point positions of the comparison source image and the comparison destination image using the RANSAC process.

  First, symbols are defined. It is assumed that the local feature of the comparison target image is Vq, the local feature point associated with the local feature is Q, and the coordinate is Q (x ′, y ′). Further, a local feature existing in the comparison source image is Vs, a local feature point associated with the local feature is S, and a coordinate is S (x, y).

  In step S501, a variable VoteMax representing the final number of votes is initialized to zero. Next, in step S502, the shortest distance corresponding point list which is the processing result of S303 is read. The content is repeated, but is a corresponding point pair of Vq and Vs such that the calculated distance between the vector features is equal to or less than the threshold value Tv and the shortest distance.

Thereafter, for the kth corresponding point registered in the shortest distance corresponding point list, the local features of the corresponding point are described as Vq (k) and Vs (k), respectively, and the distance between the vector features is equal to or less than the threshold Tv. Let the number of features be Nq (k) and Ns (k), respectively. Furthermore, local feature points associated with Vq (k) and Vs (k) are subscripts such as Qk and Sk, coordinates are Qk (x′k, y′k), Sk (x _k , y _k ), and the like. Are described together. Further, the number of pairs of corresponding points registered in the shortest distance corresponding point list created in step S303 is m.

  In step S503, a variable Count indicating the number of iterations of the similarity calculation process is initialized to zero. In step S504, it is determined whether the iteration count number Count has not exceeded a predetermined maximum number of iterations Rn. Here, when it exceeds, it progresses to step S521, the final vote number VoteMax is output, and this process is complete | finished.

  If it is determined in step S504 that the iteration count number Count does not exceed the maximum number of iterations Rn, the process proceeds to step S505, and a variable Vote indicating the number of votes is initialized to zero. Next, in step S506, two sets of coordinates of corresponding point pairs are randomly extracted from the shortest distance corresponding point list.

Here, these coordinates are described as Q1 (x ′ ₁ , y ′ ₁ ), S1 (x ₁ , y ₁ ), Q2 (x ′ ₂ , y ′ ₂ ), and S2 (x ₂ , y ₂ ). Next, in step S507, the extracted Q1 (x ′ ₁ , y ′ ₁ ), S1 (x ₁ , y ₁ ), Q2 (x ′ ₂ , y ′ ₂ ), and S2 (x ₂ , y ₂ ) Assuming that the conversion shown in (1) is satisfied, f is obtained from the variable a in (Formula 1). However, in step S507 shown in FIG. 5, a matrix composed of variables a to d is denoted by M, and a matrix composed of variables e to f is denoted by T.

ここで、第１の実施形態では、簡略化のため、相似変換だけを考える。このとき、上記（数式１）は以下の（数式２）のように書き換えられる。

Here, in the first embodiment, only the similarity transformation is considered for simplification. At this time, the above (Formula 1) is rewritten as the following (Formula 2).

このとき、変数ａ、ｂ、ｅ、ｆはｘ’_１、ｙ’_１、ｘ_１、ｙ_１、ｘ’_２、ｙ’_２、ｘ_２、ｙ_２を使って式（３）から式（６）で表される。

At this time, the variables a, b, e, and f are changed from Equation (3) to Equation (6) using x ′ ₁ , y ′ ₁ , x ₁ , y ₁ , x ′ ₂ , y ′ ₂ , x ₂ , y _2. ).

次に、投票数集計手段の説明をステップＳ５０８からＳ５１７までの処理を用いて行う。

Next, the vote counting means will be described using the processing from step S508 to S517.

  In step S508, the corresponding point selection variable k is initialized to 3 in order to select a point other than the two sets of points selected from the shortest distance corresponding point list in step S505 described above. In step S509, it is determined whether or not the corresponding point selection variable k exceeds the number m of corresponding points registered in the shortest distance corresponding point list. Here, if it exceeds, the process moves to step S518, which will be described later. If it is determined in step S509 that the corresponding point selection variable k does not exceed the number m of corresponding points registered in the shortest distance corresponding point list, the process proceeds to step S510.

In this step S510, points other than the two sets of points S1 (x ₁ , y ₁ ) and S2 (x ₂ , y ₂ ) randomly extracted from the shortest distance corresponding point list in step S505 described above correspond to the shortest distance. Extract from the point list. In the first embodiment, the extracted point is described as Sk (x _k , y _k ).

Next, in step S511, coordinates Sk ′ (x ′ _k , y ′ _k ) to which Sk (x _k , y _k ) is transferred using the equation (2) are _obtained . Thereafter, in step S512, the geometric distance between the coordinates Sk ′ (x ′ _k , y ′ _k ) and the coordinates Qk (x ′ _k , y ′ _k ) is calculated as the Euclidean distance, and the Euclidean distance is calculated from the threshold Td. A comparison is made to determine whether it is below the threshold. If the Euclidean distance is less than or equal to the threshold value Td, the process proceeds to step S513, the vote number Vote is incremented, and the coordinates of S′k are stored as positive corresponding points in step S514. Then, the process proceeds to step S515.

  If the Euclidean distance is greater than the threshold value Td, NG_Vote, which is the number of feature points that did not satisfy the geometrical relationship in step S516, is incremented. Remember as. Then, the process proceeds to step S515.

  In step S515, the corresponding point selection variable k is incremented, and the process returns to step S509 to extract an unprocessed pair on the shortest distance corresponding point pair list, and the corresponding point selection variable k is registered in the shortest distance corresponding point list. The above-described processing is repeated until the number m of corresponding points is exceeded.

  Next, step S518, which is a process when the corresponding point selection variable k exceeds the number m of corresponding points registered in the shortest distance corresponding point list in step S509, will be described. In step S518, the value of the vote number Vote and the value of the final vote number VoteMax are compared. If the value of the vote number Vote is larger than the value of the final vote number VoteMax, the process proceeds to step S519.

  In step S519, after the value of the final vote number VoteMax is replaced with the value of the vote number Vote, the iteration count number Count is incremented in step S520, and the process returns to step S504 described above.

  If the value of the vote number Vote is equal to or smaller than the value of the final vote number VoteMax in step S518, the process proceeds to step S520, the repeat count number Count is incremented, and the process returns to step S504 described above.

  In the description of the score calculation method according to the first embodiment, only the similarity transformation has been considered. However, for other geometric transformations such as affine transformation, a transformation matrix corresponding to each is obtained in step S507. It is possible to respond. For example, in the case of affine transformation, first, in step S506, the number of coordinates of the set of corresponding points is set to 3. Next, in step S507, equation (1) is used instead of equation (2), and the three sets of corresponding points selected in step S506 (6 points in total) may be used to obtain f from variable a.

  As described above, according to the present embodiment, when comparing images that are generally similar to each other, it is possible to perform comparison with high accuracy by performing OCR processing on a region having a difference in the image region.

  In this embodiment, the case where the difference area between the comparison source image and the comparison destination image is “character” or “character string” information has been described. However, the difference area is other than “character” or “character string”. However, it has an effect. In principle, it is difficult to grasp the difference in line segment information existing between feature points in the rotation / scaling invariant feature amount-based similarity comparison such as SIFT in this embodiment. On the other hand, the OCR processing is specialized for characters, but can be said to be similar class determination processing that captures line segment information. Therefore, when OCR processing is performed under the same conditions using the same OCR processing module for the difference area, even if the area is not a character, the OCR processing result is not readable but has a uniform output. Is obtained. If there is a line segment difference in the partial image of the difference area, it will appear as a difference in recognition results. At this time, for example, the character string lattices of the character recognition result may be compared as they are, and may be treated as the difference in character information depending on the difference.

  Alternatively, when a feature quantity unique to character recognition that is different from the feature quantity obtained by the image feature quantity group extraction unit 202 is obtained as an OCR processing result, the similarity may be compared with the feature quantity. For example, distance information between vectors may be used as long as a multidimensional vector is obtained as a feature quantity. As described above, according to the present embodiment, even when the difference area is not necessarily a character area, it is possible to perform a comparison that captures a difference in character information in the difference area.

  In the present embodiment, the image erecting processing unit 204 performs a geometric transformation process such as rotation / enlargement / reduction on the comparison source image using a geometric transformation coefficient such as an affine transformation obtained by the image feature amount comparison unit 203. Alignment with the comparison target image was performed. For example, even if the conditions of the comparison source image and the comparison destination image are different, such as scanned image data as the comparison source image and rasterized document image data as the comparison destination image, alignment is performed on the comparison destination image. And can be erect. Thereby, the OCR processing performed by the OCR processing unit 205 at the subsequent stage can be performed with high accuracy.

  In the present embodiment, the OCR processing unit 205 performs binarization processing only on partial images of each different area. Even if it is a printed image, if it is limited to the part where the character is written, it is often the case that a contrast that humans can visually recognize is added, and the threshold is well determined, and the possibility of separating the character from other pixels is possible Becomes higher. Thereby, even if it is a character part in an image like printed matter, such as a leaflet, it becomes possible to recognize a character accurately.

  In the above description, SIFT or SURF obtained from a luminance image is given as an example of the feature amount as a local feature. However, it is also possible to use a local feature in a color difference signal or an RGB color channel. In the description of FIG. 4, one example of the difference area between the comparison source image and the comparison destination image is shown, but any number of difference areas may be provided. The processing may be performed in association with each other so that it can be understood that the comparison source and the comparison destination are in the same region. Specifically, after extracting a group of regions having a high density of non-positive corresponding points in step S309, the processing in steps S311 to S312 may be performed for each region. Thereby, when there are a plurality of different areas between the comparison source image and the comparison destination image, specifically, even when there are a plurality of different background / character font character areas on a printed matter such as a flyer, Character parts can be compared.

[Other Embodiments]
In addition, the present invention supplies software (program) for realizing the functions of the above-described embodiments to a system or apparatus via a network or various storage media, and the computer of the system or apparatus (or CPU, MPU, etc.) programs Is read and executed. Further, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. The present invention is not limited to the above embodiments, and various modifications (including organic combinations of the embodiments) are possible based on the spirit of the present invention, and these are excluded from the scope of the present invention. is not. That is, the present invention includes all the combinations of the above-described embodiments and modifications thereof.

DESCRIPTION OF SYMBOLS 201 Image input part 202 Image feature-value group extraction part 203 Image feature-value comparison part 204 Image erecting process part 205 OCR process part 206 OCR result comparison part 207 Output part 208 Storage part

Claims (6)

An input means for inputting a comparison source image and a comparison destination image;
Estimating means for estimating a difference area between the input comparison source image and the comparison destination image;
OCR processing means for performing OCR processing on the estimated difference area;
An image processing apparatus comprising: The estimation means includes
Extraction means for extracting local feature points from the input comparison source image and comparison destination image;
Calculating means for calculating local features for the extracted local feature points;
First determination means for determining the local feature point in a correspondence relationship between the comparison source image and the comparison target image based on the calculated local feature;
First estimation means for estimating a common area between the comparison source image and the comparison destination image based on the determined local feature points in the correspondence relationship;
Second determining means for determining local feature points having no correspondence relationship between the comparison source image and the comparison destination image in the estimated common region;
Second estimation means for estimating a difference area between the comparison source image and the comparison destination image based on the determined local feature points having no corresponding relationship;
The image processing apparatus according to claim 1, further comprising:   The image processing apparatus further includes an erecting processing unit that erects the comparison source image and the comparison destination image based on a geometric relationship between the local feature points in the correspondence relationship determined by the first determination unit. The image processing apparatus according to claim 2.   4. The image processing apparatus according to claim 1, wherein the OCR processing unit performs OCR processing by performing binarization processing only on the estimated difference area. 5. Enter the comparison source image and the comparison destination image,
Estimating a difference area between the input comparison source image and the comparison destination image;
Performing OCR processing on the estimated difference area;
An image processing method.   A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 5.

Images