JP5109548B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and a program.

An image processing apparatus that compares a scanned image of a paper document before correction with a scanned image of the paper document after correction (for example, writing or character deletion using a white pen) and extracts a corrected portion. is there. Patent Document 1 below discloses an image processing apparatus that extracts a corrected portion by generating a difference between binary images of both scan images.
JP 2004-341914 A

  However, since it is difficult to scan a paper document before correction and a paper document after correction under completely the same conditions, there are portions other than the correction portion between the two scan images. There is also a difference in hue. For example, the two scanned images may have different colors due to the passage of time or the difference in scanners. Therefore, in some cases, when both scanned images are binarized, a portion that is not originally an ON pixel (for example, a blank portion) becomes an ON pixel only in one image, and as a result, a portion other than the correction portion is erroneously extracted. There was a problem of being.

  An object of the present invention is to provide an image processing apparatus and a program for accurately extracting a corrected portion.

  The invention of claim 1 includes a first scan image read from a paper document before correction, a second scan image read from the paper document after correction, the correction being made to the paper document before correction, Corrected by the correction means, the correction means for correcting the first scan image based on the first color correction parameter, the correction means for correcting the second scan image based on the second color correction parameter, and the correction means. Comparison means for comparing the hue in a given comparison area of the first scan image with the hue in the comparison area of the second scan image corrected by the correction means, and the comparison result by the comparison means is predetermined. A first scan image corrected based on the first color correction parameter, a second scan image corrected based on the second color correction parameter, when the condition is satisfied; An image processing apparatus characterized by comprising extracting means for extracting the correction area based.

  According to a second aspect of the present invention, in the first aspect of the invention, the comparison unit is corrected by the correction unit and the appearance frequency distribution of each pixel value in the comparison region of the first scan image corrected by the correction unit. And calculating the correlation coefficient between the appearance frequency distributions of the pixel values in the comparison area of the second scan image, and the extracting means calculates the correction portion when the value of the correlation coefficient is equal to or greater than a predetermined value. Is extracted. Here, the pixel value is, for example, a lightness value, a color value such as an R value, a G value, or a B value.

  According to a third aspect of the invention, in the first aspect of the invention, the image in the comparison area of the first scan image corrected by the correction means and the image in the comparison area of the second scan image corrected by the correction means. The index value is calculated based on the difference information based on and the extraction means extracts the correction portion according to the value of the index value.

  According to a fourth aspect of the present invention, in the third aspect of the present invention, the index value is a numerical value indicating the number of difference extraction pixels or the variance of the difference extraction pixels, and the extraction means has the index value equal to or less than a predetermined value. In some cases, the correction portion is extracted.

  According to a fifth aspect of the invention, in the invention according to any one of the first to fourth aspects, the correction unit changes the at least one of the first color correction parameter and the second color correction parameter while changing the first scan image. And the second scan image, and the comparison means corrects the first scan image corrected by the correction means every time at least one of the one color correction parameter and the second color correction parameter is changed. And the second scan image are compared with each other.

  The invention of claim 6 further comprises parameter set storage means for storing at least one parameter set including the first color correction parameter and the second color correction parameter in the invention of claim 5, wherein the correction means The first scan image and the second color correction parameter are changed based on the first color correction parameter and the second color correction parameter included in the selected parameter set while changing the parameter set selected from the parameter set storage unit. The two-scan image is corrected.

  According to a seventh aspect of the present invention, in the fifth or sixth aspect of the present invention, the correction means, when the comparison result by the comparison means does not satisfy the predetermined condition, the first color correction parameter and the second color correction. At least one of the parameters is changed.

  The invention of claim 8 is the invention of claim 6, wherein one parameter set is selected based on the result of the comparison from parameter sets when the result of the comparison by the comparison means satisfies the predetermined condition, Based on the first scan image corrected based on the first color correction parameter included in the parameter set and the second scan image corrected based on the second color correction parameter included in the parameter set. It is characterized by extracting a corrected portion.

  The invention of claim 9 further includes a reduction means for reducing the first scan image and the second scan image in any one of the inventions of claims 1 to 8, wherein the correction means is the reduction means. The first scan image reduced by the correction is corrected based on the first color correction parameter, and the second scan image reduced by the reduction means is corrected based on the second color correction parameter.

  According to a tenth aspect of the present invention, in the invention according to any one of the first to ninth aspects, the correction unit corrects an image in the comparison area in the first scan image, and the comparison area in the second scan image. The image is corrected.

  The invention according to claim 11 is a first scan image read from a paper document before correction, a second scan image read from a paper document after correction, the correction being made to the paper document before correction, And image acquisition means for acquiring the first scan image based on the first binarization parameter, and binarization means for binarizing the second scan image based on the second binarization parameter And an image in a given comparison area of the first scan image binarized by the binarization means, an image in the comparison area of the second scan image binarized by the binarization means, The first scan image binarized based on the first binarization parameter when the comparison result by the comparison unit satisfies a predetermined condition, and the second binarization Binarized based on parameters And 2 scanned image, extracting means for extracting the modified portion based on an image processing apparatus which comprises a.

  According to a twelfth aspect of the present invention, in the invention according to the eleventh aspect, the comparison unit includes the number of significant pixels in the comparison region of the first scan image binarized by the binarization unit, and the binarization. A difference between the number of significant pixels in the comparison area of the second scan image binarized by the means and the extraction means calculates the correction portion when the difference value is equal to or less than a predetermined value. It is characterized by extracting.

  According to a thirteenth aspect of the present invention, in the invention of the eleventh aspect, the comparison means includes an image in the comparison area of the first scan image binarized by the binarization means and a binary value by the binarization means. An index value is calculated based on difference information between the converted second scan image and the image in the comparison area, and the extraction unit extracts the correction portion according to the value of the index value. To do.

  According to a fourteenth aspect of the present invention, in the thirteenth aspect, the index value is a numerical value indicating the number of difference extraction pixels or a variance of the difference extraction pixels, and the extraction means includes the index value equal to or less than a predetermined value. In this case, the correction part is extracted.

  The invention of claim 15 is the invention according to any one of claims 11 to 14, wherein the binarization means changes at least one of the first binarization parameter and the second binarization parameter, The first scan image and the second scan image are binarized, and the comparison unit is configured to output the binary data every time at least one of the first binarization parameter and the second binarization parameter is changed. The first scan image binarized by the binarizing unit and the second scan image binarized by the binarizing unit are compared.

  According to a sixteenth aspect of the present invention, in the fifteenth aspect of the invention, the binarization unit may be configured such that when the comparison result by the comparison unit does not satisfy the predetermined condition, the first binarization parameter and the second binarization parameter. It is characterized in that at least one of the valuation parameters is changed.

  The invention of claim 17 further includes a reduction means for reducing the first scan image and the second scan image according to any one of claims 11 to 16, wherein the binarization means comprises the binarization means, The first scan image reduced by the reduction means is binarized based on the first binarization parameter, and the second scan image reduced by the reduction means is binarized based on the second binarization parameter. It is characterized by doing.

  According to an eighteenth aspect of the present invention, in the invention according to any one of the eleventh to seventeenth aspects, the binarizing means binarizes the image of the comparison area in the first scan image, and includes the second scan image. The image of the comparison area is binarized.

  According to a nineteenth aspect of the present invention, in any one of the first to eighteenth aspects, the given comparison region is a region that occupies a part of each of the first scan image and the second scan image. Features.

  The invention of claim 20 is characterized in that, in the invention of claim 19, the given comparison area is a predetermined area.

  The invention of claim 21 is the invention of claim 19, wherein the given comparison area is an area designated by a user.

  According to a twenty-second aspect of the present invention, in the invention of the nineteenth aspect, an analysis unit that analyzes a layout of the first scan image and the second scan image, and the given comparison region based on an analysis result by the analysis unit And determining means for determining.

  The invention of claim 23 is the invention of claim 19, wherein a difference between the binary image of the first scan image and the binary image of the second scan image is generated, and the given comparison is performed based on the difference. And determining means for determining an area.

  According to a twenty-fourth aspect of the present invention, a first scan image read from a paper document before correction, a second scan image read from a paper document after correction, the correction of the paper document before correction, The image acquisition means for acquiring the correction, the correction of the first scan image based on the first color correction parameter, the correction means of correcting the second scan image based on the second color correction parameter, and the correction by the correction means Comparison means for comparing the hue in a given comparison area of the first scan image with the hue in the comparison area of the second scan image corrected by the correction means, and the comparison result by the comparison means satisfies a predetermined condition. If satisfied, a first scan image corrected based on the first color correction parameter; a second scan image corrected based on the second color correction parameter; Extracting means for extracting the correction area based, it is a program causing a computer to function as a.

  The invention of claim 25 includes a first scan image read from a paper document before correction, a second scan image read from the paper document after correction, the correction being made to the paper document before correction, Image acquisition means for acquiring, binarization means for binarizing the first scan image based on a first binarization parameter, and binarizing the second scan image based on a second binarization parameter, An image in a given comparison area of the first scan image binarized by the binarization means, and an image in a given comparison area of the second scan image binarized by the binarization means; Comparing means for comparing the first scan image binarized based on the first binarization parameter and the second binarization parameter when the comparison result by the comparison unit satisfies a predetermined condition Second binarized based on A catcher down image, a program for causing a computer to function correction part extraction means for extracting, as based on.

  According to invention of Claim 1,11,24,25, a correction part can be extracted correctly.

  According to the second and fourth aspects of the present invention, it is possible to extract the corrected portion after making the hue of the first scan image and the hue of the second scan image similar to a certain degree.

  According to the invention of claim 3, the similarity between the hue of the first scan image and the hue of the second scan image can be indexed and evaluated.

  According to the fifth aspect of the present invention, it is possible to search for the first color correction parameter and the second color correction parameter that can make the hue of the first scan image and the hue of the second scan image similar.

  According to the sixth aspect of the present invention, it is possible to easily search for the first color correction parameter and the second color correction parameter that can make the hue of the first scan image and the hue of the second scan image similar to each other.

  According to the seventh aspect of the present invention, the first color correction parameter and the second color correction parameter are searched while determining whether or not the hue of the first scan image is similar to the hue of the second scan image. be able to.

  According to the eighth aspect of the present invention, it is possible to determine the optimal first color correction parameter and second color correction parameter that can make the hue of the first scan image and the hue of the second scan image similar to each other.

  According to the ninth and tenth aspects of the present invention, processing necessary for determining the first color correction parameter and the second color correction parameter can be reduced.

  According to the inventions of claims 12 and 14, the corrected portion can be extracted after making the binary image of the first scan image similar to the binary image of the second scan image.

  According to the invention of claim 13, the similarity between the binary image of the first scan image and the binary image of the second scan image can be indexed and evaluated.

  According to the invention of claim 15, the first binarization parameter and the second binarization parameter that make the binary image of the first scan image similar to the binary image of the second scan image are searched. Can do.

  According to the invention of claim 16, the first binarization parameter and the second second image are determined while determining whether or not the binary image of the first scan image is similar to the binary image of the second scan image. The value parameter can be searched.

  According to the seventeenth and eighteenth aspects of the present invention, the first binarization parameter and the second binarization parameter for determining the similarity between the binary image of the first scan image and the binary image of the second scan image are determined. Necessary processing can be reduced.

  According to the nineteenth aspect of the present invention, it is possible to accurately extract the corrected portion with relatively little processing.

  According to the inventions of claims 20, 21, 22, and 23, a region that does not include a correction portion is likely to be a comparison region.

  Embodiments of the present invention (hereinafter referred to as Embodiment 1) will be described with reference to the drawings. FIG. 1 is a diagram showing a correction extraction system 100 configured to include an image processing apparatus 2 according to Embodiment 1 of the present invention. The correction extraction system 100 includes an image processing device 2 and an image reading device 4, and the image processing device 2 and the image reading device 4 are connected to be communicable. In the correction extraction system 100, the user causes the image reading device 4 to read a paper document before correction, and registers a read image (hereinafter referred to as an image before correction) in the image processing device 2. In addition, the image reading device 4 is caused to read the corrected paper document in which correction such as addition or deletion with a white pen is performed on the uncorrected paper document, and the read image (hereinafter, corrected image) is read by the image processing device 2. sign up. Then, the user performs a predetermined correction part extraction instruction operation, and causes the image processing apparatus 2 to compare the uncorrected image with the corrected image and extract the corrected part. Here, the paper document includes a plurality of paper media.

  Next, each device will be specifically described. The image reading device 4 is realized as various scanners including an automatic paper feeding mechanism such as ADF (AUTO DOCUMENT FEEDER). The image reading device 4 optically reads a paper document, transmits the read image to the image processing device 2, and stores it in the image processing device 2. Here, the image reading device 4 transmits the pre-correction image and the post-correction image and stores them in the image processing device 2.

  The image processing apparatus 2 is realized as a personal computer or a server, for example. The image processing device 2 stores the pre-correction image and the post-correction image received from the image reading device 4. Further, the image processing apparatus 2 accepts a user instruction, compares the uncorrected image with the corrected image, and extracts a corrected portion. As shown in FIG. 2, the image processing apparatus 2 is configured to include a control unit 22, a storage unit 24, an image input unit 26, and an operation unit 28. The image processing apparatus 2 includes a display unit (not shown), a network interface, and the like in addition to the above-described units. Hereinafter, each part will be described.

  The control unit 22 is a CPU or the like and operates according to a program stored in the storage unit 24.

  The storage unit 24 is a RAM, a ROM, a hard disk or the like, and stores various information. The storage unit 24 stores a program executed by the control unit 22. The storage unit 24 stores the pre-correction image and the post-correction image input from the image input unit 26. The storage unit 24 also serves as a work memory that holds various information necessary for executing the processing of the control unit 22.

  The image input unit 26 is an interface connected to the image reading device 4, receives an image transmitted from the image reading device 4, and outputs the image to the storage unit 24. The operation unit 28 is, for example, a keyboard or a mouse, and outputs information on the operation contents of the user to the control unit 22. The above is the content of each part constituting the image processing apparatus 2.

FIG. 3 is a block diagram of functions realized by the image processing apparatus 2 when the control unit 22 operates according to the program. As shown in the figure, the image processing apparatus 2 includes a comparison area determination unit 32, a correction unit 34, a comparison unit 36, a parameter set storage unit 38, an overall position / distortion correction unit 40, a local position / distortion. Correction unit 42
And a corrected portion extraction unit 44. Further, as shown in FIG. 4, the local position / distortion correction unit 42 includes a difference extraction unit 42a, an integration processing unit 42b, a second divided region calculation unit 42c, a first divided region calculation unit 42d, A two-image dividing unit 42e, a first image dividing unit 42f, and a divided image position / distortion correcting unit 42g are included. Hereinafter, each part will be described.

(Parameter set storage unit 38)
The parameter set storage unit 38 is realized centering on the storage unit 24. The parameter set storage unit 38 stores at least one parameter set of the first color correction parameter P1 and the second color correction parameter P2. FIG. 5 is a diagram illustrating an example of the contents stored in the parameter set storage unit 38. As shown in the figure, the parameter set storage unit 38 stores a plurality of parameter sets, and each parameter set is assigned a combination number. Here, the first color correction parameter P1 and the second color correction parameter P2 are tone curves. The parameter set stored in the parameter set storage unit 38 may be determined in advance or may be registered by the user. Further, the first color correction parameter P1 and the second color correction parameter P2 may be look-up tables.

(Comparison area determination unit 32)
The comparison area determination unit 32 is realized centering on the control unit 22 and determines an area (comparison area) for comparing the uncorrected image and the corrected image corrected by the correction unit 34. In the present embodiment, the comparison area determination unit 32 sets an area occupying a part of each of the pre-correction image and the post-correction image as a comparison area. Specifically, the comparison area determination unit 32 analyzes the layout of the pre-correction image and the post-correction image. As a result of the analysis, an image area such as a character / line / table or a page containing a large number of characters / lines / tables is determined as a comparison area.

(Correction unit 34)
The correction unit 34 is realized centering on the control unit 22 and corrects, for example, the brightness of the image using the color correction parameter. In the present embodiment, the correction unit 34 corrects the lightness in the comparison area of the pre-correction image with the first color correction parameter P1 while changing at least one of the first color correction parameter P1 and the second color correction parameter P2. Then, one parameter set is specified by correcting the lightness in the comparison area of the corrected image with the second color correction parameter P2. Here, the correction unit 34 determines the combination number until the correlation coefficient R between the lightness pixel number distributions (lightness histograms) in the comparison region calculated by the comparison unit 36 is equal to or greater than a predetermined threshold TH (for example, 0.95). The correlation coefficient R is predetermined by correcting the lightness of the image before correction and the lightness of the image after correction with each of the first color correction parameter P1 and the second color correction parameter P2 of the parameter set in order from the smallest parameter set. A parameter set that is equal to or greater than the threshold value TH is specified.

  The correction unit 34 corrects the pre-correction image and the post-correction image using all the parameter sets, and then the parameter having the maximum correlation coefficient R among the parameter sets in which the correlation coefficient R is equal to or greater than the predetermined threshold value TH. A set may be specified.

(Comparator 36)
The comparison unit 36 is realized centering on the control unit 22, and each time at least one of the first color correction parameter P1 and the second color correction parameter P2 is changed, the brightness of the image before correction corrected by the correction unit 34 is calculated. The brightness of the corrected image is compared. Specifically, the comparison unit 36, every time the correction unit 34 corrects the pre-correction image and the post-correction image using each of the parameter sets, the lightness pixel number distribution (in the comparison region of the pre-correction image) ( A correlation coefficient R between the first brightness histogram) and the brightness pixel number distribution (second brightness histogram) in the comparison region of the corrected image is calculated.

  Here, when the number of pixels of lightness i in the first lightness histogram is represented as “Xi” and the number of pixels of lightness i in the second lightness histogram is represented as “Yi”, the correlation coefficient R (−1 ≦ R ≦ 1). ) Is represented by the following mathematical formula. The comparison unit 36 may create a pixel number distribution of each color plane of Red, Green, and Blue, and calculate a correlation coefficient of each color plane.

(Overall position / distortion correction unit 40)
The overall position / distortion correction unit 40 is realized with the control unit 22 as the center, and using a well-known method (for example, the method described in Japanese Patent Laid-Open No. 2006-235785), the overall position of the two images to be processed is determined. Alignment is performed to correct misalignment and distortion between both images. In the present embodiment, the pre-correction image corrected with the first color correction parameter P1 of the parameter set specified by the correction unit 34 and the post-correction image corrected with the second color correction parameter P2 of the parameter set are targeted. As shown in FIG. Here, the positional deviation and distortion of the pre-correction image are corrected according to the post-correction image using the post-correction image as a reference. Of course, the positional deviation and distortion of the corrected image may be corrected according to the pre-correction image on the basis of the pre-correction image.

  Specifically, the overall position / distortion correction unit 40 sets the corrected image (correctly, the corrected image corrected by the second color correction parameter P2 of the parameter set specified by the correction unit 34) with a predetermined threshold. In accordance with the binarized binary image, the pre-correction image (more precisely, the pre-correction image corrected with the first color correction parameter P1 of the parameter set specified by the correction unit 34) is binarized with a predetermined threshold value. Correct the overall positional deviation and distortion of the binary image. In this way, the overall position / distortion correction unit 40 aligns the positions of the pre-correction image and the post-correction image as a whole.

(Local position / distortion correction unit 42)
The local position / distortion correction unit 42 is realized with the control unit 22 as a center, and corrects a local positional deviation between two target images. In the present embodiment, the local position / distortion correction unit 42 is a pre-correction image (more precisely, the first parameter set specified by the correction unit 34) that has been subjected to overall alignment by the overall position / distortion correction unit 40. The target image is a corrected image (corrected image corrected by the one-color correction parameter P1) and an corrected image (correctly corrected image corrected by the second color correction parameter P2 of the parameter set specified by the correcting unit 34). For each page, the local positional deviation between both images is corrected. Normally, when an image is read by a scanner or the like, local distortion occurs in the scanned image due to lens distortion, motor rotation unevenness, vibration, and the like. This is because even if it is performed, there may be a local positional deviation between the pre-correction image and the post-correction image. Hereinafter, each part of the local position / distortion correction unit 42 will be described.

(Difference extraction unit 42a)
The difference extraction unit 42a extracts difference information by comparing a binary image obtained by binarizing the corrected image with a predetermined threshold and a binary image obtained by binarizing the uncorrected image with a predetermined threshold. Specifically, for example, the difference information is extracted by subtracting an image obtained by expanding ON pixels of the binary image of the pre-correction image from the binary image of the post-correction image. Here, the ON pixel is a black pixel when a white background is used in the binary image. In the above, the difference information is extracted by subtracting the image obtained by expanding the ON pixels of the binary image of the pre-correction image from the binary image of the post-correction image, but from the binary image of the pre-correction image, The difference information may be extracted by subtracting the image obtained by expanding the ON pixels of the binary image of the corrected image, or the difference information may be extracted by both methods.

(Integration processing unit 42b)
The integration processing unit 42b integrates the difference extraction pixels that can be regarded as the same region based on the difference extraction pixel that is the difference information extracted by the difference extraction unit 42a to form an integration region. Here, the integration means that the difference extraction pixels that can be regarded as the same region are set as one pixel group.

  Specifically, the integration processing unit 42b obtains a distance between the extracted pixels, for example, a Euclidean distance or a city block distance, and if the distance is equal to or less than a predetermined threshold, the pixels are regarded as the same region and integrated. This is an area.

For example, the integration processing unit 42b may determine the integration region as follows.
(1) The extracted pixels are divided into connected pixel groups by using, for example, a labeling process, a circumscribed rectangle is obtained for each connected pixel group, and the circumscribed rectangle is expanded (expanded) by a predetermined size. Then, when the extended circumscribed rectangles overlap, the connected pixel groups are regarded as the same region and are set as an integrated region.
(2) A labeling process is performed on the binary image of the pre-correction image. Then, a label (concatenated pixel group) including (belonging to) the extracted pixel is determined, and the extracted pixel belonging to the same label is regarded as the same area and is defined as an integrated area. In addition, it is good also considering the label (concatenated pixel group) containing an extraction pixel as an integrated area | region.

  When the integrated region is determined by the integration processing unit 42b, the second divided region calculating unit 42c, the first divided region calculating unit 42d, the second image dividing unit 42e, and the first image dividing unit 42f obtain the second image before correction. The value image and the binary image of the corrected image are divided into regions. Hereinafter, each of these parts will be described.

(Second divided region calculation unit 42c)
The second divided region calculation unit 42c sets the predetermined region including the integrated region integrated by the integration processing unit 42b as the second divided region for the corrected image. Specifically, the second divided region calculation unit 42c expands (expands) the circumscribed rectangle of the integrated region (pixel group) by a predetermined size to form the second divided region. However, the extension size may be zero.

  Note that the second divided region calculation unit 42c may create a convex hull for the pixel group of the integrated region, and expand (expand) the convex hull by a predetermined size to form the second divided region. Even in this case, the extension size may be zero. Here, the convex hull means the smallest convex polygon including a point set, and can be said to be the shortest cycle surrounding a plurality of points.

  The first divided region calculation unit 42d sets a region expanded by a predetermined size from the second divided region calculated by the second divided region calculation unit 42c as the first divided region for the pre-correction image.

  The second image dividing unit 42e cuts out (divides) an image corresponding to the second divided region from the binary image of the corrected image based on the second divided region calculated by the second divided region calculating unit 42c. ) The second image.

  The first image dividing unit 42f cuts out an image corresponding to the first divided region from the binary image of the pre-correction image based on the first divided region calculated by the first divided region calculating unit 42d, To do. Here, since the first divided area is an expansion of the second divided area, the size of the first image is larger than the size of the second image.

(Divided image position / distortion correction unit 42g)
The divided image position / distortion correction unit 42g locally aligns the binary image of the pre-correction image and the binary image of the post-correction image. Specifically, the divided image position / distortion correction unit 42g performs pattern matching in such a manner that the second image is superimposed on the first image while changing the position of the second image in the first image, and a position with a high degree of matching. Position / distortion correction is performed using as a correction position.

  For example, the divided image position / distortion correction unit 42g performs pattern matching at all positions while moving the second image in the vertical and horizontal directions in units of one pixel in the entire first image, and has the highest matching degree (matching degree). The position is the position / distortion correction position. Here, the degree of matching is represented by the number of pixels in which the ON pixel positions of the first image and the second image match.

  Specifically, as shown in FIG. 6, when the first image (A) has a rectangular ring-shaped image and the second image (B) has an image in which the ring-shaped image is hatched, the first image The second image (B) is moved by one pixel unit from the upper left to the right in the figure with respect to the image (A), and then moved downward by one pixel and moved from the left to the right. Repeat until the bottom.

  Here, the matching degrees of the four positions (C) to (F) are shown. The matching degree at position (C) = 12, the matching degree at position (D) = 12, the matching degree at position (E) = 52, and the matching degree at position (F) = 31. Therefore, the position (E) is the position / distortion correction position.

  Thus, the divided image position / distortion correction unit 42g obtains the positional relationship between the first image and the second image having the highest matching degree. Then, the position of the image in the region corresponding to the first image in the pre-correction image is corrected so as to maintain the positional relationship. In this way, the local position / distortion correction unit 42 locally matches the positions of the pre-correction image and the post-correction image.

(Correction part extraction part 44)
The correction portion extraction unit 44 is realized with the control unit 22 as a center, and is an image before correction that has been locally and locally aligned in units of integrated regions (more precisely, the parameter set specified by the correction unit 34). Before-correction corrected with the first color correction parameter P1) and after-correction image (more precisely, the corrected image corrected with the second color correction parameter P2 of the parameter set specified by the correction unit 34). Based on this, the region of the corrected portion is extracted. Specifically, the corrected portion extraction unit 44 generates difference information by subtracting the binary image of the image before correction from the binary image of the image after correction. Then, the corrected part is extracted based on the difference information. In the above description, the corrected portion extraction unit 44 generates difference information by subtracting the binary image of the image before correction from the binary image of the image after correction, but the correction is extracted from the binary image of the image before correction. Difference information may be generated by subtracting a binary image of the subsequent image, or difference information may be generated by both methods. Further, the corrected portion extraction unit 44 may generate difference information by subtracting the pre-correction image itself from the corrected image itself, and extract the corrected portion based on the difference information.

  Hereinafter, referring to the flowchart of FIG. 7, an example of processing executed by the image processing apparatus 2 when the user designates a pre-correction image and a post-correction image stored in the storage unit 24 and issues a correction part extraction instruction. While explaining.

When receiving the correction extraction instruction operation, the image processing apparatus 2 reads the pre-correction image and the post-correction image from the storage unit 24, and executes a process of determining a comparison area (comparison area determination process) (S101). Specifically, the image processing apparatus 2 analyzes the layout of the pre-correction image and the post-correction image, and as a result of the analysis, the image area such as a character / line / table or a page containing a large number of characters / lines / tables. The comparison area. The image processing apparatus 2 generates difference information between a binary image obtained by binarizing the pre-correction image with a predetermined threshold and a binary image obtained by binarizing the post-correction image with the predetermined threshold, and the number of difference extraction pixels The image of the page with the least number may be used as the comparison area.

  Then, the image processing apparatus 2 determines the parameter set of the first color correction parameter P1 and the second color correction parameter P2 so that the brightness of the image before correction and the brightness of the image after correction are approximately the same. A specific process is executed (S102). Here, the parameter set specifying process will be described with reference to FIG.

  In the parameter set specifying process, the image processing apparatus 2 executes the steps of S201 to S203 in order from the parameter set having the smallest combination number until the value of the correlation coefficient R becomes equal to or greater than the predetermined threshold value TH. One parameter set in which the value of R is equal to or greater than a predetermined threshold value TH is specified.

  That is, the image processing apparatus 2 calculates the lightness in the comparison area of the image before correction and the lightness in the comparison area of the image after correction for each of the first color correction parameter P1 and the second color correction parameter P2 of the parameter set of interest. A correction process (correction process) is performed (S201).

  Here, in step S201, the image processing apparatus 2 may correct the R value, the G value, and the B value instead of correcting the brightness of the pre-correction image and the post-correction image. In this case, the first color correction parameter P1 and the second color correction parameter P2 include an R value tone curve, a B value tone curve, and a G value tone curve, respectively.

  Then, the image processing apparatus 2 performs a process (comparison process) for comparing the lightness in the comparison area of the image after correction with the lightness in the comparison area of the image before correction (S202). Here, the image processing apparatus 2 calculates the pixel number distribution by brightness (first brightness histogram) in the comparison area of the image after correction and the pixel number distribution by brightness (second brightness histogram) in the comparison area of the image before correction. A correlation coefficient R is calculated.

Then, the image processing apparatus 2 determines whether or not the correlation coefficient R is greater than or equal to a predetermined threshold value TH (S203). If the correlation function R is equal to or greater than the predetermined threshold value TH (Y in S203), the parameter determination process is terminated. On the other hand, if the correlation function R is smaller than the predetermined threshold value (N in S203), the image processing apparatus 2 The processing of S201 to S203 is executed for the parameter set of the next combination number. Thus, the image processing apparatus 2 specifies one parameter set in which the correlation coefficient R is equal to or greater than the predetermined threshold value TH. Note that the image processing apparatus 2 executes the steps S201 to S203 for all parameter sets, and then the parameter having the maximum correlation coefficient R among the parameter sets in which the correlation coefficient R is equal to or greater than the predetermined threshold TH. A set may be specified.

  Here, in the comparison process (S202), the image processing apparatus 2 does not calculate the correlation coefficient for the pixel number distribution according to lightness in the comparison area between the pre-correction image and the post-correction image, but in the comparison area of both images. , Green, and Blue, the correlation coefficient may be calculated for the pixel number distribution of each color plane. In this case, when at least one of the calculated three correlation coefficients is greater than or equal to the predetermined threshold TH in step S203 (Y in S203), the image processing apparatus 2 ends the parameter set specifying process, and these 3 When any of the two correlation coefficients is smaller than the predetermined threshold value TH (N in S203), the processing of S201 to S203 may be executed for the parameter set of the next combination number.

  Further, the image processing apparatus 2 may calculate the number L of labels to be described later in the comparison process (S202). That is, after the step of S201, the image processing apparatus 2 subtracts a binary image obtained by binarizing the pre-correction image with the predetermined threshold from the binary image obtained by binarizing the post-correction image with the predetermined threshold. And an integrated region of difference extraction pixels in the comparison region is determined. Here, the image processing apparatus 2 may generate the difference information by subtracting the binary image in the comparison area of the image before correction from the binary image in the comparison area of the image after correction. Then, the image processing apparatus 2 obtains the number of integrated regions, that is, the label number L. By the way, if the comparison area includes a correction portion, if the difference between the lightness in the comparison area of the pre-correction image and the lightness in the comparison area of the post-correction image is small, only the difference extraction pixels in the correction portion are easily extracted. The number of integrated areas is reduced. Even when the comparison area does not include a correction portion, if the difference between the lightness in the comparison area of the pre-correction image and the lightness in the comparison area of the post-correction image is small, the number of difference extraction pixels to be extracted becomes small. The number of integrated areas is reduced. In view of this, when the value of the number L of labels is larger than the predetermined threshold value in step S203 (N in S203), the image processing apparatus 2 executes the processes in S201 to S203 for the parameter set related to the next combination number, On the other hand, when the value of the label number L is equal to or smaller than the predetermined threshold (Y in S203), the parameter set specifying process may be terminated. Note that L is the number of difference extraction pixels of the difference image obtained by subtracting the binary image obtained by binarizing the pre-correction image with the predetermined threshold from the binary image obtained by binarizing the corrected image with the predetermined threshold. It may be.

  Further, the image processing apparatus 2 may calculate two variance values σX and σY, which will be described later, in the comparison process (S202). That is, after the step of S201, the image processing apparatus 2 subtracts a binary image obtained by binarizing the pre-correction image with the predetermined threshold from the binary image obtained by binarizing the post-correction image with the predetermined threshold. Is generated. Here, the image processing apparatus 2 may generate the difference information by subtracting the binary image in the comparison area of the image before correction from the binary image in the comparison area of the image after correction. Then, the image processing apparatus 2 performs a projection operation for each difference extraction pixel in the comparison area, and generates a projection histogram of the difference extraction pixel for each of the horizontal direction (X direction) and the vertical direction (Y direction) of the comparison area. To do. Then, the image processing apparatus 2 calculates a variance value σX in the X direction and a variance value σY in the Y direction of the difference extraction pixel from each projection histogram. By the way, if the comparison area includes a correction portion, if the difference between the lightness in the comparison area of the pre-correction image and the lightness in the comparison area of the post-correction image is small, only the difference extraction pixels in the correction portion are easily extracted. The difference extraction pixels tend to exist together, and as a result, the values of σX and σY become smaller. Even when the comparison area does not include a correction portion, if the difference between the lightness in the comparison area of the pre-correction image and the lightness in the comparison area of the post-correction image is small, the number of difference extraction pixels to be extracted becomes small. , ΣX and σY become smaller. In view of this, if at least one of the values of σX and σY is larger than the predetermined threshold value in step S203 (N in S203), the image processing apparatus 2 performs the processing in S201 to S203 for the parameter set related to the next combination number. On the other hand, if both the values of σX and σY are equal to or smaller than the predetermined threshold (Y in S203), the parameter set specifying process may be terminated.

  Then, the image processing apparatus 2 corrects the entire image before correction and the entire image after correction using the parameter set specified in step S102 (S103). Then, the image processing apparatus 2 corrects the overall positional deviation and distortion for each page for the pre-correction image and the post-correction image by the function of the global position / distortion correction unit 40 (global positional distortion correction). Process) is executed (S104).

  Then, the image processing apparatus 2 uses the function of the local position / distortion correction unit 42 to detect a local positional deviation between the pre-correction image and the post-correction image that have been entirely aligned in step S104. And a process of correcting distortion (local position distortion correction process) is executed (S105). For example, the processing shown in the flowchart shown in FIG. 9 is executed.

  That is, the image processing apparatus 2 extracts difference information by comparing a binary image obtained by binarizing the pre-correction image with a predetermined threshold and a binary image obtained by binarizing the post-correction image using the predetermined threshold (S301). ). For example, the image processing apparatus 2 extracts difference information by subtracting an image obtained by expanding ON pixels of the binary image of the pre-correction image from the binary image of the post-correction image.

  Then, the image processing apparatus 2 integrates the difference extraction pixels that can be regarded as the same region based on the difference extraction pixel that is the difference information extracted in the step of S301 to form an integrated region (pixel group) (S302).

  Then, the image processing apparatus 2 selects one of the integrated areas determined in step S302 (hereinafter, attention area) (S303). And the control part 22 makes the predetermined area | region containing the said attention area | region the 2nd division area about a corrected image (S304). Then, the image processing apparatus 2 sets the area expanded by a predetermined size from the second divided area as the first divided area for the pre-correction image (S305).

  Then, the image processing apparatus 2 cuts out an image corresponding to the second divided area from the binary image of the corrected image and sets it as the second image (S306). Further, the image processing apparatus 2 cuts out an image corresponding to the first divided area from the binary image of the pre-correction image and sets it as the first image (S307).

  Then, the image processing device 2 moves the position of the second image in the first image, and obtains the positional relationship between the first image and the second image when the degree of matching is the highest. Then, the image processing apparatus 2 corrects the position of the image in the region corresponding to the first image in the pre-correction image so that the positional relationship is satisfied (S308).

  Then, if there is an integrated area where the steps S304 to S308 have not been executed yet (NO in S309), the image processing apparatus 2 returns to the step S303 and executes the subsequent steps. On the other hand, if the steps from S304 to S308 have been executed for all the integrated regions (YES in S309), the image processing apparatus 2 ends the local position / distortion correction processing.

  Then, the image processing apparatus 2 performs binary processing of the corrected image (correctly corrected image corrected in S103) that has been subjected to the alignment both globally and locally through the steps of S104 and S105. A difference generation process for generating difference information is performed by subtracting the binary image of the pre-correction image (the pre-correction image corrected in step S103) from the image (S106). Then, the image processing apparatus extracts a correction portion based on the difference information (S107). In the above, difference information is generated by subtracting the binary image of the pre-correction image from the binary image of the post-correction image, but the binary image of the post-correction image is converted from the binary image of the pre-correction image. Difference information may be generated by subtraction, or difference information may be generated by both methods. Further, the image processing apparatus 2 generates difference information by subtracting the uncorrected image itself from the corrected image itself in step S106, and extracts a corrected portion based on the difference information in step S107. It may be.

  The above is the content of the processing executed by the image processing apparatus 2 according to the first embodiment of the present invention. Note that step S104 may be performed before step S101. In this case, since the positional distortion correction processing of the entire image is performed in advance before the comparison area is determined, both the images before and after the correction are determined simply by determining the comparison area in either the image before or after the correction. Thus, a substantially equivalent area is determined as a comparison area.

  For example, as can be seen from FIG. 10 (a), the pre-correction image includes the wording “Sunny and cloudy.”, And the post-correction image includes the added portion (ie, horizontal double line). And the character “after”), the image processing device 2 extracts the horizontal double line as the added portion and the character “after”. Further, for example, as can be seen from FIG. 10B, if the post-correction image is obtained by deleting a part of the pre-correction image, the image processing apparatus can be obtained by subtracting the post-correction image from the pre-correction image. 2 deletes the deleted part.

  As described above, the image processing apparatus 2 according to the present invention corrects the brightness of the pre-correction image and the post-correction image to be substantially the same, generates difference information from both images, Extract. Therefore, for example, compared to the case where difference information is generated without correcting both images, it is possible to suppress extraction of a portion that should not be extracted such as a blank portion. That is, the corrected part is accurately extracted.

  In addition, the image processing apparatus 2 according to the present invention corrects the brightness of the image before correction and the brightness of the image after correction to be approximately the same, and then aligns both images, so that both images are not corrected. Compared with the case where alignment is performed, the alignment accuracy is improved. Also from this point, the corrected portion is extracted accurately.

  Further, in the image processing apparatus 2 according to the present invention, a region that does not include a corrected portion is likely to be a comparison region. Therefore, the first color correction parameter and the second color correction parameter can be easily determined without being affected by the correction portion, and as a result, the region including the correction portion becomes the comparison region (for example, the entire image is compared with the comparison region). ), The unified accuracy of brightness is improved. Therefore, also from this point, the corrected portion is accurately extracted.

  Further, in the image processing apparatus 2 according to the present invention, since the correction process is performed only on a part of each of the pre-correction image and the post-correction image (that is, the comparison region), the processing necessary for determining the parameter set is reduced. .

  The image processing apparatus 2 executes the parameter set specifying process (S102) for the reduced image of the pre-correction image and the reduced image of the post-correction image after determining the page to be the comparison area in step S101. Thus, the parameter set may be specified. In this way, it is possible to reduce the processing load necessary for specifying the parameter set.

  Further, the image area or page serving as the comparison area may be a predetermined image area or page, or may be an image area or page designated by the user. In this case, if an image area or page that is difficult to be corrected is determined in advance as a comparison area, or if the user specifies an image area or page that is not corrected as a comparison area, an appropriate parameter set can be obtained. It becomes easy to be identified. The comparison area may be the entire area of each of the pre-correction image and the post-correction image.

  Further, the image processing apparatus 2 may correct the entire image before correction and the entire image after correction in the correction processing of S201. In this case, the image processing apparatus 2 may omit step S103.

  Further, when correcting the entire image before correction and correcting the entire image after correction in the correction processing of S201, the image processing apparatus 2 positions the image before correction and the image after correction for each page before the step of S202. May be adjusted as a whole. In this case, the image processing apparatus 2 may omit not only the step of S103 but also the step of S104.

Further, when correcting the entire image before correction and correcting the entire image after correction in the correction processing of S201, the image processing apparatus 2 omits the step of S101, and displays the image before correction and the image after correction subjected to the correction processing. The comparison area may be determined by executing a comparison area determination process on the target. For example, a case where the pre-correction image and the post-correction image include only an image for one page is taken as an example. In this case, after step S201, the image processing apparatus 2 aligns the positions of the pre-correction image and the post-correction image as a whole. Then, the comparison area is determined as follows.

For example, the image processing apparatus 2 generates difference information by subtracting a binary image obtained by binarizing the pre-correction image with a predetermined threshold and a binary image obtained by binarizing the post-correction image with the predetermined threshold, An integrated region of difference extraction pixels is defined. Then, the image processing apparatus 2 determines a rectangular area having a predetermined area that does not conflict with any integrated area as a comparison area. Then, the image processing apparatus 2 proceeds to step S202.

  For example, the image processing apparatus 2 analyzes the layout of each of the pre-correction image and the post-correction image, and divides each of the pre-correction image and the post-correction image into a document area, a table area, a figure area, and the like. Then, the image processing apparatus 2 determines, as a comparison area, an area where the position / range of the areas is most similar between the uncorrected image and the corrected image. Then, the image processing apparatus 2 proceeds to step S202.

  Next, Embodiment 2 of the present invention will be described with reference to the drawings.

  Also in the second embodiment, the image processing apparatus 2 according to the present invention constitutes the correction extraction system 100 shown in FIG. 1 and is connected to the image reading apparatus 4 so as to be communicable as in the first embodiment. As in the first embodiment, the user causes the image reading device 4 to read a paper document before correction, and registers a read image (hereinafter referred to as an image before correction) in the image processing device 2. In addition, the image reading device 4 is caused to read the corrected paper document in which correction such as addition or deletion with a white pen is performed on the uncorrected paper document, and the read image (hereinafter, corrected image) is read by the image processing device 2. sign up. Then, the user performs a predetermined correction portion extraction operation, and causes the image processing apparatus 2 to compare the uncorrected image with the corrected image and extract the corrected portion. As in the first embodiment, the image reading device 4 is realized as various scanners including an automatic paper feeding mechanism such as ADF. The paper document includes a plurality of paper media.

  The image processing apparatus 2 is realized as a personal computer or a server, for example. As in the first embodiment, the image processing apparatus 2 includes a control unit 22, a storage unit 24, an image input unit 26, an operation unit 28, and other units not shown (see FIG. 2). The storage unit 24, the image input unit 26, and the operation unit 28 are the same as those described in the first embodiment.

  The control unit 22 is a CPU or the like and operates according to a program stored in the storage unit 24. Specifically, the control unit 22 compares the pre-correction image with the post-correction image and executes a process for extracting a correction part. The processing executed by the control unit 22 will be specifically described later.

  FIG. 11 is a block diagram of functions realized by the image processing apparatus 2 when the control unit 22 operates in accordance with a program stored in the storage unit 24. As shown in the figure, the image processing apparatus 2 includes a comparison area determination unit 52, a binarization unit 54, a comparison unit 56, an overall position / distortion correction unit 58, a local position / distortion correction unit 60, A correction portion extraction unit 62. Hereinafter, each part will be described. The comparison region determination unit 52, the local position / distortion correction unit 60, and the corrected portion extraction unit 62 are respectively the comparison region determination unit 32, the local position / distortion correction unit 42 described in the first embodiment, Since it has the same function as the corrected portion extraction unit 44, the description thereof is omitted here.

(Binarization unit 54)
The binarization unit 54 is realized with the control unit 22 as the center. The binarization unit 54 binarizes the image with binarization parameters. In the present embodiment, the binarization unit 54 changes at least one of the first binarization parameter B1 and the second binarization parameter B2, and uses the first binarization parameter B1 for the comparison region of the pre-correction image. The binarized image and the binarized image in the comparison region of the corrected image with the second binarized parameter B2, thereby setting the parameter set of the first binarized parameter B1 and the second binarized parameter B2. One is specified. Here, the binarization unit 54 changes at least one of the first binarization parameter B1 and the second binarization parameter B2 until the pixel number difference D calculated by the comparison unit 56 becomes equal to or less than the predetermined threshold THon. Then, the image in the comparison area of the pre-correction image and the image in the comparison area of the post-correction image are binarized, and a parameter set in which the pixel number difference D is equal to or less than the predetermined threshold THon is specified. Here, the pixel number difference D is a difference between the number of ON pixels of the binary image in the comparison area of the image before correction and the number of ON pixels of the binary image in the comparison area of the image after correction. The first binarization parameter B1 is a brightness threshold value when binarizing the image in the comparison area of the uncorrected image, and the second binarization parameter B2 is the binary image of the image in the comparison area of the corrected image. It is a lightness threshold value when it is converted into a value.

(Comparator 56)
The comparison unit 56 is realized centering on the control unit 22. The comparison unit 56 calculates the difference in the number of ON pixels of the two binary images. In the present embodiment, the comparison unit 56 is configured such that the binarization unit 54 changes the image in the comparison area of the image before correction and the image before correction while changing at least one of the first binarization parameter B1 and the second binarization parameter B2. Each time the image in the comparison area is binarized, the difference between the binary image in the comparison area of the pre-correction image and the binary image in the comparison area of the post-correction image and the number of ON pixels (pixel number difference D) is calculated. calculate.

(Overall position / distortion correction unit 58)
The overall position / distortion correction unit 58 is realized centering on the control unit 22. The overall position / distortion correction unit 58 performs overall alignment of two target images, and corrects misalignment and distortion between the two images. In the present embodiment, the overall position / distortion correction unit 58 uses a well-known method (the method described in JP-A-2006-235785), similarly to the overall position / distortion correction unit 40 of the first embodiment. For each page, in accordance with the corrected image binarized with the second binarization parameter B2 of the parameter set specified by the binarization unit 54, the binarization with the first binarization parameter B1 of the parameter set This corrects the overall positional deviation and distortion of the pre-correction image.

  Hereinafter, referring to the flowchart of FIG. 12, an example of processing executed by the image processing apparatus 2 when the user designates the pre-correction image and the post-correction image stored in the storage unit 24 and issues a correction part extraction instruction. While explaining.

  When receiving the correction extraction instruction operation, the image processing apparatus 2 executes the comparison area determination process and determines the comparison area as in the first embodiment (S401). The comparison area may be a predetermined area or an area designated by the user. In this case, the comparison area determination process may be omitted.

  Then, the image processing apparatus 2 executes a process (parameter set specifying process) for specifying a parameter set in which the pixel number difference D is equal to or less than THon (S402). Here, the parameter set specifying process will be described with reference to FIG.

  In the parameter set specifying process, the image processing apparatus 2 includes a lightness pixel number distribution (first lightness histogram) in the comparison area of the image before correction, a lightness pixel number distribution (second lightness histogram) in the comparison area of the image after correction, Is generated. Then, initial values of the first binarization parameter B1 and the second binarization parameter B2 are determined (S501). For example, the image processing apparatus 2 uses the known binarization method (eg, P-tile method, mode method, Otsu's method, etc.) based on the first lightness histogram and the second lightness histogram, respectively. The initial values of B1 and the second binarization parameter B2 are determined.

  Then, the image processing apparatus 2 binarizes the image in the comparison area of the uncorrected image and the image in the comparison area of the corrected image with each of the first binarization parameter B1 and the second binarization parameter B2. (Binarization processing) is executed (S502).

Then, the image processing apparatus 2 executes a process (comparison process) for comparing the binary image in the comparison area of the pre-correction image and the binary image in the comparison area of the post-correction image (S503). Here, the image processing apparatus 2 counts the number of ON pixels of the binary image in the comparison area of the image before correction and the number of ON pixels of the binary image in the comparison area of the image before correction, and the number of pixels. The difference D is calculated. The image processing apparatus 2 uses the first lightness histogram and the second lightness histogram to calculate the number of ON pixels of the binary image in the comparison area of the image before correction and the number of ON pixels of the binary image in the comparison area of the image after correction. You may ask for it. In this case, the image processing apparatus 2 does not have to execute the binarization process in step S502.

  Then, when the pixel number difference D is larger than the predetermined threshold THon (N in S504), the image processing device 2 determines that the first binarization parameter B1 is set so that the pixel number difference D is equal to or less than THon. The second binarization parameter B2 is changed, and the process returns to step S502. For example, when the number of ON pixels of the binary image in the comparison area of the pre-correction image is smaller than the number of ON pixels in the comparison area of the post-correction image, the image processing apparatus 2 sets the value of the first binarization parameter B1 to a predetermined value. Decrease the value or increase the value of the second binarization parameter B2 by a predetermined value. In addition, for example, when the number of ON pixels of the binary image in the comparison area of the image before correction is larger than the number of ON pixels in the comparison area of the image after correction, the image processing apparatus 2 sets the value of the first binarization parameter B1. Is increased by a predetermined value, or the value of the second binarization parameter B2 is decreased by a predetermined value.

  On the other hand, when the pixel number difference D is equal to or less than THon (Y in S504), the image processing apparatus 2 ends the parameter set specifying process. In this way, the image processing apparatus 2 specifies a parameter set of the first binarization parameter B1 and the second binarization parameter B2 in which the pixel number difference D is equal to or less than THon.

  Here, the image processing apparatus 2 may calculate the number L of labels to be described later in the comparison process (S503). That is, the image processing apparatus 2 subtracts a binary image obtained by binarizing the pre-correction image using the first binarization parameter B2 from the binary image obtained by binarizing the post-correction image using the second binarization parameter B2. Thus, difference information is generated, and an integrated region of difference extraction pixels in the comparison region is determined. Here, the image processing apparatus 2 may generate the difference information by subtracting the binary image in the comparison area of the image before correction from the binary image in the comparison area of the image after correction. Then, the image processing apparatus 2 obtains the number of integrated regions, that is, the label number L. In step S504, when the value of the label number L is larger than the predetermined threshold (N in S504), the image processing apparatus 2 changes at least one of the first binarization parameter B1 and the second binarization parameter B2. Then, the processing from S502 to S504 may be executed. On the other hand, if the value of the label number L is equal to or smaller than the predetermined threshold (Y in S504), the parameter set specifying processing may be terminated.

  Further, the image processing apparatus 2 may calculate two variance values σX and σY, which will be described later, in the comparison process (S503). That is, the image processing apparatus 2 subtracts a binary image obtained by binarizing the pre-correction image using the first binarization parameter B1 from the binary image obtained by binarizing the post-correction image using the second binarization parameter B2. Thus, difference information is generated. Here, the image processing apparatus 2 may generate the difference information by subtracting the binary image in the comparison area of the image before correction from the binary image in the comparison area of the image after correction. Then, the image processing apparatus 2 performs a projection operation for each difference extraction pixel in the comparison area, and generates a projection histogram of the difference extraction pixel for each of the horizontal direction (X direction) and the vertical direction (Y direction) of the comparison area. To do. Then, the image processing apparatus 2 calculates a variance value σX in the X direction and a variance value σY in the Y direction of the difference extraction pixel from each projection histogram. Then, in step S504, when at least one of the values of σX and σY is larger than the predetermined threshold (N in S504), the image processing apparatus 2 determines at least one of the first binarization parameter B1 and the second binarization parameter B2. On the other hand, the processing of S502 to S504 may be performed with one changed, and if the values of σX and σY are both equal to or smaller than a predetermined threshold (Y in S504), the parameter set specifying processing may be terminated.

  Then, the image processing apparatus 2 binarizes the entire image before correction and the entire image after correction using the parameter set specified in step S402 (S403). Then, the image processing apparatus 2 executes a process for correcting the overall positional deviation and distortion of the binary image of the uncorrected image in accordance with the binary image of the corrected image (overall position distortion correcting process) (S404). .

  Then, the image processing apparatus 2 detects a local positional deviation or distortion between the binary image of the pre-correction image and the binary image of the post-correction image that have been entirely aligned in step S404. A correction process (local position distortion correction process) is executed (S405). Specifically, the image processing apparatus 2 executes the process shown in the flowchart of FIG. 9 as in the first embodiment.

  Then, the image processing apparatus 2 subtracts the binary image of the pre-correction image from the binary image of the post-correction image that has been subjected to the alignment both globally and locally through the steps of S404 and S405. Thus, a difference generation process for generating difference information is executed (S406). Then, the image processing apparatus 2 extracts a correction portion based on the difference information (S407).

  As described above, the image processing apparatus 2 according to the present invention binarizes so that the binary image of the pre-correction image and the binary image of the post-correction image are substantially the same, and then the difference from both binary images. Generate information and extract corrections. Therefore, for example, compared with the case where the pre-correction image and the post-correction image are binarized with the same threshold value, it is possible to suppress extraction of a portion that should not be extracted such as a blank portion. That is, the corrected part is accurately extracted.

  Further, the image processing apparatus 2 according to the present invention binarizes the binary image of the pre-correction image and the binary image of the post-correction image so as to be substantially the same, and then aligns the binary images. Therefore, the alignment accuracy is improved. Also from this point, the corrected portion is extracted accurately.

  Further, in the image processing apparatus 2 according to the present invention, a region that does not include a corrected portion is likely to be a comparison region.

  Further, in the image processing apparatus 2 according to the present invention, since the binarization process is performed only on a part of each of the pre-correction image and the post-correction image, processing necessary for determining the parameter set is reduced.

  The image processing apparatus 2 determines the image area or page to be the comparison area in step S401, and then performs parameter set specifying processing (S402) for the reduced image of the pre-correction image and the reduced image of the post-correction image. The parameter set may be specified by executing. In this way, it is possible to reduce the processing load necessary for specifying the parameter set.

  Further, the image processing apparatus 2 may proceed to the difference information generation process (S406) without performing alignment between the pre-correction image and the post-correction image binarized in step S403. That is, the image processing apparatus 2 may omit steps S404 and S405.

  The comparison area and page may be a predetermined image area or page, or may be an image area or page designated by the user. The comparison area may be the entire area of each of the pre-correction image and the post-correction image.

  Further, the image processing apparatus 2 may binarize the entire image before correction and binarize the entire image after correction in the binarization processing of S502. In this case, the image processing apparatus 2 may omit step S403.

In addition, when the image processing apparatus 2 binarizes the entire image before correction and binarizes the entire image after correction in the binarization process of S502, the image processing apparatus 2 sets the image before correction for each page before the step of S503. You may make it match | combine the position with the image after correction entirely. In this case, the image processing apparatus 2 may omit not only step S403 but also step S404.

  Further, when the image processing apparatus 2 binarizes the entire image before correction and also binarizes the entire image after correction in the binarization process of S502, the step of S401 is omitted and the binarization process is performed. The comparison area may be determined by executing the comparison area determination process on the uncorrected image and the corrected image. For example, a case where the pre-correction image and the post-correction image include only an image for one page is taken as an example. In this case, after the step of S502, the image processing apparatus 2 aligns the positions of the pre-correction image and the post-correction image as a whole based on the binary image of the pre-correction image and the binary image of the post-correction image. . Then, the comparison area is determined as follows.

  For example, the image processing device 2 generates difference information by subtracting the binary image of the pre-correction image and the binary image of the post-correction image, and determines the integrated region of the difference extraction pixels. Then, the image processing apparatus 2 determines a rectangular area having a predetermined area that does not conflict with any integrated area as a comparison area. Then, the image processing apparatus 2 proceeds to step S503.

  For example, the image processing apparatus 2 analyzes the layout of each of the pre-correction image and the post-correction image, and divides each of the pre-correction image and the post-correction image into a document area, a table area, a figure area, and the like. Then, the image processing apparatus 2 determines, as a comparison area, an area where the position / range of the areas is most similar between the uncorrected image and the corrected image. Then, the image processing apparatus 2 proceeds to step S503.

  In addition, this invention is not applied only to the said Embodiment 1 and Embodiment 2. FIG.

  For example, the image processing device 2 and the image reading device 4 may be separate or integrated.

  The pre-correction image and the post-correction image may be images read by different image reading devices 4.

  The program for operating the control unit 22 as described above may be supplied from an information storage medium (CD-ROM, DVD-ROM, etc.) storing this program. It may be supplied from a communication medium (for example, a medium that temporarily or fluidly holds a program like a communication line or a communication system).

It is a figure showing an example of composition of a correction extraction system concerning an embodiment of the present invention. It is a figure showing an example of the composition of the image processing device concerning the embodiment of the present invention. It is a figure showing an example of the functional block of the image processing apparatus which concerns on embodiment of this invention. It is a figure showing an example of the functional block of the image processing apparatus which concerns on embodiment of this invention. It is a figure showing an example of the memory content of a parameter set storage part. It is a figure showing an example of local position and distortion correction. It is a flowchart figure showing an example of processing performed with an image processing device concerning an embodiment of the present invention. It is a flowchart figure showing an example of processing performed with an image processing device concerning an embodiment of the present invention. It is a flowchart figure showing an example of processing performed with an image processing device concerning an embodiment of the present invention. It is a figure showing an example of a mode when a correction part is extracted from the image before correction, and the image after correction. It is a figure showing an example of the functional block of the image processing apparatus which concerns on another embodiment of this invention. It is a flowchart figure showing an example of the process performed with the image processing apparatus which concerns on another embodiment of this invention. It is a flowchart figure showing an example of the process performed with the image processing apparatus which concerns on another embodiment of this invention.

Explanation of symbols

  2 image processing device, 4 image reading device, 22 control unit, 24 storage unit, 26 image input unit, 28 operation unit, 32 comparison area determination unit, 34 correction unit, 36 comparison unit, 38 parameter set storage unit, 40 overall position Distortion correction unit, 42 local position / distortion correction unit, 42a difference extraction unit, 42b integration processing unit, 42c second divided region calculation unit, 42d first divided region calculation unit, 42e second image division unit, 42f first Image segmentation unit, 42g segmented image position / distortion correction unit, 44 correction portion extraction unit, 52 comparison area determination unit, 54 binarization unit, 56 comparison unit, 58 global position / distortion correction unit, 60 local position / distortion correction unit 62 Correction part extraction unit, 100 Correction extraction system.

Claims (23)

A first scan image read from a paper document, and a second scan image read from the paper document after positive Osamu, an image obtaining means for obtaining,
For each of the plurality of combinations stored in a storage unit that stores a combination of a first color correction parameter and a second color correction parameter that is the same type of color correction parameter as the first color correction parameter. The second color corrected based on the hue in the comparison area of the first scan image corrected based on the first color correction parameter included in the combination and the second color correction parameter included in the combination. Calculation means for calculating similarity information indicating the similarity between the hue in the comparison area of the scanned image;
Determination means for determining whether the calculated similarity information satisfies a predetermined similarity condition;
Of the plurality of combinations, the first scan image corrected based on the first color correction parameter included in the combination that obtains similarity information satisfying the similarity condition, and the combination included in the combination extraction means for extracting the corrected second scanned image, a correction area on the basis of on the basis of the second color correction parameters,
An image processing apparatus comprising: The calculating means includes
For each of the plurality of combinations, an appearance frequency distribution of each pixel value in the comparison area of the first scan image corrected based on the first color correction parameter included in the combination, and included in the combination Calculating the correlation coefficient between the appearance frequency distribution of each pixel value in the comparison area of the second scan image, corrected based on the second color correction parameter, as the similarity information;
The determination means includes
Determine whether the calculated correlation coefficient is greater than or equal to a predetermined value,
The extraction means includes
Of the plurality of combinations, the first scan image corrected based on the first color correction parameter included in the combination that obtains a correlation coefficient equal to or greater than the predetermined value, and the first scan image included in the combination. Extracting the corrected portion based on the second scan image corrected based on the two-color correction parameters ;
The image processing apparatus according to claim 1. The calculating means includes
For each of the plurality of combinations, the image in the comparison area of the first scan image corrected based on the first color correction parameter included in the combination , and the second color correction parameter included in the combination be calculated is corrected, and the image in the comparison area of the second scan image, the similarity information based on the difference information on the basis of,
The image processing apparatus according to claim 1. The similarity information is a numerical value indicating the number of difference extraction pixels or the variance of difference extraction pixels,
The determination means includes
It is determined whether or not the calculated similarity information value is equal to or less than a predetermined value,
The extraction means includes
Of the plurality of combinations, the first scan image corrected based on the first color correction parameter included in the combination in which similarity information having a value equal to or less than the predetermined value is obtained, and included in the combination Extracting the corrected portion based on the second scan image corrected based on the second color correction parameter ;
The image processing apparatus according to claim 3. A selection means for sequentially selecting each of the combinations included in the plurality of combinations;
The calculating means includes
Each time a combination is selected by the selection means, the hue in the comparison area of the first scan image corrected based on the first color correction parameter included in the selected combination, and the selected combination. Calculating the similarity information indicating the similarity between the second scan image and the hue in the comparison region, which is corrected based on the second color correction parameter included;
The image processing apparatus according to claim 1, wherein: The determination means includes
Each time a combination is selected by the selection means, it is determined whether the similarity information obtained from the selected combination satisfies the similarity condition,
The extraction means includes
When it is determined that the similarity information obtained from the selected combination satisfies the similarity condition, the first scan image corrected based on the first color correction parameter included in the combination, and the combination Extracting the correction portion based on the second scan image corrected based on the second color correction parameter included in
The selection means includes
When similarity information that does not satisfy the similarity condition is obtained from the selected combination, the next combination is selected, and when similarity information that satisfies the similarity condition is obtained from the selected combination, the next combination is selected. Not making a choice,
The image processing apparatus according to claim 5. The extraction means selects one combination from combinations obtained from similarity information satisfying the similarity condition based on the similarity information obtained from these combinations, and is included in the one combination. said first color correction parameter corrected based on the first scanned image, and the second scan image corrected based on the second color correction parameter included in the combination of the one, the correction area on the basis of Extract,
The image processing apparatus according to claim 5 . Reduction means for reducing the first scan image and the second scan image ;
The first scan images reduced by the reduction means corrects, based on the first color correction parameter, means for correcting, based an said second scan images reduced by the reduction means to the second color correction parameter And further including
The image processing apparatus according to any one of claims 1 to 7, characterized in. The comparison area image of the first scan image is corrected based on the first color correction parameter, and the comparison area image of the second scan image is corrected based on the second color correction parameter. Further comprising means for correcting,
The image processing apparatus according to any one of claims 1 to 7, characterized in. A first scan image read from a paper document, and a second scan image read from the paper document after positive Osamu, an image obtaining means for obtaining,
An image in the comparison region of the first scan image binarized based on the first value included in the combination while changing the combination of the two binarization parameter values used for the binarization of the image; Calculating means for calculating similarity information indicating similarity between the second scan image and the image in the comparison region binarized based on the second value included in the combination;
Determination means for determining whether the calculated similarity information satisfies a predetermined similarity condition;
Based on the first scan image binarized based on the first value included in the combination from which the similarity information satisfying the similarity condition is obtained, and on the second value included in the combination Extraction means for extracting a correction portion based on the binarized second scan image ;
An image processing apparatus comprising: The calculating means includes
While changing the combination, the number of significant pixels in the comparison region of the first scan image binarized based on the first value included in the combination, and the second number included in the combination A difference between the number of significant pixels in the comparison area of the second scan image, binarized based on the value, is calculated as the similarity information;
The determination means includes
Determine whether the calculated difference is less than or equal to a predetermined value,
The extraction means includes
The first scan image binarized based on the first value included in the combination that obtains the difference equal to or less than the predetermined value, and the binary based on the second value included in the combination Extracting the corrected portion based on the second scanned image obtained by
The image processing apparatus according to claim 1 0, characterized in that. The calculating means includes
Based on the second value included in the combination and the image in the comparison area of the first scan image binarized based on the first value included in the combination while changing the combination Calculating the similarity information based on difference information between the second scanned image and the image in the comparison area , which is binarized
The image processing apparatus according to claim 1 0, characterized in that. The similarity information is a numerical value indicating the number of difference extraction pixels or the variance of difference extraction pixels,
The determination means includes
It is determined whether or not the calculated similarity information value is equal to or less than a predetermined value,
The extraction means includes
Based on the first scan image binarized based on the first value included in the combination from which similarity information having a value equal to or less than the predetermined value is obtained, and the second value included in the combination And extracting the corrected portion based on the binarized second scan image ,
The image processing apparatus according to claim 1 2, characterized in that. The calculating means includes
Each time the combination is changed, the binarized image based on the first value included in the changed combination is included in the comparison region of the first scan image and the changed combination. Calculating similarity information indicating a similarity between the second scan image and the image in the comparison region binarized based on the second value;
The calculating means includes
Each time the calculation means changes the combination, it is determined whether the similarity information obtained from the changed combination satisfies the similarity condition,
The extraction means includes
The first scanned image binarized based on the first value included in the changed combination when it is determined that the similarity information obtained from the changed combination satisfies the similarity condition And the second scan image binarized based on the second value included in the combination after the change, and extracting the correction portion,
The calculating means includes
When similarity information that does not satisfy the similarity condition is obtained from the changed combination, the combination is changed again, and when similarity information that satisfies the similarity condition is obtained from the changed combination, Not make any changes,
The image processing apparatus according to any one of claims 10 to 13, characterized in. And reduction means for reducing said first scanned image, and the second scanned image, and
The first scan images reduced by pre-Symbol reducing means binarizes based on said first value, the binarization on the basis the second scan images reduced by the reduction means to the second value And further comprising means for
The image processing apparatus according to any one of claims 1 0 to 1 4, characterized in. Binarized image of the comparison area in the previous SL first scanned image based on the first value, the two images of the comparison region of said second scan images based on the second value Further comprising means for valuating,
The image processing apparatus according to any one of claims 10 to 14, wherein the. Prior Symbol ratio較領zone is an area occupying a portion of each of the first scan image and the second scan image,
The image processing apparatus according to any one of claims 1 to 1 6, characterized in that. The given comparison area is a predetermined area;
The image processing apparatus according to claim 17 . The given comparison area is an area specified by the user;
The image processing apparatus according to claim 17 . Analyzing means for analyzing a layout of the first scan image and the second scan image;
Determining means for determining the pre-Symbol ratio較領region based on a result of analysis by the analysis means,
The image processing apparatus according to claim 17 , further comprising: Difference extracting pixels to identify the integrated area that integrates a specified integrated region and conflict region not pre Symbol ratio from the difference between the binary image of the binary image and the second scan image of the first scanned image further comprising determining means to determine as較領zone,
The image processing apparatus according to claim 17 . Image obtaining means for obtaining a first scan image read from a paper document, and a second scan image read from the paper document after positive Osamu, a,
For each of the plurality of combinations stored in a storage unit that stores a combination of a first color correction parameter and a second color correction parameter that is the same type of color correction parameter as the first color correction parameter. The second color corrected based on the hue in the comparison area of the first scan image corrected based on the first color correction parameter included in the combination and the second color correction parameter included in the combination. Calculation means for calculating similarity information indicating the similarity between the color of the scanned image in the comparison area and
Determining means for determining whether the calculated similarity information satisfies a predetermined similarity condition;
Of the plurality of combinations, the first scan image corrected based on the first color correction parameter included in the combination that obtains similarity information satisfying the similarity condition, and the combination included in the combination extracting means for extracting a second scan image corrected based on the second color correction parameters, the correction area on the basis of,
As a program that allows the computer to function. Image obtaining means for obtaining a first scan image read from a paper document, and a second scan image read from the paper document after positive Osamu, a,
An image in the comparison region of the first scan image binarized based on the first value included in the combination while changing the combination of the two binarization parameter values used for the binarization of the image; Calculating means for calculating similarity information indicating similarity between the second scan image and the image in the comparison area, binarized based on the second value included in the combination;
Determination means for determining whether the calculated similarity information satisfies a predetermined similarity condition;
Based on the first scan image binarized based on the first value included in the combination from which the similarity information satisfying the similarity condition is obtained, and on the second value included in the combination Extraction means for extracting a correction portion based on the binarized second scan image ;
As a program that allows the computer to function.

Images