JP2008135857A - Image processor and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor and an image processing program that can detect an image of a corrected area of a document at high speed. <P>SOLUTION: The image processor 20 extracts featured areas having a predetermined feature from a first image and a second image based upon image data before correction representing an image (first image) of the document before correction and image data after correction representing an image (second image) of the document after correction, and derives predetermined feature quantities from the featured areas, specifies a featured area having the similarity of featured quantities larger than a predetermined degree between the first and second images, selects an area which is not included in the specified feature areas among the extracted featured areas as a corrected area, determines a correction state for the document based upon the corrected area, and detects a corrected image according to the determination result. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Conventionally, as a technique for specifying a location to be added to a predetermined document such as a form, Patent Literature 1 and Patent Literature 2 describe that before pre-recording and post-recording image alignment, And post-recording image data are monochromatized and binarized, and a technique for extracting a postscript part by performing a predetermined calculation between the pre-recording image data and the post-recording image data obtained thereby is disclosed. .

  Further, in Patent Document 3, the images before and after the additional recording are divided into a predetermined number, the images are aligned for each divided area, and then the image before the additional recording is expanded to obtain the image data before the additional recording and the image data after the additional recording. A technique for extracting a postscript location by calculating an exclusive OR for each pixel of the above is disclosed.

Further, in Patent Document 4, an image of a document before appending and an image of a document after appending are acquired, both images are aligned, and both images are divided into grid-like small areas, and a black character is obtained for each divided area. The ratio, the sum of the image densities, and the sum of the densities for each color are compared, thereby identifying the divided areas with additional writing, and realigning (fine-tuning) both images for each divided area with the additional writing. In addition, a technique for extracting a postscript image by subtracting a pre-recording image from a post-recording image in the finely adjusted divided area is disclosed.
JP 2004-80601 A JP 2004-240598 A JP 2004-341914 A JP 2004-213230 A

  The present invention has been made in the technical background as described above, and it is an object of the present invention to provide an image processing apparatus and an image processing program capable of detecting at high speed an image of a region where a document has been corrected. .

  In order to achieve the above object, the image processing apparatus according to claim 1, the first image information indicating an image of a first document that is a document before correction is performed, and the document after correction is performed. Acquisition means for acquiring second image information indicating an image of the second document, and based on the first image information and the second image information, the first document image and the second document image in advance Based on extraction means for extracting a feature area having a predetermined feature, and the feature area in the image of the first document and the image of the second document based on the first image information and the second image information. Deriving means for deriving the obtained feature amount, identifying means for identifying the feature region in which the similarity of the feature amount is a predetermined degree or more between the image of the first document and the image of the second document, The first document is extracted by the extracting means. A first feature region which is a feature region extracted from an image and which is not included in the feature region specified by the specifying unit; and a feature region extracted from the image of the second document by the extracting unit. And selecting means for selecting a second feature area that is not included in the feature area specified by the specifying means as a correction area that is the corrected area, the first feature area, and the first feature area A determination unit configured to determine a correction state of the document based on the two characteristic areas; and a detection unit configured to detect a corrected image according to a determination result by the determination unit.

  The invention according to claim 2 is the invention according to claim 1, wherein the determination means sets the first feature region and the second feature region as overlapping regions, and the first feature region and the second feature region overlap each other. When the number of black pixels when the image in the overlapping area in the area is binarized is smaller than the number of black pixels when the image in the overlapping area in the second feature area is binarized, the overlapping area Is determined to be an overlapped additional recording area that is additionally recorded on an image previously existing in the first document.

  According to a third aspect of the present invention, in the second aspect of the present invention, the second image information is targeted for an area determined by the determining means as the overlapped additional recording area. The added image is detected by subtracting the first image information for each pixel.

  According to a fourth aspect of the present invention, there is provided the method according to any one of the first to third aspects, wherein the determination unit overlaps the first feature region and the second feature region. The number of black pixels when the image in the overlap area in the first feature area is binarized is compared with the number of black pixels when the image in the overlap area in the second feature area is binarized. In the case where there are a large number of images, it is determined that the overlap area is a partially erased area in which a part of an image previously existing in the first document is erased.

  According to a fifth aspect of the present invention, in the invention of the fourth aspect, the first image information is targeted for an area determined by the determining means as the partially erased area. The second image information is subtracted from each pixel to detect the partially erased image.

  The invention according to claim 6 is the invention according to any one of claims 1 to 5, wherein the determination means is included in the first feature area, and an area overlapping each other is the second area. An area that does not exist in the characteristic area is determined to be an erased area in which an image previously existing in the first document is erased.

  According to a seventh aspect of the invention, in the sixth aspect of the invention, the detection means extracts information in the area determined as the erasure area by the determination means from the first image information. By doing so, the erased image is detected.

  The invention according to claim 8 is the invention according to any one of claims 1 to 7, wherein the determination means is included in the second feature region, and regions overlapping each other are the first. An area that does not exist in the characteristic area is determined to be an additional recording area that does not have an image and is additionally recorded in the first document.

  The invention according to claim 9 is the invention according to claim 8, wherein the information in the area determined by the determining means as the non-overlapping additional area by the determining means is the second image information. The added image is detected by extracting from.

  The invention according to claim 10 is the invention according to any one of claims 1 to 9, wherein the first image information and the second image in which the predetermined feature is binarized. It is a black or white connected image in the image information.

  The invention according to claim 11 is the invention according to any one of claims 1 to 10, wherein the deriving means further includes position information indicating the position of the feature region extracted by the extracting means. And the specifying means specifies the feature region only for the feature region whose distance indicated by the position information is within a predetermined distance.

  The invention according to claim 12 is the invention according to any one of claims 1 to 11, wherein only the region where the first feature region and the second feature region overlap each other is the target. The image processing apparatus further includes an alignment unit that performs an image alignment process on at least one of the one image information and the second image information.

  According to a thirteenth aspect of the present invention, in the twelfth aspect of the present invention, prior to the alignment processing by the alignment means, the first image information and the second image information are at least one of the first image information and the second image information. The image forming apparatus further includes pre-alignment means for performing alignment processing on the entire image of one original image and the second original image.

  Further, in the invention described in claim 14, in the invention described in claim 13, a correction coefficient is calculated based on a shift amount of the corresponding feature region in the image of the first document and the image of the second document. The image forming apparatus further includes a calculation unit, and the pre-alignment unit performs a correction process on at least one of the first image information and the second image information using the correction coefficient as the alignment process.

  On the other hand, in order to achieve the above object, the image processing program according to claim 15 includes: first image information indicating an image of a first document that is a document before correction is performed; and after the correction is performed. An acquisition step for acquiring second image information indicating an image of the second document as the document, and an image of the first document and an image of the second document based on the first image information and the second image information An extraction step for extracting a feature region having a predetermined feature in the image, and, based on the first image information and the second image information, from the feature region in the image of the first document and the image of the second document A deriving step for deriving a predetermined feature amount, and a specifying step for identifying the feature region where the similarity of the feature amount is equal to or greater than a predetermined degree between the image of the first document and the image of the second document. ,in front A first feature region that is a feature region extracted from the image of the first document by the extraction step and that is not included in the feature region specified by the specification step, and a second feature of the second document by the extraction step A selection step of selecting a second feature region that is a feature region extracted from an image and that is not included in the feature region specified by the specifying step as a correction region that is the corrected region; Causing a computer to execute a determination step of determining a correction state of the document based on the first feature region and the second feature region, and a detection step of detecting a correction image according to a determination result of the determination step It is.

  According to the first and fifteenth aspects of the present invention, it is possible to obtain an effect that it is possible to detect an image of an area where a document has been corrected at high speed.

  Further, according to the second aspect of the present invention, there is an effect that it is possible to determine an overlapping additional recording area that is additionally recorded over an existing image.

  Further, according to the third aspect of the invention, it is possible to obtain an effect that an additionally written image can be detected at a higher speed than when the present invention is not applied.

  According to the fourth aspect of the present invention, there is an effect that it is possible to determine a partially erased area in which a part of an existing image is erased.

  Further, according to the invention described in claim 5, it is possible to detect an image in which a part is erased at a higher speed than in the case where the present invention is not applied.

  According to the sixth aspect of the present invention, there is an effect that it is possible to determine the erased area where the preexisting image is erased.

  Further, according to the invention described in claim 7, it is possible to obtain an effect that an erased image can be detected at a higher speed than when the present invention is not applied.

  Further, according to the eighth aspect of the present invention, there is an effect that it is possible to determine a non-overlapping additional recording area in which no image exists and is additionally recorded.

  Further, according to the ninth aspect of the present invention, it is possible to detect an additionally written image at a higher speed than when the present invention is not applied.

  Further, according to the invention described in claim 10, as compared with the case where the present invention is not applied, it is possible to specify the corrected portion more easily, and as a result, the image of the region corrected at a higher speed can be obtained. The effect that it can detect is acquired.

  Further, according to the invention described in claim 11, as a result of being able to specify the feature region in a shorter time and more accurately than in the case where the present invention is not applied, the correction was performed at higher speed and with higher accuracy. The effect that the image of the area can be detected is obtained.

  Further, according to the twelfth aspect of the present invention, it is possible to detect an image of a region that has been corrected with higher accuracy than when the present invention is not applied.

  According to the thirteenth aspect of the present invention, it is possible to detect an image of a corrected region with higher accuracy than when the present invention is not applied.

  Further, according to the fourteenth aspect of the present invention, it is possible to detect an image of a region that has been corrected with higher speed and higher accuracy than when the present invention is not applied.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a configuration example of an image processing system 10 to which the present invention is applied. As shown in the figure, the image processing system 10 according to the present embodiment includes an image processing device 20 such as a personal computer and an image reading device 30 such as a scanner. The image processing device 20 and the image reading device 30 are electrically connected, and the image processing device 20 can acquire image data obtained by reading by the image reading device 30 from the image reading device 30.

  The image processing system 10 according to the present embodiment includes a document (hereinafter, referred to as “original document before correction”) before correction by additional writing by handwriting, stamping, etc., or correction by deletion by an eraser, correction liquid, or the like. The original after the correction (hereinafter referred to as “corrected original”) is read by the image reading device 30, and the correction is performed by the image processing device 20 based on the two image data obtained thereby. The specified part is specified, and the process of detecting (extracting) the image part corrected from the specified part is performed. In the following, image data obtained by reading a document before correction is referred to as “image data before correction”, and image data obtained by reading a document after correction is referred to as “image data after correction”.

  The image reading device 30 according to the present embodiment acquires image data obtained by reading as each pixel value (pixel information) is composed of a plurality of bits (8 bits in the present embodiment). ing. In the image processing system 10 according to the present embodiment, the image reading apparatus 30 separates and reads an image of a document to be read into three primary colors R (red), G (green), and B (blue). A color scanner is applied. The image reading device 30 is not limited to such a multi-bit color scanner, but a multi-bit monochrome scanner (so-called grayscale monochrome scanner), a 1-bit (binary) color scanner. A scanner, a monochrome scanner having a 1-bit configuration, or the like can also be applied.

  Next, with reference to FIG. 2, the main configuration of the electrical system of the image processing apparatus 20 having a particularly important role in the image processing system 10 will be described.

  As shown in the figure, the image processing apparatus 20 according to the present embodiment includes a CPU (central processing unit) 20A that controls the operation of the entire image processing apparatus 20, and a work area when the CPU 20A executes various processing programs. A RAM (Random Access Memory) 20B, a ROM (Read Only Memory) 20C in which various control programs, various parameters, and the like are stored in advance, a hard disk 20D that is a storage means used for storing various information, and various types A keyboard 20E used for inputting information, a display 20F used for displaying various types of information, and an input / output interface (I / F) 20G for exchanging various types of information with external devices are provided. These units are electrically connected to each other by a system bus BUS. Here, the image reading device 30 described above is connected to the input / output interface 20G.

  Accordingly, the CPU 20A accesses the RAM 20B, ROM 20C and hard disk 20D, acquires various input information via the keyboard 20E, displays various information on the display 20F, and various information with the image reading device 30 via the input / output interface 20G. Can be exchanged.

  FIG. 3 is a functional block diagram showing a functional configuration of the image processing apparatus 20 according to the present embodiment.

  As shown in the figure, the image processing apparatus 20 according to the present embodiment includes a first alignment unit 22, a correction detection unit 24 having a correction area determination unit 24A and a second alignment unit 24B, and a storage unit. 28 and a correction area selection unit 29.

  In addition, the first alignment unit 22 has, for at least one of the pre-correction image data and the post-correction image data, a pre-correction image indicated by the pre-correction image data, and a post-correction image indicated by the post-correction image data. Between the positions of the corresponding pixels that occur with respect to the entire image due to a difference in inclination angle between the original document after correction and the original document after correction, a difference in reading environment, etc. Alignment processing for correcting is performed. In the first alignment unit 22 according to the present embodiment, predetermined features in the uncorrected image and the corrected image based on the uncorrected image data and the corrected image data (in this embodiment, 2 Extracted feature regions having a black pixel connection image and a white pixel connection image in the binarized image data), and the pre-correction image data and the post-correction image so that the corresponding feature regions of the pre-correction image and the post-correction image overlap. At least one of the image data is converted.

  Here, in the first alignment unit 22 according to the present embodiment, the conversion is performed by performing affine transformation on either one of the pre-correction image data and the post-correction image data. However, the present invention is not limited to this, and other linear transformation, primary equiangular transformation, secondary equiangular transformation, projective transformation, enlargement / reduction, rotation, translation, etc. for any one of the pre-correction image data and post-correction image data A conventionally known conversion method may be applied.

  On the other hand, the correction detection unit 24 determines the correction state of the document based on the correction region selected from the feature region extracted by the first alignment unit 22 by the correction region selection unit 29 described later by the correction region determination unit 24A. A correction image is detected according to the determination result.

  By the way, in the image alignment by the first alignment unit 22, local images of the pre-correction image and the post-correction image generated due to uneven scanning speed or distortion of the optical system at the time of image reading by the image reading device 30 are used. It is difficult to correct (align) up to a large distortion.

  Therefore, the second alignment unit 24B provided in the correction detection unit 24 targets only at least a part of the correction region, and applies at least one of the pre-correction image data and the post-correction image data, Alignment processing for correcting a shift in the position of the corresponding pixel with the corrected image is executed.

  Note that the second alignment unit 24B according to the present embodiment converts at least one of the pre-correction image data and the post-correction image data so as to eliminate the positional deviation of the corresponding pixel as much as possible by the pattern matching process.

  Here, in the second alignment unit 24B according to the present embodiment, as the conversion, a correlation coefficient between the correction areas corresponding to the uncorrected image data and the corrected image data is calculated, and the correlation coefficient is the largest. A transformation that translates at least one of the pre-correction image and the post-correction image so as to be higher is applied. However, the present invention is not limited to this, and affine transformation or bilinear for either the pre-correction image data or the post-correction image data. Other conventionally known conversion methods such as transformation, primary equiangular transformation, secondary equiangular transformation, projective transformation, enlargement / reduction, rotation, translation, etc. may be applied.

  Then, the correction area selection unit 29 selects a correction area that is a corrected characteristic area from among the characteristic areas extracted by the first alignment unit 22, and sets correction area information indicating the selected correction area. The data is output to the correction detection unit 24.

  Next, with reference to FIG. 4, the structure of the 1st position alignment part 22 which concerns on this Embodiment is demonstrated.

  As shown in the figure, the first alignment unit 22 according to the present embodiment includes a pair of feature region extraction units 22A and 22B, an association unit 22C, a correction parameter calculation unit 22D, and an image conversion unit 22E. And.

  The feature region extraction unit 22A receives pre-correction image data, the feature region extraction unit 22B receives post-correction image data, and the feature region extraction units 22A and 22B receive input image data. Based on the above, a feature region having the above-described predetermined features in the pre-correction image and the post-correction image (in this embodiment, the black pixel connected image and the white pixel connected image in the binarized image data) is extracted. After that, a predetermined feature amount (in this embodiment, a size, the number of pixels, and a handwriting length (see also FIG. 8)) derived from the feature region is derived based on the input image data. And output as feature area information.

  Further, the association unit 22C has a feature region in which the similarity between the feature amounts indicated by the feature region information output from the feature region extraction units 22A and 22B is equal to or greater than a predetermined level between the pre-correction image and the post-correction image. Specific feature region information is created and output. The specific feature region information is associated with the specified feature region pair.

  Further, the correction parameter calculation unit 22D determines the overall shift between the pre-correction image and the post-correction image based on the shift amount of the corresponding feature region indicated by the specific feature region information output from the association unit 22C. A correction coefficient for eliminating it is calculated. Then, the image conversion unit 22E uses the correction coefficient calculated by the correction parameter calculation unit 22D, and applies the pre-correction image and the correction to the pre-correction image data or the post-correction image data (here, the post-correction image data). Image transformation (here, affine transformation) for performing overall alignment of the subsequent image is executed.

  The correction coefficients applied to the affine transformation are the width direction movement amount, the vertical direction movement amount, the rotation angle, and the magnification. As for the correction coefficient, when conversion based on the correction coefficient is performed on one image (here, the corrected image), the error of the centroid position of the corresponding feature region of the pre-correction image and the post-correction image is minimized. Calculated as a value. Specifically, for example, by applying the least square method, the movement amount in the width direction and the movement amount in the vertical direction are the correction coefficients that minimize the sum of the errors of the centroid positions of the corresponding feature regions of the pre-correction image and the post-correction image. , Rotation angle, and magnification are obtained. A method for calculating a correction coefficient using the least square method is described in Japanese Patent Laid-Open No. 9-93431 as an example.

  In the correction area selection unit 29 according to the present embodiment, feature area information corresponding to the pre-correction image output from the feature area extraction unit 22A and feature area information corresponding to the post-correction image output from the feature area extraction unit 22B Of the feature regions extracted by the feature region extraction units 22A and 22B based on the specific feature region information output from the association unit 22C, feature regions not included in the feature region indicated by the specific feature region information The correction area is selected as a correction area, and correction area information indicating the selected correction area is generated and output to the correction detection unit 24.

  In the correction area selection unit 29 according to the present embodiment, the correction area is selected from the characteristic areas in the pre-correction image extracted by the characteristic area extraction unit 22A to the characteristic area indicated by the specific characteristic area information. A feature region that is not included is selected as the correction region a, and a feature region that is not included in the feature region indicated by the specific feature region information is selected from the feature regions in the corrected image extracted by the feature region extraction unit 22B. It is done by selecting as.

  Next, with reference to FIG. 5, the structure of the feature area extraction units 22A and 22B according to the present embodiment will be described. Note that the feature region extraction unit 22A and the feature region extraction unit 22B have the same configuration except for the input image data, and therefore, the configuration of the feature region extraction unit 22A will be described here.

  As shown in the figure, the feature region extraction unit 22A according to the present embodiment includes a preprocessing unit 40, a connected region extraction unit 42, a centroid calculation unit 44, and a feature amount calculation unit 46. .

  Note that the pre-processing unit 40 performs monochromatic processing and binarization processing on the input pre-correction image data. Here, the monochromatic processing is processing in which the pre-correction image data is converted to pixel data of only luminance information. The binarization process is a process for converting a monochrome image obtained by the monochrome process into a value of 1 (black) or 0 (white) at a predetermined threshold level. Note that when the input image is a monochrome image, the monochromatic processing can be omitted.

  The connected region extracting unit 42 performs a connected region extracting process for extracting a connected region as the feature region on the pre-correction image data that has been binarized by the preprocessing unit 40. In addition, the said connection area | region extraction process can be performed applying the method etc. which are described in Unexamined-Japanese-Patent No. 12-295438 as an example. Here, the connected region (feature region) is extracted as a continuous region (black run) of black pixels in the binarized image or a continuous region (white run) of white pixels that are not black pixels. In the connected region extraction processing, for example, a range (minimum, maximum) of a predetermined size (area or number of pixels) is set in advance as an extraction condition, and a region of continuous identical pixel values in the range is determined in advance. It is executed based on the set conditions.

  Here, with reference to FIG. 6, the specific example of the connection area | region extracted by the said connection area | region extraction process is demonstrated.

  The figure shows a pre-correction image 80 and a post-correction image 90 that have been binarized. From these images, image data having a continuous region of black pixels (black run) and image data having a continuous region of white pixels (white run) are extracted as connected regions under certain conditions. In the example shown in the figure, black character images “A” and “B” and white character images “A” are extracted from the pre-correction image 80 as connected regions 81 to 83, and black characters are extracted from the post-correction image 90. An example in which “A” and “B” as images and “A” as an outline character image are extracted as connected regions 91 to 93 is shown. Note that the number of data in the connected area extracted in the actual processing is extracted in units of hundreds to thousands, although it depends on the design conditions.

  Further, the center of gravity calculation unit 44 calculates the center of gravity of the connected region (feature region) extracted by the connected region extraction unit 42, and the feature amount calculation unit 46 calculates the feature amount (here, the size) of each connected region. , Number of pixels, handwriting length). In the present embodiment, the centroid calculating unit 44 and the feature amount calculating unit 46 are distinguished from each other. However, the feature amount calculating unit 46 may calculate the centroid as one feature amount of the connected region. Needless to say.

  Next, with reference to FIGS. 7 and 8, the center of gravity and the feature amount obtained from the connected region will be described. The character ‘A’ shown in FIG. 7 is, for example, one connected region 70 extracted from the pre-correction image 80. FIG. 8 shows a table made up of information on the center of gravity and feature amounts for the connected regions extracted from the pre-correction image 80 and the post-correction image 90.

  First, with reference to FIG. 7, details of the center of gravity and the feature amount of the connection region 70 will be described.

  The centroid 72 is information indicating the position of the centroid of the character A70 as a connected area, and is calculated as, for example, coordinate data (x, y) when the uncorrected image 80 is an XY coordinate plane (see also FIG. 8). .

  The size is obtained as ‘L, H’ by the width L and the height H of the circumscribed rectangle 74 that circumscribes the connecting region 70 shown in FIG. 7. Further, the number of pixels is calculated as the total number of pixels of the connection region 70 itself. Further, as shown in FIG. 7, the handwriting length is calculated as the number of constituent pixels in the connection region thinning data 76 obtained by thinning the connection region 70 and converting it as a line having a width of one pixel.

  The center-of-gravity calculation unit 44 and the feature amount calculation unit 46, for each of the connected regions (feature regions) extracted by the connected region extraction unit 42 from the uncorrected image 80 and the corrected image 90 to be compared, The feature amount information including the size, the number of pixels, the handwriting length, and the like is calculated. As shown in FIG. 8 as an example, the feature area information 50A corresponding to the uncorrected image 80 and the corrected image 90 are displayed. The corresponding feature area information 50B is organized. As shown in the figure, each connection region extracted from each of the pre-correction image 80 and the post-correction image 90 is assigned a unique ID (Identification), and each connection region is associated with the ID. The information on the center of gravity of the area and the feature amount, that is, the size, the number of pixels, and the handwriting length are recorded as feature area information by a storage unit such as the hard disk 20D.

  By the way, as described above, the associating unit 22C (see FIG. 4) according to the present embodiment provides the feature region information output from the feature region extraction units 22A and 22B between the pre-correction image and the post-correction image. A feature region pair whose feature quantity similarity indicated by is equal to or greater than a predetermined degree is specified, and specific feature region information for associating the specified feature region pair is created.

  Here, as an example, the associating unit 22C receives the feature region information 50A recorded by the feature region extracting unit 22A and the feature region information 50B recorded by the feature region extracting unit 22B, and is recorded in both feature region information. Pattern matching processing based on the obtained information is performed, and specific feature region information is created based on the result.

  In addition, in the associating unit 22C according to the present embodiment, the other center where the center of gravity is located within a predetermined distance from the center of gravity of the feature region extracted from one of the pre-correction image and the post-correction image. By specifying the feature region of the image based on the barycentric position in each feature region information, the feature regions that are candidates for feature regions having a similarity equal to or higher than a predetermined degree are narrowed down, and only the narrowed feature regions are targeted. The similarity of at least one piece of information of the feature amount (size, number of pixels, handwriting length) in the feature region information is calculated, and information indicating a pair of feature regions whose similarity is equal to or higher than a predetermined degree is used as the specific feature region information. create. In the present embodiment, the reciprocal of the distance between applied feature amounts (for example, Euclidean distance) is applied as the similarity, but the present invention is not limited to this, and each of the uncorrected image and the corrected image Needless to say, any method can be used as long as it indicates the degree of similarity of the feature amount indicated by the feature region information.

  FIG. 9 schematically shows an example of the data structure of the specific feature area information according to the present embodiment. As shown in the figure, in the associating unit 22C according to the present embodiment, the IDs assigned to the feature regions of the pre-correction image and the post-correction image whose similarity is equal to or higher than a predetermined degree are associated with each other. Specific feature region information is created as a thing.

  The processing by each component (first alignment unit 22, correction detection unit 24, correction region selection unit 29) of the image processing apparatus 20 configured as described above uses a computer by executing a program. It can be realized by a software configuration. In this case, the program includes the image processing program of the present invention. However, the present invention is not limited to realization by a software configuration, and needless to say, it can also be realized by a hardware configuration or a combination of a hardware configuration and a software configuration.

  Hereinafter, a description will be given of a case where the image processing system 10 according to the present embodiment is configured to realize processing by each of the above-described components by executing the above-described program (hereinafter referred to as “image processing program”). To do. In this case, the image processing program is installed in the image processing apparatus 20 in advance, provided in a state stored in a computer-readable recording medium, or distributed via wired or wireless communication means. Forms and the like can be applied.

  Next, the operation of the image processing system 10 according to the present embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing the flow of processing of the image processing program executed by the CPU 20A of the image processing apparatus 20. Also, here, in order to avoid complications, the pre-correction image data and the post-correction image data to be processed are input from the image reading device 30 as indicating the same size original image, and are previously stored in a predetermined area of the hard disk 20D. The case where it is stored will be described.

  In step 100 of the figure, pre-correction image data and post-correction image data to be processed are obtained by reading from the hard disk 20D, and in the next step 102, the same processing as that by the above-described feature region extraction units 22A and 22B is performed. The characteristic area information (see also FIG. 8) corresponding to the pre-correction image and the post-correction image is derived by processing and recorded. For example, if the pre-correction image 80 and the post-correction image 90 are those shown in FIG. 12, the processing in this step 102 relates to the feature region shown in FIG. 13A (in this embodiment, a connected region). The feature area information is derived for each of the pre-correction image and the post-correction image.

  In the next step 104, based on the derived feature area information, specific feature area information is created by a process similar to the process by the association unit 22C described above. By the process of step 104, specific feature region information (see also FIG. 9) associated with the feature region shown in FIG. 13B is created as an example.

  In the next step 106, a correction coefficient for eliminating the overall shift between the pre-correction image and the post-correction image is obtained by the same processing as the processing by the correction parameter calculation unit 22D described above based on the created specific feature region information. In the next step 108, an image conversion process for performing overall alignment of the pre-correction image and the post-correction image by the same process as the process by the image conversion unit 22E described above using the calculated correction coefficient. (In this embodiment, affine transformation processing) is executed, and in the next step 110, extraction is performed in the feature region information derivation process in step 102 by the same processing as the processing by the correction region selection unit 29 described above. Correction area indicating the selected feature area (correction area) by selecting a correction area that has been corrected from the selected feature areas To create a broadcast. In step 108, the process of converting the feature area information corresponding to the image data that has been subjected to the image conversion process (here, the corrected image data) into a value corresponding to the image after the image conversion process is also executed. It is preferable.

  In the processing of this step 110, the correction area is selected by selecting an area that is not included in the feature area indicated by the specific feature area information from among the feature areas of the pre-correction image extracted by the feature area extraction unit 22A. When selecting the correction area a), correction area information indicating the characteristic area shown in FIG. 13C is generated as an example, and the selection of the correction area is extracted by the characteristic area extraction unit 22B. When the region not included in the feature region indicated by the specific feature region information among the feature regions of the corrected image is selected as the correction region (correction region b), an example is illustrated in FIG. Correction area information indicating the indicated characteristic area is created.

  In the next step 112, a white image having the width of the image before correction (here, the number of pixels in the horizontal direction of the image) and the height of the image before correction (here, the number of pixels in the vertical direction of the image) is acquired. The image data shown (hereinafter referred to as “white image data”) is created, and in the next step 114, the correction detection unit 24 processes the pre-correction image data and the correction corresponding to the correction area indicated by the correction area information. A modified image detection processing routine program for detecting a modified image is executed based on the post-image data.

  Hereinafter, the corrected image detection processing routine program will be described with reference to FIG.

  In step 200 of the figure, as shown as an example, as shown below, coordinates (hereinafter referred to as “correction”) indicating the range of all the correction areas a and b indicated by the correction area information created by the processing of step 110 described above. Derived as "region coordinates").

  First, each information of the center of gravity position and the size corresponding to each correction area is acquired from the characteristic area information.

  Next, the corner point of the rectangular area having the width L and the height H indicated by the acquired size information centered on the center of gravity position indicated by the acquired center of gravity position information (in this embodiment, the upper left corner point and The position coordinates of the lower right corner point) are calculated as the correction area coordinates for each correction area.

  In the next step 202, based on the correction area coordinates for each correction area derived by the processing in step 200, is there an overlapping area between the correction area included in the correction area a and the correction area included in the correction area b? If the determination is negative and the determination is negative, the process proceeds to step 220 described later, whereas if the determination is affirmative, the process proceeds to step 204.

  In step 204, of the correction areas determined to be overlapping areas by the process of step 202 in the pre-correction image data and the post-correction image data (the image conversion process executed by the process of step 108). The image conversion processing for aligning the pre-correction image and the post-correction image for any one set of regions (hereinafter referred to as “processing target region”) is the processing by the second alignment unit 24B described above. Do the same.

  It should be noted that the size of a set of correction areas set as processing target areas does not always match. For example, when the correction area a is as shown in FIG. 13C and the correction area b is as shown in FIG. 13D, “sa”, which is a correction area included in the correction area a, The size of each correction area is different from the correction area included in the correction area “b” (“X”) to which “X” is added. In this case, in the modified image detection processing routine program according to the present embodiment, the image conversion process is performed only for the region within the circumscribed rectangle of the smaller modified region.

  In the next step 206, the pre-correction image data and the post-correction image data corresponding to the set of correction areas that are the processing target areas and the areas in which the image conversion processing has been executed by the processing in step 204 are performed. The monochrome processing and binarization processing are executed in the same manner as the preprocessing unit 40 described above, and in the next step 208, the number of black pixels of the pre-correction image data subjected to the binarization processing (hereinafter referred to as “correction”). It is determined whether or not the “number of previous black pixels” is smaller than the number of black pixels in the corrected image data subjected to binarization processing (hereinafter referred to as “number of corrected black pixels”). If it becomes, the processing target area is regarded as an “additional area with overlap”, which is an area that is added to the pre-corrected image in advance, and the process proceeds to step 210.

  In step 210, a first corrected image detection process predetermined corresponding to the overlapped additional recording area is used by using the respective image data of the pre-correction image and the post-correction image subjected to the image conversion process in the process of step 204. After that, the process proceeds to step 216.

  In the modified image detection processing routine program according to the present embodiment, as the first modified image detection process, the image data before modification corresponding to the region subjected to the image conversion process by the process of step 204 is processed. Then, after performing an image expansion process of a predetermined number of dots (for example, 1 dot) on the image indicated by the uncorrected image data, the image conversion process is performed from the corrected image data by the process of step 204 above. Although the process of subtracting the pre-correction image data corresponding to the applied area for each corresponding pixel and for each of R, G, and B is applied, the present invention is not limited to this. The form in which the subtraction process is applied without being performed, or the pre-correction image data is converted from the post-correction image data binarized by the process in step 206 above. In every pixel to, and can be R, G, forms or the like for applying the process of subtracting each separate B, also be in the form of applying other conventional known additional image detection processing.

  On the other hand, when a negative determination is made in step 208, the process proceeds to step 212, where it is determined whether the number of black pixels before correction is larger than the number of black pixels after correction. On the other hand, if the determination is affirmative, the process target area is regarded as a “partially erased area” that is an area where a part of an image previously existing in the document before correction is erased. Move to 214.

  In step 214, a second modified image detection process predetermined for a part of the erasure region is performed using the image data of the uncorrected image and the corrected image subjected to the image conversion process in step 204. After that, the process proceeds to step 216.

  In the modified image detection processing routine program according to the present embodiment, as the second modified image detection process, the modified image data corresponding to the area subjected to the image conversion process by the process of step 204 is performed. Then, after performing an image expansion process of a predetermined number of dots (for example, 1 dot) on the image indicated by the corrected image data, the image conversion process is performed from the uncorrected image data by the process of step 204 above. Although the process of subtracting the corrected image data corresponding to the applied area for each corresponding pixel and for each of R, G, and B is applied, the present invention is not limited to this. A form of applying the process of performing the subtraction without performing the process, or the corrected image data from the uncorrected image data binarized by the process of step 206 above, In every pixel to, and can be R, G, forms or the like for applying the process of subtracting each separate B, also be in the form of applying other conventional known erasing image detection processing.

  In step 216, the detected corrected image or erased image is obtained by superimposing (overwriting) the image data obtained by the process of step 210 or 214 on the white image data created by the process of step 112. In the next step 218, it is determined whether or not the processing of step 204 to step 216 has been completed for all the correction regions determined to be overlapping regions by the processing of step 202. If YES in step 204, the process returns to step 204, and the processing in step 204 and subsequent steps is executed again. On the other hand, if a positive determination is made, the process proceeds to step 220. Note that, when the processes in steps 204 to 218 are repeatedly executed, overlapping correction areas that have not been processed until then are applied as processing target areas.

  In step 220, based on the correction area coordinates for each correction area derived by the processing in step 200, there is a correction area included in the correction area a that does not overlap any correction area included in the correction area b. If the result of the determination is affirmative, it is assumed that the processing target area is an “erased area” that is an area in which an image existing in the document before correction is erased, and the process proceeds to step 222. Then, using the pre-correction image data, a third corrected image detection process determined in advance corresponding to the erased area is executed, and thereafter, the process proceeds to step 224. In the modified image detection processing routine program according to the present embodiment, a process of extracting image data corresponding to the erasure area from the pre-correction image data is applied as the third modified image detection process.

  In step 224, the detected corrected image (erased image) is synthesized by superimposing (overwriting) the image data obtained by the processing of step 222 on the white image data, and then step 226 is performed. Migrate to If the determination in step 220 is negative, the process proceeds to step 226 without executing the processes in steps 222 and 224.

  In step 226, based on the correction area coordinates for each correction area derived by the processing in step 200, there is a correction area included in the correction area b that does not overlap any correction area included in the correction area a. If the result of the determination is affirmative, the processing target area is assumed to be a “non-overlapping additional area” in which no image exists in the uncorrected document and the additional area is added. The process proceeds to 228, and the fourth corrected image detection process predetermined in correspondence with the non-overlapping additional recording area is executed using the corrected image data, and then the process proceeds to step 230. In the modified image detection processing routine program according to the present embodiment, a process of extracting image data corresponding to the non-overlapping additional area from the corrected image data is applied as the fourth modified image detection process. .

  In step 230, the corrected image (added image) detected is synthesized by superimposing (overwriting) the image data obtained in the process of step 228 on the white image data, and then this correction is performed. The image detection processing routine program is terminated. If the determination in step 226 is negative, the modified image detection processing routine / program is terminated without executing the processing in steps 228 and 230. FIG. 13E shows the conditions and image examples of the additional write area with overlap, the partially erased area, the erase area, and the additional write area without overlap, and FIG. 13F shows the first to fourth modified image detections. The detection method of the corrected image by processing is described collectively.

  When the corrected image detection processing routine program ends, the process proceeds to step 116 of the image processing program (see FIG. 10), and the corrected image is output using the white image data on which the corrected image is superimposed by the above processing. This image processing program ends. In the image processing program according to the present embodiment, both the output by display on the display 20F and the output by printing by a printer (so-called image forming apparatus) (not shown) are applied as the process of outputting the corrected image in step 116. However, the present invention is not limited to this, and it goes without saying that other output forms such as a form in which any one of these is output or a form in which sound is output by a speech synthesizer can be applied. . In the image processing program according to the present embodiment, the case where the detected corrected image is recorded by superimposing the corrected image on the white image data has been described. However, the present invention is not limited to this. It goes without saying that other forms of recording information that can specify the corrected image, such as a form of recording coordinates indicating the position of the corrected image, can be used. Further, the output image and the erased image can be output in a state where the additional image and the erased image can be distinguished, for example, by changing the color between the additionally recorded image and the erased image.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various modifications or improvements can be added to the above-described embodiment without departing from the gist of the invention, and embodiments to which such modifications or improvements are added are also included in the technical scope of the present invention.

  The above embodiments do not limit the invention according to the claims (claims), and all the combinations of features described in the embodiments are essential for the solution of the invention. Is not limited. The above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. Even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, as long as an effect is obtained, a configuration from which these some constituent requirements are deleted can be extracted as an invention.

  For example, in the above embodiment, the case where the image processing is realized by the software configuration has been described. However, the present invention is not limited to this, and for example, the image processing is realized by the hardware configuration. You can also. As an example of this case, for example, a function of executing the same processing as each component (first alignment unit 22, correction detection unit 24, correction region selection unit 29) of the image processing apparatus 20 shown in FIG. The form which creates and uses a device can be illustrated. In this case, the image processing can be speeded up as compared with the above embodiment.

  In the above-described embodiment, a case has been described in which correction regions overlapping correction regions a and b are processed one by one in the repeated processing of step 204 to step 218 of the correction image detection processing routine program. The invention is not limited to this, for example, when a plurality of correction areas included in the correction area a overlap any one of the correction areas included in the correction area b, or a plurality of correction areas included in the correction area b. When the correction area overlaps any one of the correction areas included in the correction area a, the processing from step 204 to step 218 is executed as a single correction area by combining the correction areas. You can also. Also in this case, it can be expected that the image processing speeds up as compared with the above embodiment.

  In addition, the configurations (see FIGS. 1 to 5) of the image processing system 10 and the image processing apparatus 20 described in the above embodiment are merely examples, and unnecessary portions are deleted without departing from the gist of the present invention. Needless to say, you can add new parts.

  The data structure of various information described in the above embodiment (see FIGS. 8 and 9) is also an example, and it goes without saying that changes can be made without departing from the gist of the present invention.

  Furthermore, the processing flow (see FIGS. 10 and 11) of the image processing program and the modified image detection processing routine program described in the above embodiment is also an example, and is unnecessary within the scope of the present invention. It goes without saying that steps can be deleted, new steps can be added, and the processing order can be changed.

1 is a block diagram illustrating a configuration of an image processing system according to an embodiment. It is a block diagram which shows the principal part structure of the electric system of the image processing apparatus which concerns on embodiment. It is a functional block diagram which shows the functional structure of the image processing apparatus which concerns on embodiment. It is a block diagram which shows the structure of a 1st position alignment part among the functional structures of the image processing apparatus which concerns on embodiment. It is a block diagram which shows the structure of a characteristic area extraction part among the functional structures of the image processing apparatus which concerns on embodiment. It is a figure where it uses for description of the connection area | region (characteristic area | region) which concerns on embodiment. It is a figure with which it uses for description of each information of the gravity center calculated | required from the connection area | region which concerns on embodiment, and a feature-value. It is a schematic diagram which shows the data structure of the characteristic area information which concerns on embodiment. It is a schematic diagram which shows the data structure of the specific feature area information which concerns on embodiment. It is a flowchart which shows the flow of a process of the image processing program which concerns on embodiment. It is a flowchart which shows the flow of a process of the correction image detection process routine program which concerns on embodiment. It is a figure used for description of the processing contents of the image processing program and the corrected image detection processing routine program according to the embodiment, and is a front view showing an example of an image before correction and an image after correction. It is another figure with which it uses for description of the processing content of the image processing program which concerns on embodiment, and a correction image detection processing routine program.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing system 20 Image processing apparatus 20A CPU
20D Hard disk 20F Display 22 First alignment unit 22A Feature region extraction unit 22B Feature region extraction unit 22C Corresponding unit (specifying means)
22D correction parameter calculation unit (calculation means)
22E Image converter (pre-positioning means)
24 Correction detection unit (detection means)
24A Correction area determination unit (determination means)
24B 2nd alignment part (alignment means)
28 storage unit 29 correction area selection unit (selection means)
30 Image reader (acquiring means)
40 Pre-processing unit 42 Connected region extraction unit (extraction means)
44 Center of gravity calculation unit (derivation means)
46 feature amount calculation unit (derivation means)

Claims (15)

Acquisition means for acquiring first image information indicating an image of a first document that is a document before correction and second image information indicating an image of a second document that is the document after correction; ,
Extracting means for extracting a feature region having a predetermined feature in the image of the first document and the image of the second document based on the first image information and the second image information;
Derivation means for deriving a predetermined feature amount from the feature region in the image of the first document and the image of the second document based on the first image information and the second image information;
Specifying means for specifying the feature region in which the similarity of the feature amount is equal to or greater than a predetermined degree between the image of the first document and the image of the second document;
A first feature region which is a feature region extracted from the image of the first document by the extraction unit and which is not included in the feature region specified by the specification unit, and the second document by the extraction unit Selecting means for selecting a second feature area that is a feature area extracted from the image of the image and that is not included in the feature area specified by the specifying means as a correction area that is the corrected area; ,
Determining means for determining a correction state of the document based on the first feature area and the second feature area;
Detecting means for detecting a corrected image according to a determination result by the determining means;
An image processing apparatus. The determination means uses an overlapping area in the first feature area and the second feature area as an overlapping area, and the number of black pixels when the image in the overlapping area in the first feature area is binarized is the first number. When there is a smaller number of black pixels than when the images in the overlap area in the two feature areas are binarized, there is an overlap in which the overlap area is added to the image previously existing in the first document. The image processing apparatus according to claim 1, wherein the image processing apparatus determines that the area is an additional recording area. The detection means performs the subtraction of the first image information from the second image information for each pixel for the area determined to be the additional recording area with the overlap by the determination means. The image processing apparatus according to claim 2. The determining means sets the overlapping area between the first feature area and the second feature area as an overlapping area, and the number of black pixels when the image in the overlapping area in the first feature area is binarized is the first number. Partial erasure in which a part of the image previously existing in the first document is erased when the number of black pixels when the image in the overlap area in the two feature areas is binarized is large. The image processing apparatus according to claim 1, wherein the image processing apparatus is determined to be an area. The detection means erases the part by subtracting the second image information from the first image information for each pixel for the area determined to be the partially erased area by the judgment means. The image processing apparatus according to claim 4, wherein a detected image is detected. The determination means determines that an area that is included in the first feature area and that does not have an overlapping area in the second feature area is an erased area in which an image previously existing in the first document is erased. The image processing apparatus according to any one of claims 1 to 5. The image processing apparatus according to claim 6, wherein the detection unit detects the erased image by extracting information in the area determined to be the erasure area by the determination unit from the first image information. . The determination means includes an area that is included in the second feature area and that does not have an overlapping area in the first feature area, and that is a non-overlapping additional recording area in which no image exists in the first document and is additionally recorded. The image processing device according to claim 1, wherein the image processing device is determined to be present. The image according to claim 8, wherein the detection unit detects the added image by extracting, from the second image information, information in a region determined to be the non-overlapping additional recording region by the determination unit. Processing equipment. The image processing apparatus according to claim 1, wherein the predetermined feature is a black or white connected image in the binarized first image information and second image information. . The deriving means further derives position information indicating the position of the feature region extracted by the extracting means,
The image processing apparatus according to any one of claims 1 to 10, wherein the specifying unit specifies the feature region only for a feature region whose distance indicated by the position information is within a predetermined distance. Alignment means for executing an image alignment process on at least one of the first image information and the second image information only for an area where the first feature area and the second feature area overlap each other; The image processing apparatus according to claim 1, comprising the image processing apparatus. Prior to the alignment processing by the alignment means, alignment processing is performed on the entire image of the first document image and the second document image on at least one of the first image information and the second image information. The image processing apparatus according to claim 12, further comprising pre-alignment means. A calculation unit for calculating a correction coefficient based on a shift amount of the corresponding feature area in the image of the first document and the image of the second document;
The image processing apparatus according to claim 13, wherein the pre-alignment unit performs a correction process on at least one of the first image information and the second image information using the correction coefficient as the alignment process. An acquisition step of acquiring first image information indicating an image of a first document that is a document before correction and second image information indicating an image of a second document that is the document after correction; ,
An extraction step of extracting a feature region having a predetermined feature in the image of the first document and the image of the second document based on the first image information and the second image information;
A derivation step for deriving a predetermined feature amount from the feature region in the image of the first document and the image of the second document based on the first image information and the second image information;
A specifying step of specifying the feature region in which the similarity of the feature amount is a predetermined degree or more between the image of the first document and the image of the second document;
A first feature region that is a feature region extracted from the image of the first document by the extraction step and is not included in the feature region specified by the specification step, and the second document by the extraction step A selection step of selecting a second feature region that is a feature region extracted from the image of the image and that is not included in the feature region specified by the specifying step as a correction region that is the corrected region; ,
A determination step of determining a correction state for the document based on the first feature region and the second feature region;
A detection step of detecting a corrected image according to the determination result of the determination step;
An image processing program for causing a computer to execute.

Images