JP2008206037A - Image processing apparatus and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an image processing apparatus and image processing program capable of easily excluding an abnormal document. <P>SOLUTION: On the basis of pre-postscript image data and post-postscript image data acquired by an image reader 30, an image processing apparatus 20 extracts a feature region including a predetermined feature in an image of each document, then derives a predetermined feature amount from the relevant feature region, and specifies the feature region in which similarity of the feature amount between images of each document is equal to or higher than a predetermined degree. When carrying out alignment processing over the entire image of each document to at least either the pre-postscript image data or the post-postscript image data, on the basis of at least one of the number of extracted feature regions in the images of pre-postscript and post-postscript documents, the number of specified feature regions and a physical quantity indicating position deviation between the images of the pre-postscript and post-postscript documents as a target of the alignment processing, presence/absence of abnormality in at least either the pre-postscript and post-postscript documents is determined. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Conventionally, as a technique for specifying a location to be added to a predetermined document such as a form, Patent Literature 1 and Patent Literature 2 describe that before pre-recording and post-recording image alignment, And the post-recording image data are monochromatized and binarized, and a technique for extracting a postscript location by performing a predetermined calculation between the post-recording image data obtained thereby and the pre-recording image data is disclosed. .

  Japanese Patent Application Laid-Open No. 2003-228688 describes a technique for extracting additional writing locations by storing divided position information in a barcode and adding it to a print document, performing region division and alignment based on the information, and extracting differences. Is disclosed.

  Further, in Patent Document 4, the images before and after the additional recording are divided into a predetermined number, the images are aligned for each divided region, and then the image before the additional recording is expanded to obtain the image data before the additional recording and the image data after the additional recording. A technique for extracting a postscript location by calculating an exclusive OR for each pixel of the above is disclosed.

Further, in Patent Document 5, an image of a document before appending and an image of a document after appending are acquired, both the images are aligned, and both images are divided into grid-like small areas, and a black character is obtained for each divided area. The ratio, the sum of the image densities, and the sum of the densities for each color are compared, thereby identifying the divided areas with additional writing, and realigning (fine-tuning) both images for each divided area with the additional writing. In addition, a technique for extracting a postscript image by subtracting a pre-recording image from a post-recording image in the finely adjusted divided area is disclosed.
JP 2004-080601 A JP 2004-240598 A JP 2003-018393 A JP 2004-341914 A JP 2004-213230 A

  The present invention has been made in the technical background as described above, and an object thereof is to provide an image processing apparatus and an image processing program that can easily exclude an abnormal document.

  In order to achieve the above object, the image processing apparatus according to claim 1, the first image information indicating an image of a first document that is a document before correction is performed, and the document after correction is performed. Acquisition means for acquiring second image information indicating an image of the second document, and based on the first image information and the second image information, the first document image and the second document image in advance Based on extraction means for extracting a feature area having a predetermined feature, and the feature area in the image of the first document and the image of the second document based on the first image information and the second image information. Deriving means for deriving the obtained feature amount, identifying means for identifying the feature region in which the similarity of the feature amount is a predetermined degree or more between the image of the first document and the image of the second document, Based on specific results by specific means A positioning unit that executes a positioning process on the entire image of the first document image and the second document image with respect to at least one of the first image information and the second image information; and the extraction unit The number of feature areas in the image of the first document extracted by the above, the number of feature areas in the image of the second document extracted by the extraction means, the number of feature areas specified by the specifying means, and the position The first document and the second document are based on at least one of physical quantities indicating a positional deviation between the image of the first document and the image of the second document, which is a target of alignment processing by the aligning unit. And determining means for determining the presence or absence of at least one abnormality, and preventing occurrence of a malfunction caused by the abnormality when the determining means determines that there is an abnormality It includes an execution unit for executing predetermined processing as to the.

  According to a second aspect of the present invention, in the first aspect of the invention, the determination unit includes the number of feature regions in the image of the first document, the number of feature regions in the image of the second document, Determining the presence or absence of the abnormality by determining whether the number of feature regions specified by the specifying means and at least one of the physical quantities are within a predetermined range corresponding to each of the physical regions, When the execution means determines that there is an abnormality by the determination means, a process for presenting the presence of the abnormality is executed as the predetermined process.

  According to a third aspect of the present invention, in the first aspect of the present invention, the feature amount of one of the first document image and the second document image derived by the deriving unit is obtained. , Further comprising conversion means for converting the image into respective feature amounts when the image is rotated at three angles of 90 degrees, 180 degrees, and 270 degrees, and the specifying means is derived by the derivation means Identifying the feature region in which the similarity between the feature amount of one image and the feature amounts of the three types of feature amounts obtained by the converting means and the feature amount of the other image is a predetermined degree or more; Whether the abnormality is present or not by determining whether the maximum value of the number of feature regions for each combination of the one image and the other image specified by the specifying unit is equal to or greater than a predetermined number And the execution means When the determination means determines that there is an abnormality, the process of presenting the abnormality is executed as the predetermined process, and when the determination means determines that there is no abnormality, The process for executing the alignment process by the alignment means is executed as the predetermined process.

  According to a fourth aspect of the present invention, in the first aspect of the invention, the determining unit determines whether or not the number of feature regions specified by the specifying unit is a predetermined number or more. The presence / absence of the abnormality is determined, and when the execution unit determines that there is an abnormality by the determination unit, at least one of the image of the first document and the image of the second document is rotated by a predetermined angle. As the state, the processing for executing the processing by the specifying unit and the determination unit again is executed as the predetermined processing, and when it is determined by the determination unit that there is no abnormality, the alignment processing by the alignment unit Is executed as the predetermined process.

  The invention according to claim 5 is the invention according to claim 1, wherein at least one of the first document and the second document includes a plurality of candidate documents, and the determination unit However, at least one of the number of feature areas in the image of the first document, the number of feature areas in the image of the second document, the number of feature areas specified by the specifying unit, and the physical quantity corresponds to each of The presence or absence of the abnormality is determined by determining whether or not it is within a predetermined range, and when the execution unit determines that there is an abnormality by the determination unit, the plurality of originals Applying any of the remaining manuscripts, processing that causes the processing by the acquisition unit, the extraction unit, the derivation unit, the specification unit, and the determination unit to be executed again is the predetermined process. And line, if it is determined that there is no abnormality by the determining means, and executes a process of executing the registration process by the positioning means as a processing said predetermined.

  According to a sixth aspect of the invention, in the invention according to any one of the first to fifth aspects of the invention, the positioning unit is configured to shift a deviation amount between the image of the first original and the image of the second original. A correction coefficient is calculated based on the correction coefficient, and a correction process is performed on at least one of the first image information and the second image information using the correction coefficient as the alignment process, and the physical quantity is the correction value. It is a coefficient.

  Furthermore, in the invention described in claim 7, in the invention described in any one of claims 1 to 6, in place of the number of feature areas specified by the specifying means, A ratio with the number of feature regions in the image of the first document or the image of the second document extracted by the extraction unit is used.

  On the other hand, in order to achieve the above object, the image processing program according to claim 8 includes: first image information indicating an image of a first document that is a document before correction is performed; and after the correction is performed. An acquisition step for acquiring second image information indicating an image of the second document as the document, and an image of the first document and an image of the second document based on the first image information and the second image information An extraction step for extracting a feature region having a predetermined feature in the image, and, based on the first image information and the second image information, from the feature region in the image of the first document and the image of the second document A deriving step for deriving a predetermined feature amount, and a specifying step for identifying the feature region where the similarity of the feature amount is equal to or greater than a predetermined degree between the image of the first document and the image of the second document. The above A position for performing alignment processing on the entire image of the image of the first document and the image of the second document for at least one of the first image information and the second image information based on the identification result of the fixed step A matching step, the number of feature regions in the image of the first document extracted by the extraction step, the number of feature regions in the image of the second document extracted by the extraction step, and the feature specified by the specifying step Based on the number of areas and at least one of the physical quantities indicating the positional deviation between the image of the first document and the image of the second document that are to be subjected to the alignment process in the alignment step, the first A determination step for determining whether or not there is an abnormality in at least one of the first document and the second document, and if there is an abnormality in the determination step It is intended to execute an executing step of executing a predetermined processing as to prevent the occurrence of problems caused by the abnormal if it is constant, to a computer.

  According to the first and eighth aspects of the present invention, it is possible to easily remove abnormal originals compared to the case where the present invention is not applied.

  In addition, according to the second aspect of the present invention, it is possible to easily remove an abnormal document more easily than when the present invention is not applied.

  According to the third aspect of the present invention, when the first image information and the second image information are acquired, the vertical direction between the first document and the second document is 90 degrees, 180 degrees, and 270 degrees. Even when the angle is shifted by any angle, an effect is obtained that the image processing can be executed after the vertical direction is automatically corrected.

  According to the invention of claim 4, even when the vertical direction between the first document and the second document is shifted when the first image information and the second image information are acquired, the above-mentioned is automatically performed. An effect is obtained that image processing can be executed after correcting the vertical direction.

  According to the fifth aspect of the present invention, even when a plurality of candidate originals exist in at least one of the first original and the second original, the correct original is automatically selectively applied to perform image processing. The effect that can be executed is obtained.

  Further, according to the invention described in claim 6, as a physical quantity of the present invention, the correction coefficient used in the alignment processing of the present invention can also be applied, and as a result, an abnormal document can be more easily excluded. The effect of being able to be obtained is obtained.

  Furthermore, according to the seventh aspect of the invention, it is possible to obtain an effect that an abnormal document can be excluded with higher accuracy than when the present invention is not applied.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 shows a configuration example of an image processing system 10 to which the present invention is applied. As shown in the figure, the image processing system 10 according to the present embodiment includes an image processing device 20 such as a personal computer and an image reading device 30 such as a scanner. The image processing device 20 and the image reading device 30 are electrically connected, and the image processing device 20 can acquire image data obtained by reading by the image reading device 30 from the image reading device 30.

  The image processing system 10 according to the present embodiment includes a document before additional writing by handwriting or stamping (hereinafter referred to as “original document before adding”) and a document after the additional writing (hereinafter referred to as “original document”). , “Added document”) is read by the image reading device 30, and based on the two image data obtained by this, the image processing device 20 identifies the location where the additional recording has been performed, and appends from the identified location. The process which detects the image part performed is performed. In the following, image data obtained by reading a document before appending is referred to as “image data before appending”, and image data obtained by reading a document after appending is referred to as “image data after appending”.

  The image reading device 30 according to the present embodiment acquires image data obtained by reading as each pixel value (pixel information) is composed of a plurality of bits (8 bits in the present embodiment). ing. In the image processing system 10 according to the present embodiment, the image reading apparatus 30 separates and reads an image of a document to be read into three primary colors R (red), G (green), and B (blue). A color scanner is applied. The image reading device 30 is not limited to such a multi-bit color scanner, but a multi-bit monochrome scanner (so-called grayscale monochrome scanner), a 1-bit (binary) color scanner. A scanner, a monochrome scanner having a 1-bit configuration, or the like can also be applied.

  Next, with reference to FIG. 2, the main configuration of the electrical system of the image processing apparatus 20 having a particularly important role in the image processing system 10 will be described.

  As shown in the figure, the image processing apparatus 20 according to the present embodiment includes a CPU (central processing unit) 20A that controls the operation of the entire image processing apparatus 20, and a work area when the CPU 20A executes various processing programs. A RAM (Random Access Memory) 20B, a ROM (Read Only Memory) 20C in which various control programs, various parameters, and the like are stored in advance, a hard disk 20D that is a storage means used for storing various information, and various types A keyboard 20E used for inputting information, a display 20F used for displaying various types of information, and an input / output interface (I / F) 20G for exchanging various types of information with external devices are provided. These units are electrically connected to each other by a system bus BUS. Here, the image reading device 30 described above is connected to the input / output interface 20G.

  Accordingly, the CPU 20A accesses the RAM 20B, ROM 20C and hard disk 20D, acquires various input information via the keyboard 20E, displays various information on the display 20F, and various information with the image reading device 30 via the input / output interface 20G. Can be exchanged.

  FIG. 3 is a functional block diagram showing a functional configuration of the image processing apparatus 20 according to the present embodiment.

  As shown in the figure, the image processing apparatus 20 according to the present embodiment includes a first alignment unit 21, a second alignment unit 22, an additional recording detection unit 23, a storage unit 24, and an additional recording area selection. Unit 25, abnormality detection unit 26, and abnormality notification unit 27.

  The first alignment unit 21 has an image before additional recording indicated by the image data before additional recording and an image after additional recording indicated by the image data after additional recording for at least one of the image data before additional recording and the image data after additional recording. Between the positions of the corresponding pixels that occur with respect to the entire image due to the difference in the inclination angle between the original document after the additional recording and the original document after the additional recording, the reading environment, or the like. Alignment processing for correcting is performed. In the first alignment unit 21 according to the present embodiment, based on the pre-addition image data and the post-addition image data, predetermined characteristics (in the present embodiment, 2 (A connected image of black pixels and a connected image of white pixels in the valued image data) is extracted, and before-addition image data and after-addition so that corresponding feature regions of the before-addition image and after-addition image overlap each other. At least one of the image data is converted.

  Here, in the first alignment unit 21 according to the present embodiment, the conversion is performed by performing affine transformation on either one of the pre-addition image data and the post-addition image data. However, the present invention is not limited to this, but the bilinear transformation, the primary conformal transformation, the secondary equiangular transformation, the projective transformation, enlargement / reduction, rotation, translation, etc. for any one of the pre-append image data and the post-append image data A conventionally known conversion method may be applied.

  By the way, in the image alignment by the first alignment unit 21, local images of the pre-recorded image and the post-recorded image generated due to uneven scanning speed, distortion of the optical system, etc. when the image is read by the image reading device 30. It is difficult to correct (align) up to a large distortion.

  Therefore, the second alignment unit 22 targets only the additional region selected from the feature region extracted by the first alignment unit 21 by the additional region selection unit 25 to be described later and the post-addition image data. An alignment process for correcting a shift of the corresponding pixel position between the pre-recording image and the post-recording image is performed on at least one of the image data.

  Note that the second alignment unit 22 according to the present embodiment converts at least one of the image data before additional recording and the image data after additional recording so as to eliminate the positional deviation of the corresponding pixel as much as possible by the pattern matching process.

  Here, in the second alignment unit 22 according to the present embodiment, as the conversion, the correlation coefficient between the additional recording areas corresponding to the image data before additional recording and the image data after additional recording is calculated, and the correlation coefficient is the highest. Applying a transformation that translates at least one of the pre-recording image and post-recording image so as to increase, but not limited to this, affine transformation, bilinear for either pre-recording image data or post-recording image data Other conventionally known conversion methods such as transformation, primary equiangular transformation, secondary equiangular transformation, projective transformation, enlargement / reduction, rotation, translation, etc. may be applied.

  On the other hand, the additional recording detection unit 23 performs an additional recording image based on the pre-recording image data and post-recording image data corresponding to the additional recording region, for which the second alignment unit 22 performed the alignment process on the additional recording region. Information is detected and information indicating the detection result is stored in the storage unit 24.

  In the additional recording detection unit 23 according to the present embodiment, detection of the additional recording image is performed with respect to the pre-recording image data corresponding to the additional recording area with predetermined dots for the image indicated by the pre-addition image data. After performing image expansion processing for a number (for example, 1 dot), the image data before additional recording is subtracted from the image data after additional recording for each corresponding pixel and for each of R, G, and B. However, the present invention is not limited to this, for example, the form that is performed by performing the subtraction without performing the image expansion processing, or the post-recording image data corresponding to the post-recording image data corresponding to the additional recording area as the subtraction. By applying a process of subtracting each of R, G, and B separately for each pixel to be processed, calculating a difference for each corresponding pixel in the binarized pre-addition image data and post-addition image data It may also be in the form of applying such form performing other conventional known write-once detection process.

  Further, the additional recording area selection unit 25 selects the additional recording area that is the feature area where the additional recording has been performed from the feature areas extracted by the first alignment unit 21, and adds additional recording area information indicating the selected additional recording area. The data is output to the second alignment unit 22 and the postscript detection unit 23.

  Further, the abnormality detection unit 26 determines whether or not there is an abnormality in at least one of the original document after appending and the original document after appending based on various information derived by the first alignment unit 21, and indicates the determination result. (Hereinafter referred to as “abnormality determination information”) is output to the abnormality notification unit 27.

  Then, when the abnormality determination information is input from the abnormality detection unit 26, the abnormality notification unit 27 executes a process of presenting the abnormality when the abnormality determination information indicates that there is an abnormality. To do. In addition, in the abnormality notification part 27 which concerns on this Embodiment, although the process which displays the message which shows that there exists abnormality on the display 20F is applied as a process which shows that there exists the said abnormality, it is not restricted to this. For example, other processes that indicate that there is an abnormality, such as a process that notifies that there is an abnormality in another apparatus or program, a process that emits a sound indicating that there is an abnormality by the audio reproduction apparatus, or the like may be applied. it can.

  Next, with reference to FIG. 4, the structure of the 1st position alignment part 21 which concerns on this Embodiment is demonstrated.

  As shown in the figure, the first alignment unit 21 according to the present embodiment includes a pair of feature region extraction units 21A and 21B, an association unit 21C, a correction parameter calculation unit 21D, and an image conversion unit 21E. And.

  The feature region extraction unit 21A receives pre-addition image data, the feature region extraction unit 21B receives post-addition image data, and the feature region extraction units 21A and 21B input image data. Based on the above, a feature region having the above-described predetermined feature (in this embodiment, a connected image of black pixels and a connected image of white pixels) in the pre-recorded image and the post-recorded image is extracted. After that, a predetermined feature amount (in this embodiment, a center of gravity position, a size, the number of pixels, and a handwriting length (see also FIG. 8)), which is determined in advance from the feature region, is based on the input image data. And output as feature area information.

  In addition, the association unit 21C has a feature region in which the similarity of the feature amount indicated by the feature region information output from the feature region extraction units 21A and 21B is greater than or equal to a predetermined degree between the pre-recording image and the post-recording image. Specific feature region information is created and output. The specific feature region information is associated with the specified feature region pair.

  In addition, the correction parameter calculation unit 21D calculates the overall shift between the pre-addition image and the post-addition image based on the shift amount of the corresponding feature region indicated by the specific feature region information output from the association unit 21C. A correction coefficient for eliminating it is calculated. Then, the image conversion unit 21E uses the correction coefficient calculated by the correction parameter calculation unit 21D, and the pre-additional image and the additional recording for the pre-additional image data or the post-additional image data (here, the post-additional image data). Image transformation (here, affine transformation) for performing overall alignment of the subsequent image is executed.

  The correction coefficients applied to the affine transformation are the width direction movement amount, the vertical direction movement amount, the rotation angle, and the magnification. As for the correction coefficient, when conversion based on the correction coefficient is performed on one image (here, the post-recording image), the error of the centroid position of the corresponding feature region of the pre-recording image and the post-recording image becomes the smallest. Calculated as a value. Specifically, for example, by applying the least square method, the movement amount in the width direction and the movement amount in the vertical direction are the correction coefficients that minimize the sum of the errors of the centroid positions of the corresponding feature regions of the pre-addition image and the post-addition image. , Rotation angle, and magnification are obtained. A method for calculating a correction coefficient using the least square method is described in Japanese Patent Laid-Open No. 9-93431 as an example.

  Then, in the additional area selection unit 25 according to the present embodiment, the characteristic area information corresponding to the pre-additional image output from the characteristic area extraction unit 21A and the characteristic corresponding to the post-additional image output from the characteristic area extraction unit 21B. Features not included in the feature region indicated by the specific feature region information among the feature regions extracted by the feature region extraction units 21A and 21B based on the region information and the specific feature region information output from the association unit 21C The area is selected as the additional recording area, which is the area where the additional recording has been performed, and additional recording area information indicating the selected additional recording area is created and output to the second alignment unit 22 and the additional recording detection unit 23.

  In the additional area selection unit 25 according to the present embodiment, the additional area selection is performed on the characteristic area indicated by the specific characteristic area information among the characteristic areas in the post-recording image extracted by the characteristic area extraction unit 21B. This is performed by selecting a feature area that is not included as an additional recording area. However, the present invention is not limited to this, and the feature area indicated by the specific feature area information among the feature areas in the pre-additional image extracted by the feature area extraction unit 21A. It is also possible to adopt a form in which the feature region not included in the item is selected as the additional recording region.

  In the abnormality detection unit 26 according to the present embodiment, the feature region information output from the feature region extraction unit 21A, the feature region information output from the feature region extraction unit 21B, and the specific feature output from the association unit 21C Based on the area information and the correction coefficient output from the correction parameter calculation unit 21D, it is determined whether or not there is an abnormality in at least one of the pre-addition original and the post-addition original, and abnormality determination information indicating the result is notified of abnormality. To the unit 27.

  In the anomaly detection unit 26 according to the present embodiment, the number of feature regions in the pre-addition image indicated by the feature region information output from the feature region extraction unit 21A and the post-addition image output from the feature region extraction unit 21B The number of feature areas indicated by the feature area information and the similarity of the feature quantity indicated by the feature area information output from the feature area extraction units 21A and 21B indicated by the specific feature area information output from the association unit 21C It is determined whether or not at least one of the number of feature region pairs whose A is greater than or equal to a predetermined degree and the correction coefficient output from the correction parameter calculation unit 21D is within a predetermined range corresponding to each of them. Thus, the presence or absence of the abnormality is determined.

  Next, with reference to FIG. 5, the structure of the feature area extraction units 21A and 21B according to the present embodiment will be described. Note that the feature region extraction unit 21A and the feature region extraction unit 21B have the same configuration except that the input image data is different. Therefore, the configuration of the feature region extraction unit 21A will be described here.

  As shown in the figure, the feature region extraction unit 21A according to the present embodiment includes a preprocessing unit 40, a connected region extraction unit 42, a centroid calculation unit 44, and a feature amount calculation unit 46. .

  Note that the pre-processing unit 40 performs monochromatic processing and binarization processing on the input pre-recording image data. Here, the monochromatic process is a process in which the pre-addition image data is converted to pixel data of only luminance information. The binarization process is a process for converting a monochrome image obtained by the monochrome process into a value of 1 (black) or 0 (white) at a predetermined threshold level. Note that when the input image is a monochrome image, the monochromatic processing can be omitted.

  The connected region extracting unit 42 performs a connected region extracting process for extracting a connected region as the feature region on the pre-additional image data that has been binarized by the preprocessing unit 40. In addition, the said connection area | region extraction process can be performed applying the method etc. which are described in Unexamined-Japanese-Patent No. 12-295438 as an example. Here, the connected region (feature region) is extracted as a continuous region (black run) of black pixels in the binarized image or a continuous region (white run) of white pixels that are not black pixels. In the connected region extraction processing, for example, a range (minimum, maximum) of a predetermined size (area or number of pixels) is set in advance as an extraction condition, and a region of continuous identical pixel values in the range is determined in advance. It is executed based on the set conditions.

  Here, with reference to FIG. 6, the specific example of the connection area | region extracted by the said connection area | region extraction process is demonstrated.

  In the figure, a pre-recording image 80 and a post-recording image 90 that have been binarized are shown. From these images, image data having a continuous region of black pixels (black run) and image data having a continuous region of white pixels (white run) are extracted as connected regions under certain conditions. In the example shown in the figure, black character images “A” and “B” and white character images “A” are extracted from the pre-addition image 80 as connected regions 81 to 83, and black characters are extracted from the post-addition image 90. An example in which “A” and “B” as images and “A” as an outline character image are extracted as connected regions 91 to 93 is shown. Note that the number of data in the connected area extracted in the actual processing is extracted in units of hundreds to thousands, although it depends on the design conditions.

  Further, the center of gravity calculation unit 44 calculates the center of gravity of the connected region (feature region) extracted by the connected region extraction unit 42, and the feature amount calculation unit 46 calculates the feature amount (here, the size) of each connected region. , Number of pixels, handwriting length). In the present embodiment, the centroid calculating unit 44 and the feature amount calculating unit 46 are distinguished from each other. However, the feature amount calculating unit 46 may calculate the centroid as one feature amount of the connected region. Needless to say.

  Next, with reference to FIGS. 7 and 8, the center of gravity and the feature amount obtained from the connected region will be described. The character ‘A’ shown in FIG. 7 is, for example, one connected region 70 extracted from the pre-additional image 80. FIG. 8 shows a table made up of information on the center of gravity and feature amounts for the connected regions extracted from the pre-recording image 80 and post-recording image 90.

  First, with reference to FIG. 7, details of the center of gravity and the feature amount of the connection region 70 will be described.

  The centroid 72 is information indicating the position of the centroid of the character A70 as a connected area, and is calculated as, for example, coordinate data (x, y) when the pre-append image 80 is an XY coordinate plane (see also FIG. 8). .

  The size is obtained as ‘L, H’ by the width L and the height H of the circumscribed rectangle 74 that circumscribes the connecting region 70 shown in FIG. 7. Further, the number of pixels is calculated as the total number of pixels of the connection region 70 itself. Further, as shown in FIG. 7, the handwriting length is calculated as the number of constituent pixels in the connection region thinning data 76 obtained by thinning the connection region 70 and converting it as a line having a width of one pixel.

  The center-of-gravity calculation unit 44 and the feature amount calculation unit 46, for each of the connected regions (feature regions) extracted by the connected region extraction unit 42 from the pre-addition image 80 and the post-addition image 90 to be compared, The feature amount information including the size, the number of pixels, the handwriting length, and the like is calculated. As shown in FIG. 8 as an example, the feature region information 50A corresponding to the pre-addition image 80 and the post-addition image 90 are displayed. The corresponding feature area information 50B is organized. As shown in the figure, each connection area extracted from each of the pre-recording image 80 and post-recording image 90 is given a unique ID (Identification), and each connection area is associated with the ID. The information on the center of gravity of the area and the feature amount, that is, the size, the number of pixels, and the handwriting length are recorded as feature area information by a storage unit such as the hard disk 20D.

  Incidentally, the association unit 21C (see FIG. 4) according to the present embodiment, as described above, includes the feature region information output from the feature region extraction units 21A and 21B between the pre-addition image and the post-addition image. A feature region pair whose feature quantity similarity indicated by is equal to or greater than a predetermined degree is specified, and specific feature region information for associating the specified feature region pair is created.

  Here, as an example, the association unit 21C inputs the feature region information 50A recorded by the feature region extraction unit 21A and the feature region information 50B recorded by the feature region extraction unit 21B, and is recorded in both feature region information. Pattern matching processing based on the obtained information is performed, and specific feature region information is created based on the result.

  In addition, in the associating unit 21C according to the present embodiment, the other center where the center of gravity is located within a predetermined distance from the center of gravity of the feature region extracted from one of the pre-recording image and post-recording image. By specifying the feature region of the image based on the barycentric position in each feature region information, the feature regions that are candidates for feature regions having a similarity equal to or higher than a predetermined degree are narrowed down, and only the narrowed feature regions are targeted. The similarity of at least one piece of information of the feature amount (size, number of pixels, handwriting length) in the feature region information is calculated, and information indicating a pair of feature regions whose similarity is equal to or higher than a predetermined degree is used as the specific feature region information. create. In the present embodiment, the reciprocal of the distance between applied feature amounts (for example, Euclidean distance) is applied as the similarity, but the present invention is not limited to this, and each of the pre-append image and post-append image Needless to say, any method can be used as long as it indicates the degree of similarity of the feature amount indicated by the feature region information.

  FIG. 9 schematically shows an example of the data structure of the specific feature area information according to the present embodiment. As shown in the figure, in the associating unit 21C according to the present embodiment, the IDs assigned to the feature regions of the pre-recording image and the post-recording image whose similarity is equal to or higher than a predetermined degree are associated with each other. Specific feature region information is created as a thing.

  Each component of the image processing apparatus 20 configured as described above (the first alignment unit 21, the second alignment unit 22, the additional recording detection unit 23, the additional recording region selection unit 25, the abnormality detection unit 26, and an abnormality notification) The processing by the unit 27) can be realized by a software configuration using a computer by executing a program. In this case, the program includes the image processing program of the present invention. However, the present invention is not limited to realization by a software configuration, and needless to say, it can also be realized by a hardware configuration or a combination of a hardware configuration and a software configuration.

  Hereinafter, a description will be given of a case where the image processing system 10 according to the present embodiment is configured to realize processing by each of the above-described components by executing the above-described program (hereinafter referred to as “image processing program”). To do. In this case, the image processing program is installed in the image processing apparatus 20 in advance, provided in a state stored in a computer-readable recording medium, or distributed via wired or wireless communication means. Forms and the like can be applied.

  Next, the operation of the image processing system 10 according to the present embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing the flow of processing of the image processing program executed by the CPU 20A of the image processing apparatus 20. Further, here, in order to avoid complications, the pre-addition image data and post-addition image data to be processed are input from the image reading device 30 as indicating the same size of the original image, and are previously stored in a predetermined area of the hard disk 20D. The case where it is stored will be described.

  In step 100 of the figure, the preprocessing routine program shown in FIG. 11 is executed. The preprocessing routine program will be described below with reference to FIG.

  In step 200 of the figure, the pre-addition image data to be processed is acquired by reading from the hard disk 20D, and in the next step 202, the pre-addition image is handled by the same process as the process performed by the feature area extraction unit 21A described above. Feature region information (see also FIG. 8A) is derived. For example, if the pre-append image 80 is the one shown in FIG. 12A, in the processing of this step 202, the feature region relating to the feature region shown in FIG. 12B (the connected region in this embodiment). Information will be derived.

  In the next step 204, it is determined whether or not the number of feature regions indicated by the feature region information derived by the processing in step 202 is equal to or greater than a predetermined number (for example, 4). Shifts to step 206, executes a predetermined error notification process E1, and then ends the image processing program. In the pre-processing routine program according to the present embodiment, as the error notification process E1, a process of displaying a message “The feature area is too small to perform alignment” on the display 20F is applied. Needless to say, this is not a limitation.

  On the other hand, if the determination in step 204 is affirmative, the process proceeds to step 208 where the post-recording image data to be processed is acquired by reading from the hard disk 20D. In the next step 210, the above-described feature region extraction unit 21B is acquired. The characteristic area information (see also FIG. 8B) corresponding to the post-recording image is derived by the same process as that described above. For example, when the post-recording image 90 is the one shown in FIG. 12A, in the processing of this step 210, the feature region relating to the feature region shown in FIG. 12B (the connected region in the present embodiment). Information will be derived.

  In the next step 212, it is determined whether or not the number of feature regions indicated by the feature region information derived by the processing in step 210 is equal to or greater than a predetermined number (for example, 4). Shifts to step 214, executes a predetermined error notification process E2, and then ends the image processing program. In the pre-processing routine program according to the present embodiment, as the error notification processing E2, the message “The feature area is too small to perform alignment” is the same as the error notification processing E1. Although the process which displays with the display 20F is applied, it cannot be overemphasized that it is not restricted to this.

  On the other hand, if the determination in step 212 is affirmative, the process proceeds to step 216. Based on the feature region information derived by the above processing, the specific feature region information is obtained by the same processing as the processing by the association unit 21C described above. create. By the process of step 216, specific feature region information (see also FIG. 9) associated with the feature region shown in FIG. 12C as an example is created.

  In the next step 218, whether or not the number of feature areas associated by the process of step 216 indicated by the specific feature area information created by the process of step 216 is a predetermined number (for example, 30) or more. If NO is determined and the determination is NO, the process proceeds to step 220, and after executing a predetermined error notification process E3, the image processing program is terminated. In the preprocessing routine program according to the present embodiment, as the error notification process E3, a process of displaying the message “I am trying to align different documents” on the display 20F is applied. Needless to say, this is not a limitation.

  On the other hand, when an affirmative determination is made in step 218, the process proceeds to step 222, and based on the created specific feature region information, the same process as the process performed by the correction parameter calculation unit 21D described above is performed, and the post-addition image and the post-addition process A correction coefficient for eliminating the overall image shift is calculated, and in the next step 224, the correction coefficient calculated by the process of step 222 (here, the movement amount in the width direction, the movement amount in the vertical direction, the rotation angle, When the negative determination is made (in this case, at least one of the four types of correction coefficients does not fall within the corresponding range). In the case of the above, the process proceeds to step 226, and after executing a predetermined error notification process E4, the image processing program is terminated. In the preprocessing routine program according to the present embodiment, as the predetermined range, a range that is within ± 20 mm is applied to the two types of correction coefficients of the width direction movement amount and the vertical direction movement amount, and the rotation angle is determined. Applies a range that is within ± 3 degrees, and further applies a range that is within ± 5% for the magnification, but it goes without saying that the range is not limited thereto. In the preprocessing routine program according to the present embodiment, as the error notification process E4, a message “A different document is about to be aligned” is displayed on the display 20F as in the error notification process E3. It is needless to say that the processing is applied but is not limited to this.

  On the other hand, if the determination in step 224 is affirmative, the pre-processing routine program is terminated and the process proceeds to step 102 (see FIG. 10) of the image processing program.

  In step 102, using the correction coefficient calculated in the pre-processing routine program, an image for performing overall alignment of the pre-addition image and the post-addition image by the same processing as the processing by the image conversion unit 21E described above. Conversion processing (in this embodiment, affine transformation processing) is executed. Further, in the next step 104, the above-described preprocessing routine program is derived by the same processing as the processing by the additional write area selection unit 25 described above. Of the feature areas, a write-once area that is a feature area to which additional writing has been performed is selected, and additional-write area information indicating the selected feature area (additional write area) is created.

  In the process of step 104, the additional area is selected by selecting an area that is not included in the feature area indicated by the specific feature area information from among the feature areas of the post-record image extracted by the feature area extraction unit 21B. In the case of performing the selection, additional record area information indicating the feature area shown in FIG. 12D is created as an example, and the selection of the additional record area is performed on the pre-addition image extracted by the feature area extraction unit 21A. In the case of performing by selecting a region not included in the feature region indicated by the specific feature region information among the feature regions as the additional recording region, the additional recording region information indicating the characteristic region illustrated in FIG. Will be created.

  In the next step 106, a white image having the width of the image before additional recording (here, the number of pixels in the horizontal direction of the image) and the height of the image before additional recording (here, the number of pixels in the vertical direction of the image) is acquired. The image data shown (hereinafter referred to as “white image data”) is created, and in the next step 108, 0 is substituted into the variable i.

  In the next step 110, as will be described below as an example, any one of the additional write areas indicated by the additional write area information created by the process in step 104 (hereinafter referred to as “process target additional write area”). ) Are derived (hereinafter referred to as “additional area coordinates”).

  First, the information on the center of gravity position and the size corresponding to the processing target additional recording area is derived by the processing of step 202 and is not the target of the image conversion processing by the processing of step 102 (here, before the additional recording) It is acquired from the feature area information of (image).

  Next, the corner point of the rectangular area having the width L and the height H indicated by the acquired size information centered on the center of gravity position indicated by the acquired center of gravity position information (in this embodiment, the upper left corner point and The position coordinates of the lower right corner point) are calculated as the additional recording area coordinates.

  In the next step 112, the additional recording area indicated by the additional recording area coordinates derived by the process of step 110 in the pre-additional image data and post-additional image data (image conversion process executed by the process of step 102) is determined. As an object, the image conversion process for aligning the pre-recording image and the post-recording image is executed in the same manner as the process performed by the second positioning unit 22 described above.

  In the next step 114, based on the respective image data of the pre-recording image and post-recording image corresponding to the post-recording region after the image conversion processing is executed by the processing of step 112, the above-described processing by the postscript detection unit 23 is performed. The image to be added is detected by the same processing, and in the next step 116, the image data obtained by the processing in step 114 (here, the image data before additional after the image expansion processing is used as the corresponding post-addition image data). Then, the image data obtained by subtraction for each corresponding pixel) is superimposed (overwritten) on the white image data created by the processing of step 106 described above.

  In the next step 118, the value of the variable i is incremented by 1. In the next step 120, it is determined whether or not the value of the variable i is equal to or smaller than the number N of additional write areas selected by the process of step 104. If the determination is affirmative, the process returns to step 110, and the processing after step 110 is executed again. On the other hand, if the determination is negative, the process proceeds to step 122. Note that, when the processes of Steps 110 to 120 are repeatedly executed, an additional write area that has not been set as a processing target until then is applied as a processing target additional write area.

  In step 122, the additional image is output using the white image data on which the additional image is superimposed by the above processing, and then the image processing program is terminated. In the image processing program according to the present embodiment, both the output by display on the display 20F and the output by printing by a printer (so-called image forming apparatus) (not shown) are applied as the process of outputting the additional image at step 122. However, the present invention is not limited to this, and it goes without saying that other output forms such as a form in which any one of these is output or a form in which sound is output by a speech synthesizer can be applied. . In the image processing program according to the present embodiment, the case where the detected additional image is recorded by superimposing the additional image on the white image data has been described. However, the present invention is not limited to this. It goes without saying that other forms of recording information that can specify the additional recording image, such as a recording form of the coordinates indicating the position of the additional recording image, can be used.

[Second Embodiment]
The configuration of the image processing system according to the second embodiment is different from the image processing system according to the first embodiment only in the configuration of the first alignment unit 21. First, the second embodiment is described. The configuration of the first alignment unit 21 ′ according to will be described in detail with reference to FIG. In FIG. 13, the same components as those in FIG. 4 are denoted by the same reference numerals as those in FIG.

  As shown in the figure, the first alignment unit 21 ′ according to the second embodiment is different from the corresponding unit 21C in that a 90-degree rotation unit 21F and a selection unit 21G are newly provided. The only difference from the first alignment unit 21 according to the first embodiment is that the corresponding portion 21C ′ is used.

  The 90-degree rotation unit 21F uses the feature amount indicated by the feature region information of the post-recording image derived by the feature region extraction unit 21B, and the post-recording image has three types of 90 degrees, 180 degrees, and 270 degrees. The characteristic area information obtained by this conversion is output respectively.

  As shown in FIG. 8 as an example, the feature region information according to the present embodiment represents the position of the feature region by the coordinate value in the XY coordinate system of the barycentric position. In the 90-degree rotation unit 21F, The above-described conversion is performed by converting the coordinate value into a coordinate value when the post-recording image is rotated at three angles of 90 degrees, 180 degrees, and 270 degrees.

  As described above, in the first alignment unit 21 ′ according to the present embodiment, the 90-degree rotation unit 21F is connected to the feature region extraction unit 21B to create feature region information corresponding to the post-addition image. However, the present invention is not limited to this, and the feature region information corresponding to the pre-addition image may be created by being connected to the feature region extraction unit 21A.

  On the other hand, the association unit 21C ′ is indicated by the feature amount indicated by the feature region information output from the feature region extraction unit 21A and the feature region information output from the feature region extraction unit 21B and the 90-degree rotation unit 21F. A feature region in which a similarity between a feature amount (a feature amount when the post-recording image is rotated at three angles of 90 degrees, 180 degrees, and 270 degrees and a feature amount when the image is not rotated) is a predetermined degree or more Specific feature region information is created and output. The specific feature region information is associated with the specified feature region pair. Thus, as an example, the specific feature area information as shown in FIG. 9 includes four sets (combination of the image before appending and the image after appending not rotated, the combination of the image before appending and the image after appending rotated 90 degrees, The combination of the pre-recording image and the post-recording image rotated by 180 degrees, and the combination of the pre-recording image and the post-recording image rotated by 270 degrees) are generated and output.

  Further, the selection unit 21G selects a correction parameter calculation unit 21D, a write-once region, and selects the one with the largest number of associated feature regions from the four sets of specific feature region information output from the association unit 21C ′. While outputting to the selection part 25 and the abnormality detection part 26, the feature area information of the post-recording image matched in specific feature area information with the largest number of the said feature area is selected, and the additional recording area selection part 25 and abnormality Output to the detector 26.

  Next, a case will be described in which the operation of the image processing system 10 according to the second embodiment is realized by a software configuration by executing an image processing program for the processing by each of the above constituent elements. The image processing program according to the second embodiment is the same as that according to the first embodiment except for the preprocessing routine program, and therefore, hereinafter, referring to FIG. Only the preprocessing routine program according to the second embodiment will be described.

  In step 300 of the figure, the pre-addition image data to be processed is acquired by reading from the hard disk 20D, and in the next step 302, the pre-addition image is handled by the same processing as the processing by the feature region extraction unit 21A described above. Feature region information (see also FIG. 8A) is derived. For example, if the pre-append image 80 is the one shown in FIG. 12A, in the process of this step 302, the feature region relating to the feature region shown in FIG. 12B (in this embodiment, the connected region). Information will be derived.

  In the next step 304, it is determined whether or not the number of feature regions indicated by the feature region information derived by the processing in step 302 is equal to or greater than a predetermined number (for example, 4). Shifts to step 306, executes a predetermined error notification process E1, and then ends the image processing program. In the pre-processing routine program according to the present embodiment, as the error notification process E1, a process of displaying a message “The feature area is too small to perform alignment” on the display 20F is applied. Needless to say, this is not a limitation.

  On the other hand, if the determination in step 304 is affirmative, the process proceeds to step 308, where post-append image data to be processed is acquired by reading from the hard disk 20D. In the next step 310, the above-described feature region extraction unit 21B is acquired. The characteristic area information (see also FIG. 8B) corresponding to the post-recording image is derived by the same process as that described above. For example, when the post-recording image 90 is the one shown in FIG. 12A, in the process of this step 310, the feature region related to the feature region shown in FIG. 12B (the connected region in this embodiment). Information will be derived.

  In the next step 312, it is determined whether or not the number of feature regions indicated by the feature region information derived by the processing in step 310 is equal to or greater than a predetermined number (for example, 4). Shifts to step 314, executes a predetermined error notification process E2, and then ends the image processing program. In the pre-processing routine program according to the present embodiment, as the error notification processing E2, the message “The feature area is too small to perform alignment” is the same as the error notification processing E1. Although the process which displays with the display 20F is applied, it cannot be overemphasized that it is not restricted to this.

  On the other hand, if the determination in step 312 is affirmative, the process proceeds to step 316, and based on the feature region information derived by the above processing, the specific feature region information is processed by the same processing as the processing by the association unit 21C ′ described above. J1 is created and stored in the RAM 20B. By the process of step 316, specific feature region information (see also FIG. 9) associated with the feature region shown in FIG. 12C as an example is created and stored.

  In the next step 318, the feature amount indicated by the feature region information of the post-recording image derived by the feature region extraction unit 21B is converted into the post-recording image by the same processing as the processing by the 90-degree rotation unit 21F described above. In the next step 320, the feature area information obtained by the process of step 302 and the feature area obtained by the process of step 318 are converted. Based on the information, the specific feature region information J2 is created by the same processing as the processing by the association unit 21C ′ described above, and stored in the RAM 20B.

  In the next step 322, the feature amount indicated by the feature region information of the post-recorded image in the rotated state derived by the process of step 318 is derived by the same process as the process by the 90-degree rotation unit 21F described above. Then, the post-recording image is converted so as to become a feature amount when the image is further rotated by 90 degrees (a feature amount when the original post-recording image is rotated by 180 degrees), and in the next step 324, the above step 302 is performed. Based on the feature region information obtained by the processing of step 322 and the feature region information obtained by the processing of step 322, specific feature region information J3 is created by the same processing as the processing by the association unit 21C ′ described above, Store in the RAM 20B.

  In the next step 326, the feature amount indicated by the feature area information of the post-recorded image in the state rotated by 180 degrees derived by the process of step 322 is obtained by the same process as the process by the 90-degree rotation unit 21F described above. Then, the post-recording image is converted so as to become a feature amount when the image is further rotated by 90 degrees (a feature amount when the original post-recording image is rotated by 270 degrees). Based on the feature region information obtained by the processing of step 326 and the feature region information obtained by the processing of step 326, specific feature region information J4 is created by the same processing as the processing by the association unit 21C ′ described above, Store in the RAM 20B.

  In the next step 330, the specific feature region information J1 to J4 stored in the RAM 20B is read out by the above processing, and the one with the maximum number of feature regions associated with each specific feature region information is selected and the number thereof. It is determined whether or not the ratio of the number of feature regions extracted by the feature region extraction unit 21A is equal to or higher than a predetermined rate (for example, 60%). If the determination is negative, the process proceeds to step 332. Then, after executing the predetermined error notification process E3, the image processing program is terminated. In the preprocessing routine program according to the present embodiment, as the error notification process E3, a process of displaying the message “I am trying to align different documents” on the display 20F is applied. Needless to say, this is not a limitation.

  On the other hand, if the determination in step 330 is affirmative, the process proceeds to step 334, and the correction described above is performed based on the specific feature region information in which the number of feature regions associated in the processing of step 330 is maximized. A correction coefficient for eliminating an overall shift between the pre-recording image and the post-recording image is calculated by a process similar to the process by the parameter calculation unit 21D. In the next step 336, the correction calculated by the process of step 334 is performed. When the coefficients (here, the four types of correction coefficients of the movement amount in the width direction, the movement amount in the vertical direction, the rotation angle, and the magnification) are within a predetermined range, and when the negative determination is made (here If at least one of the four types of correction coefficients does not fall within the corresponding range), the process proceeds to step 338, and a predetermined error notification process E is performed. After executing, it terminates the image processing program. In the preprocessing routine program according to the present embodiment, as the predetermined range, a range that is within ± 20 mm is applied to the two types of correction coefficients of the width direction movement amount and the vertical direction movement amount, and the rotation angle is determined. Applies a range that is within ± 3 degrees, and further applies a range that is within ± 5% for the magnification, but it goes without saying that the range is not limited thereto. In the preprocessing routine program according to the present embodiment, as the error notification process E4, a message “A different document is about to be aligned” is displayed on the display 20F as in the error notification process E3. It is needless to say that the processing is applied but is not limited to this.

  On the other hand, if the determination in step 336 is affirmative, the pre-processing routine program is terminated, and the process proceeds to step 102 (see FIG. 10) of the image processing program.

[Third Embodiment]
In the third embodiment, a modification of the second embodiment will be described. The configuration of the image processing system 10 according to the third embodiment is substantially the same as that according to the second embodiment, and a description thereof will be omitted here.

  Hereinafter, the case where the operation of the image processing system 10 according to the third embodiment is realized by a software configuration by executing an image processing program for the processing by each component will be described. Note that the image processing program according to the third embodiment is the same as that according to the first and second embodiments except for the preprocessing routine program, and therefore, refer to FIG. 15 below. Only the preprocessing routine program according to the third embodiment will be described.

  In step 400 of the figure, the pre-addition image data to be processed is acquired by reading from the hard disk 20D, and in the next step 402, the pre-addition image is handled by the same process as the process performed by the feature area extraction unit 21A described above. Feature region information (see also FIG. 8A) is derived. For example, if the pre-append image 80 is as shown in FIG. 12A, in the process of this step 402, the feature area related to the feature area shown in FIG. 12B (in this embodiment, a connected area). Information will be derived.

  In the next step 404, it is determined whether or not the number of feature regions indicated by the feature region information derived by the processing in step 402 is equal to or greater than a predetermined number (for example, 4). Shifts to step 406, executes a predetermined error notification process E1, and then ends the image processing program. In the pre-processing routine program according to the present embodiment, as the error notification process E1, a process of displaying a message “The feature area is too small to perform alignment” on the display 20F is applied. Needless to say, this is not a limitation.

  On the other hand, if the determination in step 404 is affirmative, the process proceeds to step 408, where post-append image data to be processed is acquired by reading from the hard disk 20D. In the next step 410, the above-described feature region extraction unit 21B is acquired. The characteristic area information (see also FIG. 8B) corresponding to the post-recording image is derived by the same process as that described above. For example, when the post-recording image 90 is the one shown in FIG. 12A, in the process of this step 410, the feature region relating to the feature region shown in FIG. 12B (in this embodiment, the connected region). Information will be derived.

  In the next step 412, it is determined whether or not the number of feature regions indicated by the feature region information derived by the processing in step 410 is greater than or equal to a predetermined number (for example, 4). Shifts to step 414, executes a predetermined error notification process E2, and then ends the image processing program. In the pre-processing routine program according to the present embodiment, as the error notification processing E2, the message “The feature area is too small to perform alignment” is the same as the error notification processing E1. Although the process which displays with the display 20F is applied, it cannot be overemphasized that it is not restricted to this.

  On the other hand, if the determination in step 412 is affirmative, the process proceeds to step 416, and based on the feature region information derived by the above processing, the specific feature region information is processed by the same processing as the processing by the association unit 21C ′ described above. Is stored in the RAM 20B. By the processing of step 416, specific feature region information (see also FIG. 9) associated with the feature region shown in FIG. 12C as an example is created and stored.

  In the next step 418, the ratio of the number of feature regions associated in the specific feature region information created by the processing in step 416 to the number of feature regions extracted by the feature region extraction unit 21A is a predetermined rate. (For example, 60%) or not is determined, and if a negative determination is made, the process proceeds to step 420.

  In step 420, the feature amount indicated by the feature region information of the post-recording image derived by the feature region extraction unit 21B is obtained by performing the same process as the process performed by the 90-degree rotation unit 21F described above, and the post-recording image is displayed by 90 degrees. In the next step 422, it is determined whether or not the rotation angle of the postscript image after the process of step 420 has reached 360 degrees, and a negative determination is made. If YES, the process returns to step 416. If YES, the process proceeds to step 424.

  When repeatedly executing the processing from step 416 to step 422, in step 416, the specific region information is created by applying the feature region information obtained by the previous step 420.

  In step 424, after executing a predetermined error notification process E3, the image processing program is terminated. In the preprocessing routine program according to the present embodiment, as the error notification process E3, a process of displaying the message “I am trying to align different documents” on the display 20F is applied. Needless to say, this is not a limitation.

  On the other hand, if the determination in step 418 is affirmative, the process proceeds to step 426, and based on the specific feature region information created in the immediately preceding step 416, the same processing as that performed by the correction parameter calculation unit 21D described above. Through the process, a correction coefficient for eliminating the overall deviation between the pre-recording image and the post-recording image is calculated. In the next step 428, the correction coefficient (here, the movement amount in the width direction) calculated by the process of step 426 is calculated. When the negative determination is made (in this case, at least one of the above four types of correction), it is determined whether each of the four types of correction coefficients (vertical movement amount, rotation angle, and magnification) is within a predetermined range. If the coefficient does not fall within the corresponding range), the process proceeds to step 430, and after executing a predetermined error notification process E4, the image process To end the program. In the preprocessing routine program according to the present embodiment, as the predetermined range, a range that is within ± 20 mm is applied to the two types of correction coefficients of the width direction movement amount and the vertical direction movement amount, and the rotation angle is determined. Applies a range that is within ± 3 degrees, and further applies a range that is within ± 5% for the magnification, but it goes without saying that the range is not limited thereto. In the preprocessing routine program according to the present embodiment, as the error notification process E4, a message “A different document is about to be aligned” is displayed on the display 20F as in the error notification process E3. It is needless to say that the processing is applied but is not limited to this.

  On the other hand, if the determination in step 428 is affirmative, the pre-processing routine program is terminated and the process proceeds to step 102 (see FIG. 10) of the image processing program.

[Fourth Embodiment]
In the fourth embodiment, when at least one of the original before appending and the original after appending includes a plurality of candidate originals, a correct one is selected from the candidate originals and the postscript image is added. An example in the case of performing image processing for extracting the image will be described. Here, in order to avoid complications, a description will be given of an example in which a plurality of candidate documents exist only in the document before the additional writing. The configuration of the image processing system 10 according to the fourth embodiment is substantially the same as that according to the first embodiment, and a description thereof will be omitted here.

  Hereinafter, a case will be described in which the operation of the image processing system 10 according to the fourth embodiment is realized by a software configuration by executing an image processing program. The image processing program according to the fourth embodiment is the same as that according to the first to third embodiments except for the pre-processing routine program, and therefore, refer to FIG. 16 below. Only the preprocessing routine program according to the fourth embodiment will be described. Also, here, in order to avoid complications, a plurality of pre-additional image data indicating different pre-addition originals (processing candidate originals) to be processed and a single post-addition original to be processed A case will be described in which image data is input from the image reading device 30 as indicating an original document image of the same size and stored in advance in a predetermined area of the hard disk 20D.

  In step 500 of the figure, post-recording image data to be processed is acquired by reading from the hard disk 20D, and in the next step 502, the post-recording image is processed by the same process as the process performed by the feature region extraction unit 21B described above. Feature region information (see also FIG. 8B) is derived. For example, when the post-recording image 90 is the one shown in FIG. 12A, in the processing of this step 502, the feature region related to the feature region shown in FIG. 12B (the connected region in this embodiment). Information will be derived.

  In the next step 504, it is determined whether or not the number of feature regions indicated by the feature region information derived by the processing in step 502 is equal to or greater than a predetermined number (for example, 4). The process proceeds to step 506, executes a predetermined error notification process E2, and then ends the image processing program. In the pre-processing routine program according to the present embodiment, as the error notification process E2, a process of displaying a message “The feature area is too small to perform alignment” on the display 20F is applied. Needless to say, this is not a limitation.

  On the other hand, if an affirmative determination is made in step 504, the process proceeds to step 508, and one of the plurality of pre-additional image data to be processed (hereinafter referred to as “pre-processing target image data”). Is read out from the hard disk 20D, and in the next step 510, feature area information corresponding to the image before the additional recording (FIG. See also A). For example, if the pre-append image 80 is the one shown in FIG. 12A, in the processing of this step 510, the feature region relating to the feature region shown in FIG. 12B (in this embodiment, the connected region). Information will be derived.

  In the next step 512, it is determined whether or not the number of feature regions indicated by the feature region information derived by the processing in step 510 is equal to or greater than a predetermined number (for example, 4). Shifts to step 522 to be described later, and shifts to step 514 if an affirmative determination is made.

  In step 514, based on the feature region information derived by the above processing, specific feature region information is created by the same processing as the processing by the association unit 21C described above, and stored in the RAM 20B. By the process of step 514, specific feature region information (see also FIG. 9) associated with the feature region shown in FIG. 12C as an example is created and stored.

  In the next step 516, the ratio of the number of feature regions associated in the specific feature region information created by the processing in step 514 to the number of feature regions extracted by the feature region extraction unit 21A is a predetermined rate. (For example, 60%) is determined. If the determination is negative, the process proceeds to step 522 to be described later, whereas if the determination is affirmative, the process proceeds to step 518.

  In step 518, based on the specific feature area information created in the process of step 514, the overall shift between the pre-recording image and the post-recording image is eliminated by the same process as the process performed by the correction parameter calculation unit 21D described above. In the next step 520, the correction coefficients calculated by the processing in step 518 (here, four types of correction coefficients including the width direction movement amount, the vertical direction movement amount, the rotation angle, and the magnification) are calculated. ) Is determined to be within a predetermined range, and if a negative determination is made (in this case, at least one of the four types of correction coefficients is not within the corresponding range), the process proceeds to step 522. To do. In the preprocessing routine program according to the present embodiment, as the predetermined range, a range that is within ± 20 mm is applied to the two types of correction coefficients of the width direction movement amount and the vertical direction movement amount, and the rotation angle is determined. Applies a range that is within ± 3 degrees, and further applies a range that is within ± 5% for the magnification, but it goes without saying that the range is not limited thereto.

  In step 522, it is determined whether or not pre-additional image data that has not yet been processed since this pre-processing routine program has been executed is stored in a predetermined area of the hard disk 20D. On the other hand, if the determination is negative, the process proceeds to step 524. When the processes in steps 508 to 522 are repeatedly executed, in step 508, the pre-additional image data that has not been read is read from the hard disk 20D.

  In step 524, after executing a predetermined error notification process E5, the image processing program is terminated. In the preprocessing routine program according to the present embodiment, as the error notification process E5, a process of displaying a message “There is no pre-additional image corresponding to the candidate” on the display 20F is applied. However, it goes without saying that this is not a limitation.

  On the other hand, if the determination in step 520 is affirmative, the preprocessing routine program is terminated and the process proceeds to step 102 (see FIG. 10) of the image processing program.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various modifications or improvements can be added to the above-described embodiment without departing from the gist of the invention, and embodiments to which such modifications or improvements are added are also included in the technical scope of the present invention.

  The above embodiments do not limit the invention according to the claims (claims), and all the combinations of features described in the embodiments are essential for the solution of the invention. Is not limited. The above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. Even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, as long as an effect is obtained, a configuration from which these some constituent requirements are deleted can be extracted as an invention.

  For example, although cases have been described with the above embodiments where image processing is realized by a software configuration, the present invention is not limited to this, and for example, the image processing is realized by a hardware configuration. You can also. As a form example in this case, for example, each component of the image processing apparatus 20 shown in FIG. 3 (first alignment unit 21 (21 ′), second alignment unit 22, additional recording detection unit 23, additional recording area selection) The form which produces and uses the functional device which performs the process same as the part 25 and the abnormality detection part 26) can be illustrated. In this case, the image processing can be speeded up as compared with the above embodiments.

  In each of the above embodiments, the case where the present invention is applied to an image processing system that performs image processing for extracting a postscript image for a predetermined document has been described. However, the present invention is not limited to this, and for example, The present invention can also be applied to an image processing system that performs other image processing using a plurality of document images, such as an image processing system that performs image processing that extracts an erased image in addition to a postscript image for a predetermined document. Also in this case, the same effects as those of the above embodiments can be obtained.

  In the preprocessing routine program of the first embodiment, it is determined whether or not the number of feature regions associated with the specific feature region information is a predetermined number or more. In the pre-processing routine programs of the second to fourth embodiments, the ratio of the number of feature regions associated with the specific feature region information to the number of feature regions extracted by the feature region extraction unit is a predetermined rate or more. Although the case where it is determined whether or not there has been described, the present invention is not limited to this, and the preprocessing routine program of the first embodiment described above is the same as that of the second to fourth embodiments. The ratio of the preprocessing routine program may be used. Conversely, in the preprocessing routine program of the second to fourth embodiments, the preprocessing routine program of the first embodiment is used. The number of feature areas of may be in the form used.

  Further, in each of the above embodiments, as a physical quantity indicating the positional deviation of the present invention, image conversion for performing overall alignment of the pre-recording image and the post-recording image (affine transformation in each of the above embodiments) is performed. However, the present invention is not limited to this and the present invention is not limited to this. It is also possible to adopt a form in which another physical quantity indicating a positional deviation from the image is applied. Also in this case, the same effects as those of the above embodiments can be obtained.

  In each of the above-described embodiments, the case has been described where notification by displaying a message indicating the content of an abnormality on the display 20F is applied as an abnormal state notification method by the abnormality notification unit, but the present invention is not limited thereto. For example, other notification methods such as notification by printing the message by the image forming apparatus, notification by sounding the message by the audio reproduction apparatus, notification to a higher-level program, etc. You can also Also in this case, the same effects as those of the above embodiments can be obtained.

  In addition, the configurations (see FIGS. 1 to 5 and 13) of the image processing system 10 and the image processing apparatus 20 described in the above embodiments are merely examples, and are not necessary within the scope of the present invention. Needless to say, you can delete parts or add new ones.

  The data structures of various information described in the above embodiments (see FIGS. 8 and 9) are also examples, and it goes without saying that changes can be made without departing from the gist of the present invention.

  Furthermore, the processing flow of the image processing program and the preprocessing routine program described in the above embodiments (see FIGS. 10, 11, and 14 to 16) is an example, and does not depart from the gist of the present invention. It goes without saying that unnecessary steps can be deleted, new steps can be added, and the processing order can be changed within the range.

1 is a schematic diagram illustrating a configuration of an image processing system according to an embodiment. It is a block diagram which shows the principal part structure of the electric system of the image processing apparatus which concerns on embodiment. It is a functional block diagram which shows the functional structure of the image processing apparatus which concerns on embodiment. It is a block diagram which shows the structure of the 1st position alignment part which concerns on the 1st, 4th embodiment among the functional structures of the image processing apparatus which concerns on embodiment. It is a block diagram which shows the structure of a characteristic area extraction part among the functional structures of the image processing apparatus which concerns on embodiment. It is a figure where it uses for description of the connection area | region (characteristic area | region) which concerns on embodiment. It is a figure with which it uses for description of each information of the gravity center calculated | required from the connection area | region which concerns on embodiment, and a feature-value. It is a schematic diagram which shows the data structure of the characteristic area information which concerns on embodiment. It is a schematic diagram which shows the data structure of the specific feature area information which concerns on embodiment. It is a flowchart which shows the flow of a process of the image processing program which concerns on embodiment. It is a flowchart which shows the flow of a process of the pre-processing routine program which concerns on 1st Embodiment. It is a figure where it uses for description of the processing content of the image processing program which concerns on embodiment. It is a block diagram which shows the structure of the 1st position alignment part which concerns on 2nd, 3rd embodiment among the functional structures of the image processing apparatus which concerns on embodiment. It is a flowchart which shows the flow of a process of the pre-processing routine program which concerns on 2nd Embodiment. It is a flowchart which shows the flow of a process of the pre-processing routine program which concerns on 3rd Embodiment. It is a flowchart which shows the flow of a process of the pre-processing routine program which concerns on 4th Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing system 20 Image processing apparatus 20A CPU
20D hard disk 20F display 21, 21 ′ first alignment unit 21A feature region extraction unit 21B feature region extraction unit 21C, 21C ′ corresponding unit (specifying means)
21D correction parameter calculation unit 21E image conversion unit (positioning means)
21F 90 degree rotating part (conversion means)
21G selection part 22 2nd position alignment part 23 Additional recording detection part 24 Storage part 25 Additional recording area selection part 26 Abnormality detection part (determination means)
27 Abnormality notification unit (execution means)
30 Image reader (acquiring means)
40 Pre-processing unit 42 Connected region extraction unit (extraction means)
44 Center of gravity calculation unit (derivation means)
46 feature amount calculation unit (derivation means)

Claims (8)

Acquisition means for acquiring first image information indicating an image of a first document that is a document before correction and second image information indicating an image of a second document that is the document after correction; ,
Extracting means for extracting a feature region having a predetermined feature in the image of the first document and the image of the second document based on the first image information and the second image information;
Derivation means for deriving a predetermined feature amount from the feature region in the image of the first document and the image of the second document based on the first image information and the second image information;
Specifying means for specifying the feature region in which the similarity of the feature amount is equal to or greater than a predetermined degree between the image of the first document and the image of the second document;
Based on the result of specification by the specifying means, alignment processing is performed on the entire image of the first document image and the second document image for at least one of the first image information and the second image information. Alignment means;
The number of feature areas in the image of the first document extracted by the extraction means, the number of feature areas in the image of the second document extracted by the extraction means, the number of feature areas specified by the specification means, And the at least one physical quantity indicating a positional deviation between the image of the first document and the image of the second document to be subjected to the alignment process by the alignment unit, and the first document and the Determination means for determining the presence / absence of at least one abnormality of the second document;
An execution means for executing a process predetermined as preventing the occurrence of a malfunction caused by the abnormality when the determination means determines that there is an abnormality;
An image processing apparatus. The determination means includes at least one of the number of feature areas in the image of the first document, the number of feature areas in the image of the second document, the number of feature areas specified by the specifying means, and the physical quantity, respectively. Determining the presence or absence of the abnormality by determining whether or not it is within a predetermined range as corresponding to
The image processing apparatus according to claim 1, wherein when the determination unit determines that there is an abnormality, the execution unit executes a process of presenting the abnormality as the predetermined process. The feature amount of one of the image of the first document and the image of the second document derived by the deriving unit is rotated at three angles of 90 degrees, 180 degrees, and 270 degrees. Conversion means for converting each feature amount in the case of
The specifying means is a degree of similarity between the feature quantity of the one image derived by the derivation means and the feature quantities of the three types of feature quantities obtained by the conversion means, and the feature quantity of the other image. Identifying the feature region that is greater than or equal to a predetermined degree,
The determination means determines whether or not the abnormality exists by determining whether or not a maximum value of the number of feature regions for each combination of the one image and the other image specified by the specification means is a predetermined number or more. Determine
The execution means executes the process of presenting the presence of the abnormality as the predetermined process when the determination means determines that there is an abnormality, and the determination means determines that there is no abnormality The image processing apparatus according to claim 1, wherein a process for executing an alignment process by the alignment unit is executed as the predetermined process. The determination means determines the presence or absence of the abnormality by determining whether or not the number of feature regions specified by the specifying means is a predetermined number or more,
When the determination unit determines that there is an abnormality, the execution unit sets the at least one of the image of the first document and the image of the second document as being rotated by a predetermined angle, and again specifies the specification unit and The process for executing the process by the determination unit is executed as the predetermined process, and the process for executing the alignment process by the alignment unit when the determination unit determines that there is no abnormality is the predetermined process. The image processing apparatus according to claim 1, wherein the image processing apparatus executes the processed process. At least one of the first document and the second document has a plurality of candidate documents,
The determination means includes at least one of the number of feature areas in the image of the first document, the number of feature areas in the image of the second document, the number of feature areas specified by the specifying means, and the physical quantity, respectively. Determining the presence or absence of the abnormality by determining whether or not it is within a predetermined range as corresponding to
The execution means applies any one of the remaining originals of the plurality of originals when the determination means determines that there is an abnormality, and the acquisition means, the extraction means, the derivation means, and the specification means And a process for executing the process by the determination unit again as the predetermined process, and a process for executing the alignment process by the alignment unit when the determination unit determines that there is no abnormality. The image processing apparatus according to claim 1, wherein the image processing apparatus is executed as the predetermined process. The alignment unit calculates a correction coefficient based on a shift amount between the image of the first document and the image of the second document, and the first image information and the first image using the correction coefficient as the alignment process. Correction processing is performed on at least one of the two image information,
The image processing apparatus according to claim 1, wherein the physical quantity is the correction coefficient. Instead of the number of feature areas specified by the specifying means, a ratio between the number of the specific areas and the number of feature areas in the image of the first document or the image of the second document extracted by the extraction means. The image processing apparatus according to any one of claims 1 to 6. An acquisition step of acquiring first image information indicating an image of a first document that is a document before correction and second image information indicating an image of a second document that is the document after correction; ,
An extraction step of extracting a feature region having a predetermined feature in the image of the first document and the image of the second document based on the first image information and the second image information;
A derivation step for deriving a predetermined feature amount from the feature region in the image of the first document and the image of the second document based on the first image information and the second image information;
A specifying step of specifying the feature region in which the similarity of the feature amount is a predetermined degree or more between the image of the first document and the image of the second document;
Based on the identification result of the identifying step, alignment processing is performed on at least one of the first image information and the second image information in the entire image of the first document image and the second document image. An alignment step;
The number of feature regions in the image of the first document extracted by the extraction step, the number of feature regions in the image of the second document extracted by the extraction step, the number of feature regions specified by the specification step, And based on at least one of physical quantities indicating a positional deviation between the image of the first document and the image of the second document to be subjected to the alignment process in the alignment step. A determination step of determining whether or not there is an abnormality in at least one of the second originals;
An execution step of executing a process predetermined as preventing occurrence of a malfunction caused by the abnormality when it is determined that there is an abnormality in the determination step;
An image processing program for causing a computer to execute.

Images