JP4120795B2 - Retouching extraction device and retouching extraction method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a retouching extraction apparatus and a retouching extraction method, and more particularly to a retouching extraction apparatus and a retouching extraction method based on image differences for improving retouching extraction accuracy for a handwriting input system.
[0002]
[Prior art]
Conventionally, even if the user has revised an idea or the like that has come up with the contents of the printer output document on the paper, in order to reflect it on the original electronic file document, the user must revisit the amendment at a later date. It was necessary to input using word processor software.
[0003]
In order to solve such inconvenience, the correction correction content for the paper document is used as image information by subtracting the image before correction correction created by the printer driver from the correction superimposed image input by the image input means such as a scanner. A technique for extracting has been developed (see, for example, Patent Document 1).
[0004]
However, even with such a technique, there has been a problem that, for example, information already determined in a fixed format is erroneously corrected, and further, the contents of the document itself are intentionally altered.
[0005]
Therefore, it is possible to detect four characteristic points in the image, such as the corners of the outer frame, and perform interpolation processing so that each point moves to the corresponding point in the electronic image. There is a method in which a difference process is performed after alignment by a method of performing a correction process by an affine transformation formula that performs translation correction by performing parallel movement and rotation around that point (for example, see Patent Document 2). .
[0006]
Furthermore, a means for creating hypertext, a means for storing a related information file, a means for creating media paper information from hypertext, a means for outputting media paper, a means for giving selection information, and a media paper There is also a document information management system provided with means for reading selection information and connection information from, a means for searching for the relevant information file, and a means for outputting the searched relevant information file (see, for example, Patent Document 3). . This technology incorporates paper documents into the digital world document information management system by building continuity and relationship of information between the digital world such as computers and paper documents. It is intended to enable direct access to the digital world using a document as a medium, and to realize hypertext (paper hypertext) using a paper document.
[0007]
[Patent Document 1]
JP 11-272654 A
[Patent Document 2]
JP 2001-283231 A
[Patent Document 3]
Japanese Patent Laid-Open No. 10-228468
[0008]
[Problems to be solved by the invention]
  However, in the invention described in Patent Document 1, the electronic document image created by the printer driver when extracting the correction correction content and the image,deviceThis is extracted by performing a simple difference process between the retouched superimposed image input from, but this method causes a total or local shift between the electronic manuscript and the retouched superimposed image. In addition, after-difference processing occurs as a result of the deviation. In addition, when there is an addition on a printed frame, character, or the like, the added information may be lost due to the difference processing.
[0009]
In addition, the invention described in Patent Document 2 cannot cope with local magnification fluctuations of an added image due to paper feed unevenness during printer, drum rotation unevenness, optical system speed unevenness during scanning, and the like. In this prior art, no countermeasure is taken against the disappearance of the writing information due to the excessive removal during the difference processing.
[0010]
Therefore, in the present invention, the addition of characters on a document is targeted, and further, by using the character recognition process, the processing target range is narrowed down to suppress the occurrence of leftover in an area where there is no additional information, An object of the present invention is to perform more accurate retouching extraction by performing an interpolation operation on a retouched image that has been partially lost due to difference processing. That is, an object of the present invention is to improve extraction accuracy in a retouching extraction apparatus from an electronic document and a retouched superimposed image.
[0011]
[Means for Solving the Problems]
  To achieve the above object, according to the first aspect of the present invention, the retouching extraction apparatus includes an image reading device in a state in which a manuscript image of an electronic document and a handwritten retouching are superimposed after the electronic document is output to paper. With images digitized againCharacter recognition means for performing character recognition processing on a given area and outputting a recognition result of each character image in the area, the certainty of the character in the recognition result, and the character area information; and the character A position detecting unit for detecting a region in the superimposed image corresponding to the character image from the region information of the character image obtained from the recognizing unit; and a rewritten superimposed image of the character image detected by the position detecting unit. Recognition result for determining the necessity of additional extraction operation by comparing the character recognition result and the certainty factor of the recognition result character with the recognition result and the certainty factor of the recognition result character obtained by the character recognition means By the comparison means and the recognition result comparison meansImage difference means for performing difference calculation on the character image area determined to require difference calculation, and image interpolation means for performing interpolation processing on the image subjected to difference processing by the image difference means. It is characterized by.
[0012]
  According to the invention of claim 2, the retouching extraction device isCharacter recognition processing is performed on the given area of the original image of the electronic document and the image digitized again by the image reading device in a state where the handwritten writing is superimposed after the electronic document is output to paper, and for each character in the area A character recognition unit that outputs region information of the character image, a position detection unit that detects a region in the superimposed image corresponding to the character image from the region information of the character image obtained from the character recognition unit, Necessity of the retouching extraction operation by counting and comparing the sum of black pixels in the region in the retouched superimposed image of the character image detected by the position detection means and the sum of black pixels of the character image in the electronic original image A pixel counting means for determining the presence / absence of sex, an image difference means for performing a difference calculation on a region of the character image determined to require a difference calculation by the pixel counting means, and an image difference means An image interpolation means for performing interpolation processing on the difference processed image I, to have aIt is a feature.
[0013]
  According to invention of Claim 3, a retouching extraction apparatus isThe position detection means includes correlation position detection means for calculating an image correlation function between the area of the electronic original image and the retouched superimposed image and detecting a positional relationship between the two images where the correlation function reaches a maximum value. The retouching extraction apparatus according to claim 1 or 2, characterized in that it is characterized in that
[0014]
  According to invention of Claim 4, a retouching extraction apparatus isThe position detection means calculates the sum of the black pixels after the difference between the area of the electronic original image and the retouched superimposed image, and detects the positional relationship between the two images where the sum reaches the minimum value. The retouching extraction apparatus according to claim 1, wherein:
[0015]
  According to the invention of claim 5, the retouching extraction device is:The image difference unit performs a difference process between a pixel shift processing unit that slightly shifts from the corresponding position of both the electronic original image and the retouched superimposed image, and two images that are slightly shifted by the pixel shift processing unit. 3. The retouching extraction apparatus according to claim 1, further comprising difference processing means for performing the processing.
[0016]
  According to the invention described in claim 6, the retouching extraction device comprises:The image difference means includes an image thickening processing means for thickening an electronic original image, a difference processing means for performing a difference process between the electronic original image thickened by the image thickening processing means and the retouched superimposed image, The retouching extraction apparatus according to claim 1, wherein
[0017]
  According to the seventh aspect of the present invention, the retouching extraction device comprises:The image interpolating means includes a contour extracting means for creating a contour image of the image after the difference processing, and the electronic original image subjected to the pixel shifting process or the image thickening process from the contour image created by the contour extracting means. The end point detecting means for detecting the coordinates of the end points to be interpolated by the difference and the interpolation direction, the end point connecting means for connecting the end points from the end point information detected by the end point detecting means to form a closed curve, and the end point connection The retouching extraction apparatus according to claim 5, further comprising: an image coloring unit that colors the inside of the closed curve created by the unit.
[0018]
  According to invention of Claim 8,Character recognition processing is performed on a given area between an original image of an electronic document and an image digitized again by the image reading apparatus in a state where handwritten writing is superimposed after the electronic document is output to paper, and one character in the area A character recognition step for outputting the recognition result of each character image, the certainty factor of the character in the recognition result, and the region information of the character, and the character image from the region information of the character image obtained from the character recognition step. A position detecting step for detecting a region in the corresponding retouched superimposed image, a character recognition result for the region in the retouched superimposed image of the character image detected by the previous position detecting step, and a certainty factor of the recognition result character, Recognition result comparison that determines the necessity of additional extraction operation by comparing the recognition result obtained by the character recognition step and the certainty of the recognition result character And an image difference step for performing difference calculation on the small area determined to require difference calculation by the recognition result comparison step, and an image for performing interpolation processing on the image subjected to difference processing by the image difference step And an interpolation step.
[0019]
  According to the invention of claim 9,Character recognition processing is performed on the given area of the original image of the electronic document and the image digitized again by the image reading device in a state where the handwritten writing is superimposed after the electronic document is output to paper, and for each character in the area A character recognition step for outputting region information of the character image, a position detection step for detecting a region in the superimposed image corresponding to the character image from the region information of the character image obtained from the character recognition step, Necessity of the retouching extraction operation by counting and comparing the sum of the black pixels in the region in the retouched superimposed image of the character image detected by the position detection step and the sum of the black pixels of the character image in the electronic original image A pixel counting step for determining the presence or absence of sex, and an image difference step for performing a difference calculation on the small area determined to require a difference calculation by the pixel counting step; Is a retouched extraction method characterized by having an image interpolation step of performing interpolation processing on the difference processed image by said image difference step.
[0020]
  According to the invention of claim 10,The position detection step includes a correlation position detection step of calculating an image correlation function between the area of the electronic original image and the retouched superimposed image and detecting a positional relationship between the two images where the correlation function reaches a maximum value. The retouched extraction method according to claim 8 or 9, characterized in that it is a feature.
[0021]
  According to the invention of claim 11,The position detecting step calculates a sum of black pixels after the difference between the area of the electronic original image and the retouched superimposed image, and detects a positional relationship between the two images where the sum reaches a minimum value. 10. The retouching extraction method according to claim 8 or 9, characterized by comprising:
[0022]
  According to invention of Claim 12,The image difference step includes a pixel shift processing step for slightly shifting from the corresponding positions of both the electronic original image and the retouched superimposed image, and a difference process between the two images slightly shifted by the pixel shift processing step. The difference extraction step to perform, The rewriting extraction method of Claim 8 or 9 characterized by the above-mentioned.
[0023]
  According to the invention of claim 13,The image difference step includes an image thickening process step for thickening an electronic original image, a difference processing step for performing a difference process between the electronic original image thickened by the image thickening process step and the retouched superimposed image, 10. The retouching extraction method according to claim 8 or 9, characterized by comprising:
[0024]
  According to the invention of claim 14,The image interpolation step includes a contour extraction step of creating a contour image of the image after the difference processing, and the electronic original image that has been subjected to the pixel shifting process or the image thickening process from the contour image created by the contour extraction step. An end point detection step for detecting the coordinates and interpolation direction of the end points to be interpolated by subtracting, an end point connection step for connecting the end points from the end point information detected by the end point detection step to form a closed curve, and the end point connection The retouching extraction method according to claim 12 or 13, further comprising an image coloring step of coloring the inside of the closed curve created by the step.
[0027]
DETAILED DESCRIPTION OF THE INVENTION
Next, an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention will be described in detail with reference to the accompanying drawings.
[0028]
FIG. 1 shows a processing procedure for rewriting extraction in one embodiment of a retouching extraction apparatus and retouching extraction method based on image differences according to the present invention.
[0029]
In step S101, the number N of areas to be extracted and the position and size of each extraction target area are set, and the electronic original image and the image are output to paper and added to the image reading apparatus such as a scanner. The retouched superimposed image is input.
[0030]
In step S102, the number-of-regions variable n is set to 1.
In step S103, an area to be extracted is extracted from the superimposed image, and an area corresponding to the area extracted from the superimposed image is extracted from the electronic original image.
In step S104, the electronic original image cut out in step S103 is divided into M small areas.
[0031]
FIG. 2 shows the division processing step S104.
[0032]
In FIG. 2, in step S201, the division number variable m is set to 0 in order to count the division number.
In step S202, character recognition processing is performed on the area of the electronic document image cut out in step S103. For example, a method described in Japanese Patent Laid-Open No. 10-171927 “Character recognition device, character recognition method, and so on” can be used.
[0033]
If the area of the electronic original image includes a character area, the number L of characters in the area of the electronic original image, the recognition result of each character {Rrl}, the recognition certainty {Rcl}, and the character area are processed in step S202. {Crl} is obtained.
[0034]
In step S203, the character number variable l is set to 1.
In step S204, a region Crl 'in the retouched superimposed image corresponding to the character region Crl in the electronic document image is detected.
[0035]
As a means for detecting the corresponding area, a correlation function between the character image of the character area Crl in the electronic original image and the retouched superimposed image is calculated in the vicinity of Crl, and the correlation function shows the maximum value. Substitute “Crl ′” as “position”.
[0036]
Further, as another means of step S204, an area having the same size as Crl is cut out in the vicinity of Crl in the retouched superimposed image, and the character area Crl in the electronic original image is subjected to differential processing from the area to sum the remaining black pixels. Calculate
[0037]
This remaining black pixel total calculation is executed a plurality of times while slightly shifting when extracting the Crl region from the retouched superimposed image, and the place where the total of the remaining black pixels shows the minimum value is assigned to Crl ′ as the “most probable position”. To do.
[0038]
In step S205, character recognition processing is performed on Crl 'on the side of the retouched superimposed image obtained in step S204. This is the same as the character recognition method performed in step S202. Therefore, as in step S202, the recognition result {Rrl '} and the recognition certainty {Rcl'} of the region {Crl '} are obtained.
[0039]
In step S206, the results Rrl and Rrl 'of each character recognition are compared. If Rrl and Rrl ′ are different from each other as a result of character recognition (No), there is a possibility that some rewriting is added to the region Crl ′. Therefore, the process proceeds to step S208, the division number variable m is increased by 1, and Ram is set. After substituting Crl for Cram 'in the area Ram' corresponding to Ram, the process proceeds to step S209.
[0040]
If Rrl and Rrl 'are the same as a result of character recognition (Yes in step S206), the process proceeds to step S207. Here, Rcl is multiplied by a constant α of 1 or less and compared with Rcl ′. When Rcl ′ is smaller (No), that is, when the recognition reliability is significantly reduced, some rewriting is added to the region Crl ′. Therefore, the process proceeds to step S208, the division number variable m is incremented by 1, Crl is substituted into Ram, and Crl 'is substituted into area Ram' corresponding to Ram, and the process proceeds to step S209. If Rcl 'is larger (Yes in step S207), the possibility of writing is low, and the process proceeds to step S209 without going to step S208.
[0041]
In step S209, the number-of-characters variable l is incremented by 1, and compared with the number of characters L in step S210. If the number-of-characters variable l exceeds the number of characters L (Yes), the number of divided areas m is substituted for the number of divisions M in step S211. {Ram} and {Ram ′} are output to step S105.
If not exceeded (No in step S210), the processing from step S204 is repeated.
[0042]
FIG. 3 shows another means of the division processing step S104.
[0043]
In step S301, m is set to 0 in order to count the number of divisions.
In step S302, character recognition processing is performed on the area of the electronic document image cut out in step S103. For this, the same one as in step S202 of FIG. 2 can be used.
[0044]
If the area of the electronic original image includes a character area, the number of characters L in the area of the electronic original image, the recognition result {Rrl}, the recognition certainty {Rcl}, and the character area are processed in step S302. {Crl} is obtained.
[0045]
In step S303, the character number variable l is set to 1.
In step S304, a region Crl 'in the retouched superimposed image corresponding to the character region Crl in the electronic document image is detected. As means for detecting the corresponding area, the same means as in step S204 in FIG. 2 can be used.
[0046]
In step S305, black pixel counting processing is performed on Crl on the electronic document image side and Crl 'on the retouched superimposed image side obtained in step S304. The number of black pixels is Nbl and Nbl ′.
[0047]
In step S306, the black pixel numbers Nbl and Nbl 'obtained in step S305 are compared.
[0048]
When Nbl ′ is multiplied by a constant β of 1 or less and compared with Nbl, Nbl is smaller (No in step S306), that is, when a significant increase in black pixels is observed on the side of the superimposed image, the region Crl ′ Since there is a possibility that some writing has been added, the process proceeds to step S307, the divisor variable m is incremented by 1, Crl is substituted into Ram, and Crl 'is substituted into the area Ram' corresponding to Ram, and then to step S308. move on.
[0049]
If Nbl is larger (Yes in step S306), the possibility of writing is low, and the process proceeds to step S308 without going to step S307.
[0050]
In step S308, the number-of-characters variable l is incremented by 1, and compared with the number of characters L in step S309. If the number-of-characters variable l exceeds the number of characters L (Yes), the number of divided areas m is substituted for the number of divisions M in step S310. {Ram} and {Ram ′} are output to step S105. If not exceeded (No in step S309), the processing from step S304 is repeated.
[0051]
Further, the portion that is not recognized as a character in step S202 and step S302 is stored as a pasting destination of a retouched image to be described later.
[0052]
In step S105 in FIG. 1, the division number variable m is set to 1.
In step S106, image difference processing is performed on the regions Ram and Ram 'determined to be possible to be added in step S104, and the resulting difference processed image is output to step S107.
[0053]
FIG. 4 shows the image difference processing step S106.
[0054]
In step S401, the ordinate variable y is set to -a.
In step S402, the abscissa variable x is set to -a.
[0055]
In step S403, an image with pixels shifted by (x, y) is extracted from the area Ram on the electronic document side. In step S404, the difference between Ram and Ram ′ extracted by pixel shifting in step S403 is performed.
[0056]
In step S405, x is incremented by 1. If the abscissa variable x exceeds a in step S406 (Yes), the process proceeds to step S407. If not exceeded (No in step S406), the processing from step S403 is repeated.
[0057]
In step S407, y is incremented by 1. If the abscissa variable y exceeds a in step S408 (Yes), the difference processing result in step S404 is output to step S107. If not exceeded (No in step S408), the processing from step S402 is repeated. A value of 1 to 2 is used.
[0058]
FIG. 5 shows another means of the division processing step S106.
[0059]
In step S501, image thickening processing is performed on Ram.
In step S502, difference processing is performed using Ram ′ and Ram subjected to image thickening processing, and the difference processing result is output to step S107.
[0060]
If direct difference processing is performed without performing the processing of FIG. 4 or FIG. 5, there is a possibility that the original component on the side of the superimposed image is left behind. This is because the electronic manuscript image is an electronic manuscript imaged directly inside the computer, while the retouched superimposed image is an electronic manuscript once output on paper and imaged by a device such as a scanner, so the image quality changes greatly. Because there is a possibility of doing.
[0061]
In step S107 in FIG. 1, an image interpolation process is performed on the difference processed image input from step S106.
[0062]
FIG. 6 shows the image interpolation processing step S107.
[0063]
In step S601, a contour extraction process is performed on the difference processing image input from step S106 to generate a difference contour image.
[0064]
In step S602, the difference processing shown in FIG. 4 or 5 is performed again on the difference contour image generated in step S601 using the electronic original image to create an end point image.
[0065]
In step S603, the end point of the line segment in the end point image created in step S602 is detected, and the coordinates and the line segment vector near the end point are calculated.
[0066]
In step S604, the end points detected in step S603 are connected. The rule for connecting end points is to select two unconnected end points, determine whether the coordinates of one end point are on the extension of the line with the other end point, and if it can be considered to be on the extension Connect two points with a straight line. If it cannot be regarded as being on the extension, one end point is replaced with another unconnected end point and the same determination is performed.
[0067]
In step S605, it is determined whether or not all end points are connected. If there are unconnected end points, the operation of step S604 is repeated to create an estimated contour image. If all the end points are connected, the process proceeds to step S606.
[0068]
In step S606, the inside of the contour is colored with respect to the estimated contour image created by the processing in steps S604 and S605. A difference-processed image is used to determine whether the contour is “inside” or “outside”. Since the black pixel portion in the difference processed image is inside the contour in the estimated contour image, the inside / outside of the contour can be determined. The retouched small area image created in this way is output to step S108.
[0069]
In step S108 in FIG. 1, the retouched image created in step S107 is synthesized based on the area information obtained from step S104 and the background image.
[0070]
In step S109, the division number variable m is increased by 1. In step S110, it is determined whether or not the number M of divided areas is exceeded. If not (No), the process is repeated from step S106. If exceeded (Yes in step S110), in step S111, the retouched image synthesized in step S108 is synthesized based on the area information at the position extracted in step S103 with respect to the blank image.
[0071]
In step S112, the number-of-regions variable n is incremented by 1. In step S113, it is determined whether or not the number of regions N is exceeded. If not (No), the processing is repeated from step S103. In step S113, Yes), step S114 outputs the retouched image.
[0072]
7 and 8 schematically describe the above-described series of processing using images.
[0073]
In FIG. 7, two images of an electronic original and a retouched superimposed image are provided in the above-described step S101.
[0074]
First, in FIG. 7 (1), a region for retouching extraction is extracted from these two images (see step S103).
Next, character recognition processing is performed on the electronic document image side of the extracted area in FIG. 7 (2) to obtain character image information in the area.
The area information of the character image previously obtained from the electronic document image is applied to the retouched superimposed image side ((3) in FIG. 7), and character recognition or black pixel counting is performed for each character image and compared with the electronic document image side. (Fig. 7 (4)). At this time, the area that has not been recognized as the “character image” by the character recognition processing is stored as a background for compositing the retouched image ((5) in FIG. 7).
[0075]
With the above operation, it is possible to narrow down the area that needs the retouching extraction operation as shown in the lower part of FIG. These processes correspond to step S104 described above.
[0076]
FIG. 8 shows the processing after step S106 described above. From the divided image area information obtained in step S104, the electronic original image and the retouched superimposed image in the upper left part of FIG. 8 are obtained.
[0077]
The difference process shown in FIG. 8 (1) is performed between the two images (see step S106) to obtain a difference image. By performing contour extraction on the difference image obtained here as shown in FIG. 8 (2) (see step S601), a difference contour image is obtained.
[0078]
The end point image is obtained by performing the difference processing (see step S602) in FIG. 8 (4) on this image. In FIG. 8, a thickened character image on the electronic document image side is prepared in advance. (FIG. 8 (3)), and difference processing is performed. That is, the method of steps S501 and S502 described above is employed for the image difference process.
[0079]
An end point detection process is performed on the end point image thus obtained, and the coordinates of each end point and a vector on the end point of the line segment including the end point are calculated (see step S603).
[0080]
These end points are connected to each other in FIG. 8 (5) (see step S604), and when all end points are connected and an end contour estimation image is generated, coloring processing is performed in FIG. 8 (6).
[0081]
The coloring operation is divided into a portion where the contour of the contour estimation image is colored at the boundary and a portion where the contour is not colored. For this determination, the difference image created in FIG. 8 (1) as shown in FIG. And a coloring pattern having a high correlation with the reference image is adopted.
The retouched image in the lower right of FIG. 8 is obtained by the above operation.
[0082]
FIG. 9 shows a configuration example of a retouching extractor in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
[0083]
In FIG. 9, the retouching extractor 900 includes a region extraction unit 901, a region division unit 902, an image difference unit 903, an image interpolation unit 904, and an image composition unit 905.
[0084]
The area extraction unit 901 uses the electronic original image and the retouched superimposed image, the number of areas to be retouched and the position and size of each area set in step S101 of FIG. The desired area is cut out from both images in S103.
[0085]
The area dividing unit 902 performs further area division in step S <b> 104 on the area extracted by the area extracting unit 901.
[0086]
FIG. 10 shows a configuration example of a region divider in one embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
[0087]
10, the area divider 1000 is an example of the area dividing unit 902 in FIG. 9, and includes a character recognition unit 1001, a position detection unit 1002, and a recognition result comparison unit 1003.
[0088]
The character recognition unit 1001 performs the character recognition process in step S202 of FIG. 2 on the area on the electronic document side extracted by the area extraction unit 901 of FIG.
[0089]
If a character image is included in the electronic original image, the recognition result, the certainty factor, and the area information (position and size) of each character are obtained for each character.
[0090]
Next, the position detecting unit 1002 detects a region on the retouched superimposed image corresponding to the image region for each character of the electronic document image output from the previous character recognizing unit 1001, and obtains a deviation amount between the two regions. Here, the operation of step S204 in FIG. 2 is performed.
[0091]
The recognition result comparison unit 1003 performs character recognition processing in step S205 of FIG. 2 using the character region information of each electronic document image obtained by the previous position detection unit 1002 for the rewritten superimposed image. The character recognition processing result obtained here and the certainty factor for the result are compared with the character recognition result obtained by the character recognition unit 1001 and the certainty factor for the result. This corresponds to the operations in step S206 and step S207 in FIG.
[0092]
If the character area considered here is not retouched, the character recognition processing result performed on the retouched superimposed image and the character recognition processing result performed on the electronic document image match, and the certainty level The same level can be obtained. If there is any additional component in the character area, the difference in the result between the character recognition processing results of both images of one character or a plurality of characters, or a change in certainty factor occurs even if the result is the same.
[0093]
The area information in which the additional component obtained in this way may be superimposed and the area information on the additional superimposed image obtained from the position detection unit 1002 are saved by the substitution operation in step S208 in FIG. It outputs to the image difference part 903 of FIG.
[0094]
FIG. 11 shows a configuration example of a region divider in another embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
[0095]
In FIG. 11, an area divider 1100 is another embodiment of the area dividing unit 902 in FIG. 9, and includes a character recognition unit 1101, a position detection unit 1102, and a pixel formation unit 1103.
[0096]
Since the character recognizing unit 1101 has a function equivalent to that of the character recognizing unit 1001 in FIG. 10, the position information of the character image included in the region extracted by the region extracting unit 901 in FIG. 9 is obtained. This corresponds to the operation in step S302 in FIG.
[0097]
Next, the position detection unit 1102 detects a region on the retouched superimposed image corresponding to the character image region for each character of the electronic document image output from the previous character recognition unit 1101, and obtains a deviation amount between the two regions. This corresponds to the operation of step S304 in FIG.
[0098]
Next, the pixel counting unit 1103 extracts a portion corresponding to the position of the character image obtained by the previous position detection unit 1102 from the electronic original image and the retouched superimposed image, and within the region corresponding to step S305 in FIG. The black pixel count is performed on the two images, and the comparison in step S306 is performed.
[0099]
If the character area considered here has not been rewritten, the number of black pixels in the character area extracted image on the side of the superimposed image and the number of black pixels in the character area extracted image of the electronic document image are substantially the same.
[0100]
When there is any additional component in the character area, the number of black pixels in the character area extracted image on the side of the superimposed image is significantly larger than the number of black pixels in the character area extracted image of the electronic document image.
[0101]
The area information on which the additional component obtained in this way may be superimposed and the area information on the additional superimposed image obtained from the position detection unit 1102 are saved by the substitution operation in step S307 in FIG. Output to 903.
[0102]
In the area dividers of FIGS. 10 and 11, an area that is not determined as a character image is stored as a pasting destination after interpolation processing described later.
[0103]
FIG. 12 shows a configuration example of a position detector in one embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
[0104]
In FIG. 12, a position detector 1200 is an example of the position detection unit 1002 and 1102 in FIGS. 10 and 11, and includes a correlation calculation unit 1201 and a maximum correlation detection unit 1202.
[0105]
The correlation calculation unit 1201 creates a reference image by cutting out an image from the electronic original image based on the image area information for each character input from the character recognition unit 1001 and 1101 in FIGS. The correlation function between the image and the superimposed image is calculated.
[0106]
Next, the maximum correlation detection unit 1202 detects the coordinate reaching the maximum value using the correlation function calculated by the correlation calculation unit 1201, and uses the coordinate as the relative position between the reference image and the retouched superimposed image, thereby recognizing the result comparison unit shown in FIG. 1003 and output to the pixel counting unit 1103 in FIG.
[0107]
FIG. 13 shows a configuration example of a position detector in another embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
[0108]
In FIG. 13, a position detector 1300 is another example of the position detection unit 1002 and 1102 in FIGS. 10 and 11, and includes a difference processing unit 1301 and a minimum remaining black pixel detection unit 1302.
[0109]
The difference processing unit 1301 creates a reference image by cutting out an image from the electronic original image based on the image area information for each character input from the character recognition unit 1001 and 1101 in FIGS. The total value of the remaining black pixels obtained by subtracting the reference image from the image is calculated.
[0110]
This difference calculation and remaining black pixel sum calculation are executed a plurality of times while shifting the positional relationship between the reference image and the retouched superimposed image, and the sum of the remaining black pixels is used as a function of the reference image shift amount.
[0111]
Next, the minimum remaining black pixel detection unit 1302 detects a coordinate reaching the minimum value by the residual black pixel function calculated by the difference processing unit 1301, and the coordinate is set as a relative position between the reference image and the superimposed image in FIG. The result is output to the recognition result comparison unit 1003 and the pixel counting unit 1103 in FIG.
[0112]
The image difference unit 903 in FIG. 9 performs the difference process in step S106 of FIG. 1 on the region where the writing component input from the region dividing unit 902 may be superimposed, and the image of the processing result is converted into an image. The data is output to the interpolation unit 904.
[0113]
FIG. 14 shows a configuration example of an image differentiator in one embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
[0114]
14, an image differentiator 1400 is an example of the image difference unit 903 in FIG. 9, and includes a pixel shift processing unit 1401 and a difference processing unit 1402.
[0115]
The pixel shift processing unit 1401 is based on the region information of the electronic document image that may have the additional component input from the region dividing unit 902 in FIG. 9 and the region information on the additional superimposed image corresponding to the region. Image clipping is performed from both the electronic original image and the retouched superimposed image.
[0116]
When the image is cut out from the electronic document image, the cut out region is cut out while shifting by several pixels. This corresponds to the processing in steps S401 to S403 and steps S405 to S408 in FIG.
[0117]
Next, the difference processing unit 1402 performs difference processing in step S404 between the small region of the superimposed image extracted by the pixel shift processing unit 1401 and the small region of the electronic original image extracted by the pixel shift processing unit 1401. And the processing result is output to the image interpolation unit 904 in FIG.
[0118]
FIG. 15 shows a configuration example of an image differentiator in another embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
[0119]
In FIG. 15, an image differentiator 1500 is another example of the image difference unit 903 in FIG. 9, and includes an image thickening processing unit 1501 and a difference processing unit 1502.
[0120]
The image thickening processing unit 1501 is based on the region information of the electronic document image that may have the additional component input from the region dividing unit 902 in FIG. 9 and the region information on the additional superimposed image corresponding to the region. Then, image clipping is performed from both the electronic document image and the superimposed image, and image thickening processing is performed on the image clipped from the electronic document image. This image cut-out and thickening process corresponds to step S501 in FIG.
[0121]
Next, in the difference processing unit 1502, the small region of the superimposed image extracted by the image thickening processing unit 1501 and the small region of the electronic document image extracted by the image thickening processing unit 1501 are illustrated in FIG. The difference processing in step S502 is executed, and the processing result is output to the image interpolation unit 904 in FIG.
[0122]
The image interpolation unit 904 in FIG. 9 performs the image interpolation process in step S107 in FIG. 1 on the difference image input from the image difference unit 903, and outputs the processing result to the image composition unit 905.
[0123]
FIG. 16 shows a configuration example of an image interpolator in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
[0124]
In FIG. 16, an image interpolator 1600 is an example of the image interpolation unit 904 in FIG. 9, and includes an outline extraction unit 1601, an end point detection unit 1602, an end point connection unit 1603, an image coloring unit 1604, and a region synthesis. Part 1605.
[0125]
The contour extraction unit 1601 performs the contour emphasis processing in step S601 in FIG. 6 on the image after the difference processing input from the image difference unit 903 in FIG. 9 to create a difference contour image.
[0126]
Next, the end point detection unit 1602 performs the difference process in step S602 in FIG. 6 between the difference contour image created by the contour extraction unit 1601 and the electronic document image, creates an end point image, and coordinates of the end points in the image To extract. This corresponds to step S603 in FIG.
[0127]
Next, in the end point connection unit 1603, the end point connection in step S604 in FIG. 6 is performed on the end points on the extension of the line segment based on the end point image and end point information created in the end point detection unit 1602, and the estimated contour image Create
[0128]
Next, the image coloring unit 1604 performs the coloring process in step S606 of FIG. 6 on the inside of the estimated contour image.
[0129]
Finally, the region composition unit 1605 performs a pasting process on the image stored as the paste destination image by the region division unit 902 in FIG. 9 and outputs the image to the image composition unit 905.
[0130]
9, based on the extracted area information input from the area extracting unit 901, the image after the image interpolation process input from the image interpolating unit 904, that is, the retouched image in step S111 in FIG. Processing is performed on a blank image, and as a result, a single retouched image is completed.
[0131]
【The invention's effect】
  According to the invention described in claim 1, the retouching extraction device is,Improved accuracy of retouching extractionIn addition, the calculation efficiency is improved.
[0132]
  According to the invention of claim 2, the retouching extraction device isThe accuracy of retouching extraction is improved, and the calculation efficiency is improved.
[0133]
  According to invention of Claim 3, a retouching extraction apparatus isThe retouching extraction apparatus according to claim 1 or 2, wherein the retouching extraction accuracy is improved.
[0134]
  According to invention of Claim 4, a retouching extraction apparatus isThe retouching extraction apparatus according to claim 1 or 2, wherein the retouching extraction accuracy is improved.
[0135]
  According to the invention of claim 5, the retouching extraction device is:In the retouching extraction apparatus according to claim 1 or 2, it is possible to prevent a leftover due to a deterioration in image quality of a retouched superimposed image, and the accuracy of rewriting extraction is improved.
[0136]
  According to the invention described in claim 6, the retouching extraction device comprises:In the retouching extraction apparatus according to claim 1 or 2, it is possible to prevent a leftover due to a deterioration in image quality of a retouched superimposed image, and the accuracy of rewriting extraction is improved.
[0137]
  According to the seventh aspect of the present invention, the retouching extraction device comprises:In the retouching extraction device according to claim 5 or 6, interpolation after overremoving of retouching components is possible, and accuracy of retouching extraction is improved.
[0138]
  According to the invention described in claim 8, the retouching extraction method is,Improved accuracy of retouching extractionIn addition, the calculation efficiency is improved.
[0139]
  According to the invention of claim 9, the retouching extraction method is:,Improved accuracy of retouching extractionIn addition, the calculation efficiency is improved.
[0140]
  According to the invention described in claim 10, the retouching extraction method comprises:In the retouching extraction method according to 8 or 9, the retouching extraction accuracy is improved.
[0141]
  According to the invention described in claim 11, the retouching extraction method comprises:In the retouching extraction method according to 8 or 9, the retouching extraction accuracy is improved.
[0142]
  According to the invention described in claim 12, the retouching extraction method comprises:In the retouching extraction method according to 8 or 9, it is possible to prevent a leftover due to a deterioration in image quality of the retouched superimposed image, and the accuracy of rewriting extraction is improved.
[0143]
  According to the invention as set forth in claim 13, the retouching extraction method comprises:In the retouching extraction method according to 8 or 9, it is possible to prevent a leftover due to a deterioration in image quality of the retouched superimposed image, and the accuracy of rewriting extraction is improved.
[0144]
  According to the invention of claim 14, the rewriting extraction method is:In the retouching extraction method according to claim 12 or 13, interpolation after overremoving of retouching components is possible, and accuracy of retouching extraction is improved.
[Brief description of the drawings]
FIG. 1 is a flowchart of retouching extraction in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 2 is a flowchart of an area dividing step in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 3 is a flowchart of an area dividing step in another embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
FIG. 4 is a flowchart of an image thickening difference in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 5 is a flowchart of an image thickening difference in another embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
FIG. 6 is a flowchart of an image interpolation step in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 7 is an explanatory diagram of region extraction / region division in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 8 is an explanatory diagram of image differences and the like in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 9 is a configuration example of a retouching extractor in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 10 is a configuration example of a region divider in one embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 11 is a configuration example of a region divider in another embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
FIG. 12 is a configuration example of a position detector in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 13 is a configuration example of a position detector in another embodiment of the retouching extraction apparatus and retouching extraction method according to the present invention.
FIG. 14 is a configuration example of an image differentiator in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 15 is a configuration example of an image differentiator in another embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
FIG. 16 is a configuration example of an image interpolator in an embodiment of a retouching extraction apparatus and a retouching extraction method according to the present invention.
[Explanation of symbols]
900 Retouching extractor
901 Region extraction unit
902 Region division unit
903 Image difference unit
904 Image interpolation unit
905 Image composition unit
1000, 1100 region divider
1001, 1101 Character recognition unit
1002, 1102 Position detection unit
1003 Recognition result comparison part
1103 Pixel counting unit
1200, 1300 Position detector
1201 Correlation calculator
1202 Maximum correlation detector
1301, 1402, 1502 Difference processing unit
1302 Minimum remaining black pixel detection unit
1400, 1500 Image differentiator
1401 Pixel shift processing unit
1501 Image thickening processing unit
1600 Image interpolator
1601 Outline extraction unit
1602 Endpoint detection unit
1603 End point connection
1604 Image coloring section
1605 area synthesis unit

Claims (14)

Character recognition processing is performed on the given area of the original image of the electronic document and the image digitized again by the image reading device in a state in which the electronic document is output to paper and then the handwritten writing is superimposed . A character recognition means for outputting a recognition result of each character image, a certainty factor of the character in the recognition result, and region information of the character;
Position detecting means for detecting a region in the superimposed image corresponding to the character image from the region information of the character image obtained from the character recognition unit;
The character recognition result and the recognition result character confidence level for the region in the retouched superimposed image of the character image detected by the position detection unit, and the recognition result obtained by the character recognition unit and the confidence level of the recognition result character. A recognition result comparison means for determining whether or not there is a need for an additional extraction operation by comparing;
Image difference means for performing difference calculation on the region of the character image determined to require difference calculation by the recognition result comparison means ;
Image interpolation means for performing interpolation processing on an image subjected to difference processing by the image difference means;
A retouching extraction device characterized by comprising: Character recognition processing is performed on the given area of the original image of the electronic document and the image digitized again by the image reading device in a state where the electronic document is output to paper and then the handwritten writing is superimposed. Character recognition means for outputting region information of the character image of
  Position detecting means for detecting a region in the superimposed image corresponding to the character image from the region information of the character image obtained from the character recognition unit;
  The sum of the black pixels in the region in the retouched superimposed image of the character image detected by the position detecting means and the sum of the black pixels of the character image in the electronic original image are counted and compared to perform the retouching extraction operation. Pixel counting means for determining the necessity or not,
  Image difference means for performing difference calculation on the area of the character image determined to require difference calculation by the pixel counting means;
  Image interpolation means for performing interpolation processing on an image subjected to difference processing by the image difference means;
  A retouching extraction device characterized by comprising: The position detecting means includes
  2. Correlated position detecting means for calculating an image correlation function between an area of an electronic original image and a retouched superimposed image and detecting a positional relationship between two images where the correlation function reaches a maximum value. Or the rewriting extraction apparatus of 2 description. The position detecting means includes
  Differential position detection means for calculating the sum of black pixels after the difference between the area of the electronic original image and the retouched superimposed image and detecting the positional relationship between the two images where the sum reaches the minimum value;
  The retouching extraction apparatus according to claim 1 or 2, characterized by comprising: The image difference means includes
  Pixel shift processing means for slightly shifting from the corresponding positions of both the electronic original image and the retouched superimposed image;
  Difference processing means for performing difference processing between two images slightly shifted by the pixel shift processing means;
  The retouching extraction apparatus according to claim 1 or 2, characterized by comprising: The image difference means includes
  Image thickening processing means for thickening an electronic original image;
  Difference processing means for performing difference processing between the electronic original image thickened by the image thickening processing means and the retouched superimposed image;
  The retouching extraction apparatus according to claim 1, wherein The image interpolation means includes
  Contour extracting means for creating a contour image of the image after the difference processing;
  Endpoint detection means for detecting the coordinates and interpolation direction of the endpoint to be interpolated by subtracting the pixel shifting process or the image thickening electronic document image from the outline image created by the outline extraction means;
  An end point connecting means for connecting the end points from the end point information detected by the end point detecting means to form a closed curve;
  Image coloring means for coloring the inside of the closed curve created by the end point connecting means;
  The retouching extraction apparatus according to claim 5 or 6, characterized by comprising: Character recognition processing is performed on a given area between an original image of an electronic document and an image digitized again by the image reading apparatus in a state where handwritten writing is superimposed after the electronic document is output to paper, and one character in the area A character recognition step of outputting a recognition result of each character image, a certainty factor of the character in the recognition result, and region information of the character;
  A position detection step of detecting a region in the superimposed image corresponding to the character image from the region information of the character image obtained from the character recognition step;
  Character recognition result and recognition result character confidence level for the region in the superimposed image of the character image detected by the previous position detection step, recognition result obtained by the character recognition step and recognition result character confidence level A recognition result comparison step for determining whether or not there is a need for an additional extraction operation by comparing; and
  An image difference step for performing a difference calculation on the small area determined to require a difference calculation by the recognition result comparison step;
  An image interpolation step for performing an interpolation process on the image subjected to the difference process by the image difference step;
  A retouching extraction method characterized by comprising: Character recognition processing is performed on the given area of the original image of the electronic document and the image digitized again by the image reading device in a state where the handwritten writing is superimposed after the electronic document is output to paper, and for each character in the area A character recognition step for outputting region information of the character image of
  A position detection step of detecting a region in the superimposed image corresponding to the character image from the region information of the character image obtained from the character recognition step;
  By counting and comparing the sum of black pixels in the region in the retouched superimposed image of the character image detected by the previous position detection step and the sum of black pixels in the character image in the electronic original image, A pixel counting step for determining the necessity or not,
  An image difference step for performing a difference calculation on the small area determined to require a difference calculation by the pixel counting step;
  An image interpolation step for performing an interpolation process on the image subjected to the difference process by the image difference step;
  A retouching extraction method characterized by comprising: The position detecting step includes
  A correlation position detection step of calculating an image correlation function between the area of the electronic original image and the retouched superimposed image and detecting a positional relationship between the two images where the correlation function reaches a maximum value;
  The retouched extraction method according to claim 8 or 9, characterized by comprising: The position detecting step includes
  A difference position detection step of calculating a sum of black pixels after the difference between the area of the electronic original image and the retouched superimposed image and detecting a positional relationship between the two images where the sum value reaches a minimum value;
  The retouched extraction method according to claim 8 or 9, characterized by comprising: The image difference step includes
  A pixel shifting process step for slightly shifting from the corresponding positions of both the electronic original image and the retouched superimposed image;
  A difference processing step for performing a difference process between the two images slightly shifted by the pixel shift processing step;
  The retouched extraction method according to claim 8 or 9, characterized by comprising: The image difference step includes
  An image thickening process step for thickening the electronic original image;
  A difference processing step for performing a difference process between the electronic original image thickened by the image thickening processing step and the retouched superimposed image;
  The retouched extraction method according to claim 8 or 9, characterized by comprising: The image interpolation step includes:
  A contour extraction step of creating a contour image of the image after the difference processing;
  An end point detection step for detecting the coordinates of the end points to be interpolated and the interpolation direction by subtracting the electronic document image subjected to the pixel shifting process or the image thickening process from the contour image created by the contour extraction step;
  An end point connecting step of connecting end points from the end point information detected by the end point detecting step to form a closed curve;
  An image coloring step for coloring the inside of the closed curve created by the end point connecting step;
  14. The method for extracting and rewriting additional writing according to claim 12 or 13, wherein:

Images