JP4061256B2 - Image distortion correction apparatus, image reading apparatus, image forming apparatus, and program - Google Patents

Description

  The present invention relates to an image distortion correction apparatus, an image reading apparatus, an image forming apparatus, and a program.

Many originals read using a flatbed scanner are sheet-like originals, and an openable / closable pressure plate is provided on the contact glass. After placing the original on the contact glass, the pressure plate is closed and the original is scanned. Yes. However, the original is not limited to a sheet, and book originals (books, booklets, etc.) may be handled as originals. In such cases, the book original is placed on the contact glass and the original is scanned. Will do.
However, when a book document is used as the document, the page binding portion 101 of the book document 100 is lifted from the contact glass 102 as shown in FIG. As described above, when the page binding portion 101 of the book document 100 is lifted from the contact glass 102, the page binding portion 101 is separated from the focal plane. , Image degradation such as blurred characters occurs. The deteriorated image of the page binding portion 101 is difficult to read, and the recognition rate when performing character recognition processing by OCR is significantly reduced. In particular, in the case of thick bookbinding, the ratio is high, and when the page binding portion 101 of the book document 100 is pressed so as not to leave the focal plane, the book document 100 itself may be damaged.

In order to solve such a problem, a method of correcting image distortion using a method of estimating the three-dimensional shape of an object from image density information has been proposed. As a method for estimating the three-dimensional shape of an object from such image density information, a method called Shape from Shading described in Non-Patent Document 1 is a typical example.
Patent Document 1 proposes a method of correcting the distortion by measuring the shape of a book by a triangulation method.
Further, Patent Document 2 proposes a method for estimating the three-dimensional shape of the book surface using the shape of the page outline of the scanned read image.
However, according to the method called “Shape from Shading” described above, the amount of calculation is large, and the calculation time of the distortion correction processing is long.
In addition, according to the method described in Patent Document 1, a special shape measuring device for measuring the shape of a book by the triangulation method is required, which is not appropriate for downsizing the device.
Furthermore, according to the method described in Patent Document 2, distortion correction can be performed with a small amount of calculation, but effective correction when the page outline is not completely contained in the image and is cut off in the middle. I can't.
Therefore, Japanese Patent Application No. 2002-247463 has proposed an image distortion correction apparatus capable of effectively correcting the distortion of a scanned scan image whose page outline is cut off in the middle with a small amount of calculation. In this case, not only the page outline but also the character line information and ruled line information are used to correct image distortion, and the intrinsic parameters of the scanner (image reading means) (lens focal plane distance, scan optical axis position (address)) ) Is not used, the output image of any scanner can be corrected.
Japanese Patent Laid-Open No. 5-161002 JP 11-41455 A Japanese Patent Application No. 2002-247463 T. Wada, H. Uchida and T. Matsuyama, “Shape from Shading with Interreflections under a Proximal Light Source: Distortion-Free Copying of an Unfolded Book”, International Journal Computer Vision 24 (2), 125-135 (1997)

However, there are cases where the intrinsic parameters of the scanner are known, and in that case, the correction accuracy is generally improved by using the parameters.
An object of the present invention is to provide an image distortion correction apparatus, an image reading apparatus, an image forming apparatus, and a program capable of providing an appropriate correction method in each case where the scanner parameter is known and unknown, and improving the correction accuracy. Is to provide.

In order to solve the above-described problems, the present invention provides a scan obtained by reading a book document image placed on a scan surface so that the page binding portion is substantially parallel to the main scanning direction. In an image distortion correction apparatus that performs distortion correction by extending an image in the main scanning direction and the sub-scanning direction, the distance between the book document and the scanning surface is determined from the scanned image and the optical axis position information of the scanner lens that has read the scanned image. And a sub-scanning direction distortion correcting unit that corrects the distortion of the scanned image based on the distance calculated by the lifting amount calculating unit. The sub-scanning direction distortion correcting unit includes the floating amount calculating unit. The distance between the book manuscript calculated by the means and the scanning surface is stored, and a predetermined number of pages to be scanned continuously are stored using the stored distance. And performing査direction distortion correction.
According to a second aspect of the present invention, in the image distortion correction apparatus according to the first aspect, the sub-scanning direction distortion correction unit folds a cross-sectional curve of the book document based on a distance from the book document and a scan surface. The correction is performed by obtaining the corresponding points before and after the correction so that the approximation is the length of the corrected image in the sub-scanning direction.

According to a third aspect of the present invention, in the image distortion correction apparatus according to the first or second aspect , the sub-scanning direction distortion correction unit performs correction for each of right and left pages of the scanned image. .
According to a fourth aspect of the present invention, there is provided an image reading apparatus comprising: an image reading unit that reads a book document image placed on a scan surface so that a page binding portion is substantially parallel to a main scanning direction; The image distortion correction apparatus according to any one of claims 1 to 3 , wherein the image distortion correction apparatus according to any one of claims 1 to 3 corrects a scanned image read by the unit.
According to a fifth aspect of the present invention, there is provided an image forming apparatus for reading a book document image placed on a scan surface so that a page binding portion is substantially parallel to a main scanning direction, and the image reading device. The image distortion correction apparatus according to any one of claims 1 to 3 , wherein the scanned image read by the means is corrected, and an image for printing an image based on the image data corrected by the image distortion correction apparatus on a sheet. And a printing apparatus.
According to a sixth aspect of the present invention, a scan image obtained by reading a book document image placed on the scan surface is expanded in the sub-scan direction so that the page binding portion is substantially parallel to the main scan direction. A program for causing a computer to perform distortion correction, and a lift amount calculating means for obtaining a distance between a book document and a scan surface from the scan image and optical axis position information of a scanner lens that has read the scan image on the computer, Based on the distance calculated by the lift amount calculation means, the distortion of the scanned image is corrected , and the distance between the book document and the scan surface calculated by the lift amount calculation means is stored, and a predetermined number to be scanned continuously Pro for relative page, to function as a sub-scanning direction distortion correcting means for correcting the distortion in the sub-scanning direction using the distances the storage Lamb.
According to a seventh aspect of the present invention, in the program according to the sixth aspect , the sub-scanning direction distortion correcting unit approximates a cross-sectional curve of the book document based on a polygonal line based on a distance between the book document and a scan surface, The correction is performed by obtaining corresponding points before and after the correction so that the accumulation becomes the length in the sub-scanning direction of the corrected image.

In addition, according to an eighth aspect of the present invention, in the program according to the sixth or seventh aspect , the sub-scanning direction distortion correcting unit performs correction for each of right and left pages of the scanned image.

  According to the present invention, since an appropriate correction method is selected and corrected when the scanner parameter is known and unknown, an image distortion correction apparatus, an image reading apparatus, an image forming apparatus, and Can provide a program.

An embodiment of the present invention will be described with reference to FIGS. The image distortion correction apparatus according to the present embodiment is provided in a digital copying machine as an image forming apparatus, and a scanner unit of the digital copying machine is applied as the image reading apparatus.
Here, FIG. 1 is a longitudinal sectional view showing a configuration of the scanner unit 1. As shown in FIG. 1, the scanner unit 1 includes a contact glass 2 on which a document is placed, a first traveling body 5 including an exposure lamp 3 for exposing a document and a first reflection mirror 4, and a second reflection mirror 6. And a second traveling body 8 composed of the third reflecting mirror 7, a CCD (Charge Coupled Device) 9 as an imaging device for reading an image of the document, a lens unit 10 for forming an image on the CCD 9, and a document placed thereon. A document scale 11 that prevents the contact glass 2 from shifting and coming off, a white reference plate 12 for shading correction installed under the document scale 11, and a frame 14 are provided. The CCD 9 is formed on the sensor board 13.
During scanning of a document, the first traveling body 5 and the second traveling body 8 are moved in the sub-scanning direction by a stepping motor 24 (see FIG. 3). That is, the first traveling body 5 and the second traveling body 8 travel under the contact glass 2, the exposure lamp 3 exposes and scans the document, and the reflected light is reflected on the first reflecting mirror 4, the second reflecting mirror 6, and the like. The light is reflected by the third reflecting mirror 7 and imaged on the CCD 9 through the lens unit 10. Here, an image reading means is realized.
Such a scanner unit 1 is a printer unit (not shown) that is an image printing apparatus that forms an image on a sheet by, for example, electrophotography, in accordance with image data based on an image of a document read by the scanner unit 1. ).

FIG. 2 is a perspective view showing an upper portion of the digital copying machine 16 on which the scanner unit 1 is mounted. As shown in FIG. 2, the scanner unit 1 is provided with a pressure plate 17 that can be opened and closed with respect to the contact glass 2, and an open / close sensor 18 that detects opening and closing of the pressure plate 17.
As the printer provided in the digital copying machine 16, various printing methods such as an ink jet method, a sublimation type thermal transfer method, a silver salt photography method, a direct thermal recording method, and a melt type thermal transfer method are applied in addition to the electrophotographic method. be able to. Since the specific configuration is well known, detailed description is omitted.
FIG. 3 is a block diagram showing the electrical connection of the control system of the scanner unit 1. As shown in FIG. 3, the control system includes an image processing unit 20 that is a circuit that performs various image processing on image data read by the CCD 9, and a first control unit 19 that controls the entire scanner unit 1. An image read by the CCD 9 and an operation panel 22 for accepting various operations to the digital copying machine 16 and receiving various operations on the digital copying machine 16, which is a circuit for controlling the traveling body 5 and the second traveling body 8. A memory 23 for storing data, predetermined data, and the like is connected. The operation panel 22 is provided with a copy start key for declaring the start of copying. Further, in the traveling body control unit 21, the exposure lamp 3, the stepping motor 24 that drives the first traveling body 5 and the second traveling body 8, and the first traveling body 5 and the second traveling body 8 are in the home position. A scanner home position sensor (HP sensor) 25 for detecting whether or not and an open / close sensor 18 are connected.

Here, FIG. 4 is a block diagram showing a basic internal configuration of the image processing unit 20. As shown in FIG. 4, the image processing unit 20 includes an analog video processing unit 26 that performs analog image signal amplification processing and digital conversion processing when a document is read by the CCD 9, a shading correction processing unit 27 that performs shading correction processing, and shading. An image data processing unit 28 that performs various image data processing such as MTF correction, scaling processing, and γ correction on the digital image signal after correction processing to generate a scan image, and a scan image that is a characteristic function of the present embodiment The image distortion correction unit 29 that realizes the above-described distortion correction function. The digital image signal after the image processing as described above is transmitted to the printer unit via the main control unit 19 and used for image formation.
As shown in FIG. 5, the main control unit 19 includes a CPU (Central Processing Unit) 31 that centrally controls each unit. The CPU 31 includes a ROM (Read Only Memory) that stores BIOS and the like. Only memory (RAM) 32 and a RAM (Random Access Memory) 33 that stores various data in a rewritable manner and functions as a work area of the CPU 31 are connected by a bus 34 to constitute a microcomputer. Further, an HDD 35 in which a control program is stored, a CD-ROM drive 36 that reads a CD (Compact Disc) -ROM 37, and an interface (I / F) 38 that controls communication with a printer unit and the like are connected to the bus 34. ing.

A CD-ROM 37 shown in FIG. 5 implements the storage medium of the present invention, and stores a predetermined control program. The CPU 31 reads the control program stored in the CD-ROM 37 with the CD-ROM drive 36 and installs it in the HDD 35. As a result, the main control unit 19 is in a state in which various processes as described later can be performed.
As the storage medium, not only the CD-ROM 37 but also various types of media such as various optical disks such as DVD, various magnetic disks such as various magneto-optical disks and flexible disks, and semiconductor memory can be used. Alternatively, the program may be downloaded from a network such as the Internet and installed in the HDD 35. In this case, the storage device storing the program in the server on the transmission side is also a storage medium of the present invention. Note that the program may operate on a predetermined OS (Operating System), in which case the OS may execute a part of various processes described later, or a word processor. It may be included as part of a group of program files that constitute predetermined application software such as software or an OS.
Next, the contents of various processes executed by the CPU 31 provided in the main control unit 19 based on the control program will be described. Here, among the processes executed by the CPU 31, the scanned image in the image distortion correction unit 29, which is an image distortion correction device that realizes the distortion correction function of the scanned image, which is a characteristic function of the scanner unit 1 of the present embodiment. Only the distortion correction processing will be described.
FIG. 6 is a flowchart schematically showing the flow of a distortion correction process for a scanned image. Here, as shown in FIG. 7, the book document 40 is placed on the contact glass 2 so that the page binding portion 41 and the main scanning direction of image reading of the scanner portion 1 are parallel to each other. The case will be described.
First, in step S1, the scan image of the book document 40 placed on the contact glass 2 output from the image data processing unit 28 is input. Here, FIG. 8 shows an example of the input image. As shown in FIG. 9, the input scan image of the book document 40 is distorted in the vicinity of the page binding portion 41.
Next, optimal binarization processing of a scanned image (for example, a monochrome multi-valued image) of the book document 40 is executed (step S2), and black pixels in the sub-scanning direction (the density value among the pixels of the scanned image is determined in advance). A histogram with the number of pixels darker than the density value is obtained (step S3). FIG. 10 is a black pixel histogram on the left side of the binding portion boundary line of the image shown in FIG. The horizontal axis in FIG. 10 indicates the position of the black pixel in the main scanning direction (the pixel whose density value is darker than the predetermined density value among the pixels of the scanned image), and the vertical axis in FIG. This indicates the number of black pixels for each position. Note that the position in the sub-scanning direction where the pixel having the darkest density value among the pixels in the scan image is selected as the binding part boundary line.
Note that the binarization process when the scanned image is a color multi-valued image focuses on, for example, any one of the RGB components (for example, the G component), and black pixels that are larger than the predetermined density threshold of the G component are black pixels. And a pixel smaller than a predetermined density threshold of the G component may be a white pixel. Alternatively, RGB may be color-converted to be divided into a luminance component and a color difference component, and threshold processing may be performed with the luminance component.
In the subsequent step S4, page outline / ruled line / character line extraction processing is executed. FIG. 11 is a flowchart schematically showing the flow of page outline / ruled line / character line extraction processing.

算出された比率ｋが、予め定められた閾値よりも大きい場合に、スキャン画像にページ外形が存在すると判断する。
なお、スキャン画像の上下にページ外形が存在する場合には、ヒストグラムの左右両端に高い縦棒が現れることになるので、このような場合には、ヒストグラムの左右両端の高い縦棒に基づいてスキャン画像にページ外形が存在するか否かの判断がそれぞれ実行される。
ここに、ページ外形判別手段の機能が実行される。
以上の処理により、スキャン画像にページ外形が存在すると判断された場合には、左右ページの上下辺のいずれにページ外形が存在しているのかという情報とともにページ外形を抽出し、ＲＡＭ３３に一時的に記憶する。
なお、このスキャン画像にページ外形が存在するか否かの判断処理は、スキャン画像の綴じ部境界線を境にした左右ページ毎に実行される。 [Extract page outline from scanned image]
First, the process of extracting the page outline from the scanned image in step S41 will be described. Here, FIG. 12 is an explanatory diagram showing an example of a scan image having a page outline at the upper end, and FIG. 13 is a black pixel histogram on the left side of the binding boundary line of the scan image shown in FIG. The x-axis of the histogram shown in FIG. 13 indicates the main scanning direction (vertical direction in FIG. 12) of the scan image, and the upper end of the scan image is associated with the left end of the histogram. Note that in the case of a scanned image having a page outline at the lower end, the lower end of the scanned image is associated with the right end of the histogram. Accordingly, when a page outline exists at the upper end of the scanned image as shown in FIG. 12, a black band appears at the upper portion of the scanned image, and thus a high vertical bar appears at the left end of the histogram shown in FIG. In this embodiment, it is determined whether or not a page outline exists in the scanned image using such characteristics.
More specifically, as shown in FIG. 13, the distance AO from the binding boundary line to the left end of the scan image (left end in FIG. 12), the height BO of the histogram vertical bar, and the ratio is expressed by the following formula ( 1)

When the calculated ratio k is larger than a predetermined threshold, it is determined that a page outline exists in the scanned image.
In addition, when page outlines exist above and below the scanned image, high vertical bars appear at the left and right ends of the histogram. In such a case, scanning is performed based on the high vertical bars at the left and right ends of the histogram. A determination is made as to whether or not a page outline exists in the image.
Here, the function of the page outline discrimination means is executed.
If it is determined by the above processing that the page outline exists in the scanned image, the page outline is extracted together with information on which of the upper and lower sides of the left and right pages the page outline exists, and temporarily stored in the RAM 33. Remember.
Note that the process of determining whether or not a page outline exists in the scanned image is executed for each of the left and right pages with the binding line boundary line of the scanned image as a boundary.

[Extract ruled lines from scanned images]
In the subsequent step S42, a ruled line extraction process from the scanned image is executed.
The ruled line extraction process from the scanned image in step S42 will be described.
[Detection of ruled line candidates]
Here, FIG. 14 is an explanatory diagram showing an example of a scan image having a long ruled line, and FIG. 15 is a black pixel histogram on the left side of the binding boundary line of the scan image shown in FIG.
The x-axis of the histogram shown in FIG. 15 indicates the main scanning direction (vertical direction in FIG. 14) of the scan image, and the upper end of the scan image is associated with the left end of the histogram. As shown in FIG. 14, when a ruled line exists in the scanned image, a narrow peak appears in the histogram shown in FIG. In the present embodiment, using such characteristics, it is determined whether or not a ruled line exists in the scanned image.
More specifically, first, the height H of a narrow peak appearing in the histogram shown in FIG. 15 is obtained, and the width W at the center position (position where the height is half) of each obtained peak is obtained. If there is a peak whose peak height H is higher than a predetermined threshold thH and whose width W at the center position of the peak is smaller than a predetermined threshold thW, the peak is set as a ruled line candidate. .
Here, the function of ruled line candidate extraction means is executed.
Subsequently, it is further determined whether or not the peak is a ruled line candidate by using the continuity of the ruled line. As shown in FIG. 16, an appropriate position on the candidate ruled line (for example, the position of the center line of the page) is used as a start point, and the candidate ruled line is searched from the start point to the left and right, and the cut point (the ruled line is blurred and interrupted) Are accumulated). If the number of cut points is less than a predetermined threshold, this candidate is determined to be a ruled line. In this way, by determining whether or not a ruled line is based on the ruled line continuity, it becomes possible to eliminate horizontally written character lines, dotted lines, and the like composed of small characters erroneously detected as ruled lines.
Here, the function of the ruled line discriminating means is executed.

[Rectangle detection]
After discriminating ruled lines as described above, the coordinates of each ruled line are detected. As shown in FIG. 17, the ruled line coordinates are detected when the coordinate value in the main scanning direction of the ruled line (the y-axis direction in FIG. 17) is the midpoint coordinate of the black pixel run of the ruled line part. The main scanning direction coordinate value at the left end x1 is y1.
[Select optimal ruled line]
Next, a ruled line optimal for distortion correction is selected from the candidate ruled lines. When a plurality of ruled lines are detected as shown in FIG. 18, it is necessary to select which ruled line is used to correct the distortion. As an example of an optimal ruled line selection criterion, the length of the ruled line is longer than a predetermined threshold value, and the ruled line is placed in the left and right constant width regions (shaded regions in FIG. 18) sandwiching the binding boundary line. The ruled line closest to the upper or lower page outline is selected on the condition that a part is applied. FIG. 18 shows a case where one ruled line is selected from each of the left and right pages. Here, ruled line <1> and ruled line <2> are selected.
Further, another example of the criterion for selecting the optimal ruled line is that the length of the ruled line is longer than a predetermined threshold value and is within a constant width area on the left and right sides of the binding part boundary line (the shaded area in FIG. 19). ) Is part of the ruled line, and the closest ruled line is selected for the top page outline at the top of each page and the bottom page outline at the bottom of each page. FIG. 19 shows a case where the left and right pages are further divided into upper and lower parts, and one ruled line is selected for each of the four blocks. Here, the ruled line <1> is selected in the upper left block, the ruled line <2> is selected in the lower right block, and the ruled line <3> is selected in the lower left block. In the upper right block in FIG. 19, the above two conditions (the length of the ruled line is longer than a predetermined threshold value, and a part of the ruled line is applied to the left and right constant width regions sandwiching the binding boundary line. There is no ruled line selected because there is no ruled line that satisfies
Note that both of the above two conditions (the length of the ruled line is longer than a predetermined threshold and a part of the ruled line is placed in the left and right constant width regions sandwiching the binding boundary line) Alternatively, only one of them may be satisfied. In the above example, “closest to the page outline” is used as the selection criterion. However, the selection criterion is not limited to this, and “the ruled line has the largest curve” may be used. Here, the “bend of the ruled line” is expressed by a difference in coordinate values in the main scanning direction between the left and right end points of the ruled line.

[Determine the coordinate value of the optimal ruled line]
When the optimal ruled line is selected, the coordinate value (in the main scanning direction) of the ruled line is determined. The coordinate value (in the main scanning direction) of the ruled line is determined by approximating and extending the selected ruled line until reaching both ends of the left and right pages. In FIG. 20, for the BC portion where the ruled line exists, the coordinate value has already been determined by the ruled line coordinate detection process described above, so the coordinate value of the ruled line (in the main scanning direction) is determined for the other extended parts. Will do. More specifically, the AB portion shown in FIG. 20 estimates the coordinate value (in the main scanning direction) by linear approximation, and the CD portion estimates the coordinate value (in the main scanning direction) by a polynomial approximation curve.
[Reject inappropriate lines]
Finally, remove inappropriate ruled lines. This is because, when the coordinate value is estimated by polynomial approximation as described above, if the shape of the estimated curve by polynomial approximation is inappropriate, the distortion may increase at the time of correction. It eliminates ruled lines. As an example of an inappropriate approximate curve shape, as shown in FIG. 21, there is a curve <1> in which the curve goes to the outside of the book, or a curve <2> in which the curve goes far beyond the center line. .
If the ruled line is excluded because the shape of the estimated curve is inappropriate, the optimum ruled line is selected again, and the above process is repeated.
If it is determined by the above processing that a ruled line exists in the scanned image, the ruled line is extracted together with information on which position of the ruled line exists on each of the left and right pages, and temporarily stored in the RAM 33.

[Extract text lines from scanned images]
In a succeeding step S43, a character line extraction process from the scanned image is executed. The character line extraction process from the scanned image in step S43 will be described. In the present embodiment, first, it is determined whether the character line in the scanned image is a vertically written character line or a horizontally written character line.
[Determination of character line]
A method for determining whether a character line in a scanned image is a vertically written character line or a horizontally written character line will be described. Here, FIG. 22 is a black-and-white inversion number histogram in the sub-scanning direction of the image shown in FIG. The horizontal axis in FIG. 22 is in the main scanning direction of black pixels in the sub-scanning direction (left-right direction) (pixels whose density value is darker than a predetermined density value among pixels obtained by inverting the scan image in black and white). The vertical axis in FIG. 22 indicates the number of black pixels for each position. FIG. 23 is a black-and-white inversion number histogram of the image shown in FIG. 8 in the main scanning direction. The horizontal axis in FIG. 23 represents the black pixel in the main scanning direction (vertical direction) (the pixel whose density value is darker than the predetermined density value among the pixels obtained by inverting the scan image in black and white) in the sub-scanning direction. The vertical axis in FIG. 23 indicates the number of black pixels for each position. When the characters in the image are horizontal scans as shown in FIG. 8, the histogram in the sub-scan direction as shown in FIG. 22 changes drastically, but the change in the histogram in the main scan direction as shown in FIG. Few. Although not particularly illustrated, when the character line in the scanned image is a vertically written character line, the histogram in the main scanning direction changes drastically, but the change in the histogram in the sub-scanning direction is small.

主走査方向ｙの位置でのヒストグラム値Ｐｎｔ（ｙ）の平均値ｍｅａｎ_Ｈが算出される。ここで、ｈｅｉｇｈｔは画像の高さである。そして、下記に示す式（３）により、

副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈが得られる。
同様に、下記に示す式（４）により、

副走査方向ｘの位置でのヒストグラム値Ｐｎｔ（ｘ）の平均値ｍｅａｎ_Ｖが算出される。ここで、ｗｉｄｔｈは画像の幅である。そして、下記に示す式（５）により、

主走査方向のヒストグラムの副走査方向に関する分散σ_Ｖが得られる。
上述したようにスキャン画像中の文字行が横書き文字行である場合には、副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈが、主走査方向のヒストグラムの副走査方向に関する分散σ_Ｖより大きい。逆に、スキャン画像中の文字行が縦書き文字行である場合には、主走査方向のヒストグラムの副走査方向に関する分散σ_Ｖが、副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈより大きい。つまり、分散σ_Ｈと分散σ_Ｖとの比較により、スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別が可能になっている。
なお、スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別に、黒白反転数ヒストグラムを用いたのは、文字行と写真部分との混同を避けるためである。一般に、黒画素ヒストグラムの値が同程度の場合、文字領域のほうが写真領域よりも黒白反転数ヒストグラムの値が大きくなるからである。
ここに、文書判別手段の機能が実行される。 Specifically, the discrimination method as described above is realized by the following equations. First, according to the following equation (2):

An average value mean _H of the histogram values Pnt (y) at the position in the main scanning direction y is calculated. Here, height is the height of the image. And according to the equation (3) shown below,

A variance σ _{H in} the main scanning direction of the histogram in the sub scanning direction is obtained.
Similarly, according to equation (4) below:

An average value mean _V of the histogram values Pnt (x) at the position in the sub-scanning direction x is calculated. Here, width is the width of the image. And according to the equation (5) shown below,

A variance σ _{V in} the sub-scanning direction of the histogram in the main scanning direction is obtained.
As described above, when the character line in the scanned image is a horizontally written character line, the variance σ _H in the main scanning direction of the histogram in the sub-scanning direction is larger than the variance σ _{V in} the sub-scanning direction of the histogram in the main scanning direction. . Conversely, when the character line in the scanned image is a vertically written character line, the variance σ _V in the sub-scanning direction of the histogram in the main scanning direction is larger than the variance σ _{H in} the main scanning direction of the histogram in the sub-scanning direction. . That is, by comparing the variance σ _H and the variance σ _V , it is possible to determine whether the character line in the scanned image is a vertically written character line or a horizontally written character line.
The reason why the black and white inversion number histogram is used to determine whether the character line in the scanned image is a vertically written character line or a horizontally written character line is to avoid confusion between the character line and the photograph portion. This is because, in general, when the values of the black pixel histogram are approximately the same, the value of the black / white inversion number histogram is larger in the character region than in the photo region.
Here, the function of the document discrimination means is executed.

[Coordinate detection of horizontal text lines]
After determining the character line as described above, first, the coordinates of each horizontal character line are detected. When detecting the coordinates of a horizontally written character line, a circumscribed rectangle extracting process for each character used for the character recognition process is performed and a horizontally writing character line is extracted. Since the character recognition process is a well-known technique, its description is omitted. Here, FIG. 24 shows an example of the result of the character circumscribed rectangle extraction process and the character line extraction process of the scanned image. Then, the coordinates of the center point of the circumscribed rectangle of each character are regarded as the coordinates of the character, and the coordinates of the horizontal character line are detected.
[Select optimum horizontal text line]
Next, a horizontal character line optimal for distortion correction is selected from the extracted horizontal character lines.
When a plurality of horizontally written character lines are detected, it is necessary to select which horizontally written character line is used for distortion correction. An example of the optimum horizontal writing character line selection criterion is basically the same as the above-mentioned optimum ruled line selection criterion, and the horizontal writing character line length BC is larger than a predetermined threshold as shown in FIG. On the condition that a part C of a horizontal writing line is covered within a long and constant width area (shaded area in FIG. 25) on either side of the binding boundary line, either upper or lower page Select the horizontal text line closest to the outline. Here, B is the center of the leftmost rectangle of the character line, and C is the center of the rightmost rectangle. The optimum horizontal writing character line may be selected by selecting the horizontal writing character line closest to the page outline from each of the left and right pages, or the left and right pages are further divided into upper and lower parts. A horizontally written character line that is closest to the page outline one by one in the block may be selected.
Regarding the above two conditions (the length of the horizontal writing character line is longer than a predetermined threshold, and a part of the horizontal writing character line is applied within the left and right constant width regions across the binding portion boundary line). , Not both of them, but only one of them may be satisfied. In the above example, “closest to the page outline” is used as the selection criterion. However, the selection criterion is not limited to this, and “horizontal writing character line has the largest curve” may be used. Here, “curve of horizontally written character line” is expressed by the difference in the coordinate values in the main scanning direction of the center coordinates of the character circumscribed rectangle at both ends of the horizontally written character line.

[Determine the coordinate value of the optimal horizontal text line]
When the optimum horizontal writing character line is selected, the coordinate value (in the main scanning direction) of the horizontal writing character line is determined. The coordinate value (in the main scanning direction) of the horizontal writing line is obtained by connecting the center points of the circumscribed rectangles in the horizontal writing line and approximating the straight line part and the curved part and extracting them (main scanning). The coordinate value of the direction will be determined. More specifically, D shown in FIG. 25 is a binding boundary line, a coordinate value (in the main scanning direction) is estimated with a polynomial approximation curve between BDs, and an approximation between A and B at the leftmost end A coordinate value (in the main scanning direction) is estimated with a straight line value.
[Reject inappropriate horizontal text]
Finally, remove inappropriate horizontal text lines. This is because, when the coordinate value is estimated by polynomial approximation as described above, if the shape of the estimated curve by polynomial approximation is inappropriate, the distortion may increase at the time of correction. This eliminates horizontal text lines. An example of an inappropriate approximate curve shape is the same as in the case of the ruled line described above, and although not particularly illustrated, the curve is directed outwardly from the book, or is greatly inwardly penetrated beyond the center line. Is the case.
If the horizontal writing character line is excluded because the shape of the estimated curve is inappropriate, the optimal horizontal writing character line is selected again, and the above processing is repeated.
If it is determined by the above processing that there is a horizontally written character line in the scanned image, the horizontally written character line is extracted together with information indicating in which position the horizontally written character line exists on each of the left and right pages, and stored in the RAM 33. Memorize temporarily.
[Detection of coordinates of each character in vertical writing]
Subsequently, the coordinates of each vertically written character line are detected. When detecting the coordinates of a vertically written character line, a circumscribed rectangle extraction process for each character used in the character recognition process is performed. Since the character recognition process is a well-known technique, its description is omitted. Then, the coordinates of the center point of the circumscribed rectangle of each character are regarded as the coordinates of the character.
[Extract horizontal text line]
Next, a horizontal writing character line is extracted from each vertical writing character line. In the horizontal character line extraction, an outer shape obtained by concatenating the uppermost characters of each vertical character line or an outer shape obtained by concatenating the lowermost characters of each vertical character line is regarded as a horizontal character line. More specifically, when one character line is selected from each of the left and right pages, the outline obtained by concatenating each character at the top or bottom of each vertical character line is regarded as one line and extracted. . In addition, when the left and right pages are further divided into upper and lower parts, and one character line is selected for each of the four blocks, the top left and upper right parts are connected to each character at the top of each vertically written character line. Are extracted as a single line in the lower left part and the lower right part.
At this time, as shown in FIG. 26, the length BC of the extracted outer shape is longer than a predetermined threshold value, and within the left and right constant width regions (shaded regions in FIG. 26) sandwiching the binding boundary line. The contour is extracted on condition that a part C of the extracted contour is applied. Note that the above two conditions (the length of the extracted outer shape is longer than a predetermined threshold and a part of the extracted outer shape is applied to the left and right constant width regions sandwiching the binding portion boundary line) May satisfy only one of them instead of both. Therefore, if the above condition is not satisfied, there is no outline of the vertically written character line for distortion correction.

[Determine the coordinate value of the extracted outline]
When the outer shape is extracted according to the vertical writing character line, the coordinate value (in the main scanning direction) of the outer shape of the vertical writing character line is determined. As shown in FIG. 27, the coordinate value of the outer shape of the vertically written character line is the center of the upper side of the circumscribed rectangle of each character to be connected when the uppermost character of each vertically written character line is connected. Coordinate values (in the main scanning direction) of the outline of a vertically written character line are determined by connecting points and approximating and extracting a straight line portion and a curved line portion. In addition, as shown in FIG. 27, in the case of the outer shape in which each character at the bottom of each vertically written character line is connected, the lower side center point of the circumscribed rectangle of each character to be connected is connected, and the straight line portion and the curve are connected. By approximating and extracting the part, the coordinate value (in the main scanning direction) of the outline of the vertically written character line is determined. More specifically, D shown in FIG. 26 is a binding portion boundary line, a coordinate value (in the main scanning direction) is estimated with a polynomial approximation curve between BDs, and an approximation between A and B at the left end is approximated. A coordinate value (in the main scanning direction) is estimated with a straight line value.
[Eliminating inappropriate vertical text lines]
Lastly, eliminate inappropriate vertical text lines. This is because, when the coordinate value is estimated by polynomial approximation as described above, if the shape of the estimated curve by polynomial approximation is inappropriate, the distortion may increase at the time of correction. This eliminates the outline of vertical character lines. Examples of inappropriate approximate curve shapes are the same as in the case of the ruled lines and horizontal text lines described above, and although not particularly shown, when the curve goes to the outside of the book or greatly beyond the center line It is a case where it digs into.
If the outline of the vertically written character line is excluded because the shape of the estimated curve is inappropriate, it means that there is no outline of the vertically written character line for distortion correction.
If it is determined by the above processing that the outline of the vertical text line exists in the scanned image, the vertical text will be displayed along with information on where the vertical text line outline exists on the left and right pages. The outline of the row is extracted and temporarily stored in the RAM 33.
In the following description, the outer shape of a horizontal character line and a vertical character line is treated as a character string.
As described above, the page outline / ruled line / character line extraction process (step S4) is completed by the processes of steps S41 to S43.

In a subsequent step S5 (see FIG. 6), an image distortion correction process is executed. As shown in FIG. 28, the image distortion correction processing is roughly performed as a page outline positioned near the upper side (or lower side) of the scanned image as a reference line (reference line 1) for distortion correction (expansion). A process of selecting one of / ruled line / character line (step S51: reference line 1 selection process), which corresponds to the reference line 1 and is used as a reference line 2 for calculating a correction rate (expansion rate). A process of selecting one of page outline / ruled line / character line located near the upper side (or lower side) of the page (step S52: reference line 2 selection process), and expands to a scanned image based on the reference line 1 and the reference line 2 Processing to correct distortion in the main scanning direction, and the distance (floating height) from the scan plane of the page plane calculated from the reference line 1 or the reference line 2 or the character circumscribed rectangle of the corrected image. Z Process of correcting the distortion in the sub-scanning direction is subjected to decompression processing in the scan image Te: is composed of (step S53 main scanning direction distortion correction process and the sub-scanning direction distortion correction processing) and.
Here, in the reference line 1 selection process (step S51) and the reference line 2 selection process (step S52), the page outline / position located near the upper side (or the lower side) of the scanned image as the reference line 1 or the reference line 2. One of the ruled line / character line is selected for each of the left and right pages. The priority order of selection of the page outline, ruled line, and character line in this embodiment is as follows.
Page outline> ruled line> text line. This selection priority is used because character lines have lower extraction accuracy than page outlines and ruled lines, and more accurate distortion correction rates can be obtained by using page outlines outside the image. This is because it can.

Next, the main scanning direction distortion correction process and the sub scanning direction distortion correction process (step S53) will be described. This process differs depending on whether the scanner parameter is known or unknown. That is, when the scanner model is known, the position of the optical axis of the scanner lens of the scanner in the main scanning direction (Ak in FIG. 29) and the distance between the center of the scanner lens and the scanning plane (focal plane distance in FIG. 29) Are previously stored as scanner parameters.
First, the case where the scanner parameters are known will be described. In this case, basically, if one line in the main scanning direction is corrected, then correction in the sub-scanning direction is performed for that line next to all the lines in the main scanning direction, or the reference line 1 or the reference line 2. This is performed for the main scanning direction line of the curved portion.
Correction in the main scanning direction is performed as follows. In FIG. 30, the reference line 1 and the reference line 2 before correction are represented by solid lines, and the corrected line is represented by dotted lines. The corrected dotted line is obtained by extending the straight portions (flat portions) of the reference line 1 and the reference line 2 before correction as they are. Here, at the position X in the sub-scanning direction, the point P on the reference line is corrected to P ′, and the point Q on the reference line is corrected to Q ′. Now, assuming that an arbitrary point Y in the main scanning direction at the position X is corrected to Y ′, the following relational expression holds:
YP / YQ = Y'P '/ Y'Q'
Therefore, if the position of each point in the main scanning direction is expressed as P (y) for the point P,
(Y (y) -P (y)) / (Y (y) -Q (y)) =
(Y ′ (y) −P ′ (y)) / (Y ′ (y) −Q ′ (y))
So, transform this,
Y (y) =
((P (y) −Q (y)) / (P ′ (y) −Q ′ (y))) Y ′ (y) +
(P ′ (y) Q (y) −P (y) Q ′ (y)) / (P ′ (y) −Q ′ (y))
It becomes.
Using the above equation, the position Y (y) before correction of the point that should come to the position of Y ′ (y) after correction can be obtained. That is, the pixel value of Y (y) before correction may be moved to Y ′ (y) after correction. However, since the calculated value of Y (y) is generally a decimal, a linear interpolation result of pixel values corresponding to integer positions before and after that is used. That is, in FIG. 31, if the integers before and after Y (y) are N and N + 1, and the corresponding pixel values are D (N) and D (N + 1), the pixel value D (Y (Y (y)) y)) is calculated so as to satisfy the linear relationship shown in the figure.

The above is a case where all of the reference line 1 and the reference line 2 are any one of the page outline / ruled line / character line. If an appropriate page outline, ruled line, and character line are not extracted, the following is performed. To do. FIG. 32 shows an example in which no page outline / ruled line / character line exists above the right page. In this case, the locus of the lens optical axis (L in FIG. 32) is regarded as the reference line 2 and the same processing as described above is performed. In this case, however, Q and Q ′ in FIG. If there is no page outline / ruled line / character line at the bottom of the page, L is regarded as the reference line 1 (P and P ′ match). If there is no page outline / ruled line / character line at the top / bottom of the page, there is no clue and the page is not corrected.
The above processing is executed independently for each of the left and right pages of the scanned image. Moreover, the information about the reference line 1 and the reference line 2 in a certain spread left and right page may be held, and this may be used to correct other spread left and right pages. For example, when scanning a continuous spread page, the spread left and right pages are considered as one set, and the extraction of the reference line 1 and the reference line 2 is performed only once in 5 sets. If 2 is extracted, the information may be used as it is in the subsequent 4 sets. Thereby, time for extracting the reference line 1 and the reference line 2 can be saved, so that the overall processing speed is improved.

Correction in the sub-scanning direction is performed as follows. That is, using the amount of change in the distance of the page surface from the scan surface, the image is expanded by expanding the book original so that the cross-sectional curve is a straight line. Specifically, the image is expanded based on the amount of change in the height of the page surface from the scan surface. As shown in FIG. 33, the shape of the book manuscript page surface is read to form a minute triangle for each line, and the image length Ln in one line of page reading is expressed by the following equation: Ln = square root {1+ (Tn−Tn−1) ** 2}
And the cumulative image length Ln is taken as the page expansion length. As a result, the hypotenuse of the approximate triangle becomes almost equal to the shape of the curved page (approximate polygonal line), and the accurate page length is obtained by setting the accumulation as the image length of the page. In particular, the accuracy of correction of the length is high due to the shape approximation with the minimum pitch for each line.
By calculating the pixel interval of the target pixel on the adjacent line in the main scanning direction as the reference “1”, the reading line interval changes, that is, the sampling pixel interval is sequentially changed with respect to the original when it is flat. On the other hand, the image expansion process is applied. Also, geometrically, the image projection double length in the main scanning direction and the distance from the scan surface of the page surface are proportional to each other, and as shown in FIG. 29, the address A3 of the reference line 1 or the reference line 2 in the main scanning direction. And the distance T from the scan plane of the page plane from the address Ka of the straight line portion (flat portion) of the reference line 1 or the reference line 2 is expressed by the following equation: T = focal plane distance * (A3-Ak) / {(Ak-Ka )-(A3-Ak)}
Can be obtained. The correction position of the page can be calculated by approximating a straight line with a minute pitch for each pixel based on the difference in the depth of the adjacent main scanning lines calculated in this way. Note that the method of obtaining the reference line 1 / reference line 2 and the correspondence when there is no appropriate reference line 1 / reference line 2 are basically the same as in the case of correction in the main scanning direction.

The distance T of the page plane calculated in this way from the scan plane is used to calculate the image length Ln in the sub-scanning direction to be restored for each line, as shown in FIG. That is, the following equation Ln = square root {1+ (Tn−Tn−1) ** 2}
Thus, the image length Ln in the sub-scanning direction to be corrected for each line is calculated. Therefore, the accumulation of the image length Ln becomes the page length in the sub-scanning direction.
Normally, the amount of change in the height increases as the distance from the scan surface of the page surface increases. Therefore, in the present embodiment, image expansion is limited according to the distance of the page surface at that position from the scan surface. For example, the difference between adjacent boundary addresses is limited to (height [mm] / 5) [pixel] at that position to suppress errors in sub-scanning direction correction due to erroneous detection of the boundary.
In correction in the sub-scanning direction, the image length Ln differs depending on the position, and the positions of corresponding points in the original image and the corrected image are generally different as shown in FIG. When the point existing in N is corrected to N ′ and the point N + 1 is corrected to N + 1 ′, the pixel value to be obtained is the pixel value at the position of the integer X ′ (which generally does not always match N + 1 ′). . When the distance between N ′ and X ′ is a, and the distance between X ′ and N + 1 ′ is b, the corresponding point X ′ of X ′ in the original image is
(X−N) / (N + 1−X) = a / b
Are in a relationship.
If the position of X is known from the above equation, the X pixel value before correction may be moved to X ′ after correction. However, since the calculated value of X is generally a decimal, a linear interpolation result of pixel values corresponding to integer positions before and after that is used. That is, in FIG. 35, assuming that the pixel values corresponding to N and N + 1 before and after X are D (N) and D (N + 1), the pixel value D (X) in X satisfies the linear relationship as shown in the figure. Calculate as follows. The above processing is performed for all positions in the sub-scanning direction, or for positions in the sub-scanning direction where the reference line 1 or the reference line 2 is a curve.
The above processing is executed independently for each of the left and right pages of the scanned image. In addition, the distance from the scanning surface of the book document in a certain spread left and right page and the information about the reference line 1 or the reference line 2 may be held and used for correcting other spread left and right pages. For example, when scanning a continuous spread page, the spread left and right pages are considered as one set, and the calculation of the distance from the scan surface of the book document and the extraction of the reference line 1 and the reference line 2 are performed only once in 5 sets. If the calculation of the distance and the extraction of the reference line 1 and the reference line 2 are performed in a certain set, the information may be used as it is in the subsequent four sets. This saves time for calculating the distance from the scan surface of the book document and extracting the reference line 1 and the reference line 2, thereby improving the overall processing speed.
This is the description of the main scanning direction distortion correction processing and the sub scanning direction distortion correction processing (step S53) in the case where the scanner parameters are known.

Next, a case where the scanner parameter is unknown will be described.
The correction in the main scanning direction is basically the same as when the scanner parameters are known, but the processing when an appropriate page outline, ruled line, and character line are not extracted is slightly different. In FIG. 32, when the scanner parameter is known, the locus of the lens optical axis (L in FIG. 32) is regarded as the reference line 2, but when unknown (FIG. 36), the center line (FIG. 36) parallel to the sub-scanning direction. C) is considered as reference line 2. Also in this case, Q and Q ′ in FIG. 30 coincide. If there is no page outline / ruled line / character line below the page, C is regarded as the reference line 1 and the same is done (P and P ′ match). If there is no page outline / ruled line / character line at the top / bottom of the page, there is no clue and the page is not corrected.
Next, sub-scanning direction distortion correction processing will be described. Here, FIG. 37 is a flowchart schematically showing the flow of the sub-scanning direction distortion correction processing. As shown in FIG. 37, in step S101, a circumscribed rectangle A (see FIG. 38) of a character is extracted by a method used for character recognition processing based on the scanned image corrected for distortion in the main scanning direction. Here, since the character recognition process is a well-known technique, the description thereof is omitted. The reason why the circumscribed rectangle A of the character is extracted in this way is to correct the distortion in the sub-scanning direction based on the change in the shape of the circumscribed rectangle A of the character. Here, as shown in FIG. 38, the horizontal side length w, the vertical side length h, and the character center B of the character circumscribing rectangle A are defined. Here, the center B of the character is the intersection of the diagonal lines of the circumscribed rectangle A.
Subsequently, as shown in FIG. 39, after the scanned image is divided into a plurality of strip-shaped regions C in a direction parallel to the page binding portion 41 of the book document 40 (step S102), each strip region C is included therein. A feature amount relating to the included character circumscribing rectangle A is obtained (step S103). Here, the circumscribed rectangle A included in a strip area C is a circumscribed rectangle A whose center is included in the strip area C. For example, the circumscribed rectangle A included in the strip region C1 of FIG. 39 is a shaded rectangle in the drawing.

Now, the feature amount related to the character circumscribed rectangle A is as follows.
(Length of horizontal side of character) / (Length of vertical side of character) = w / h
Required based on That is, for each strip region C, the average value of w / h values of all the character circumscribed rectangles A included therein is used as the feature amount of the strip region C.
However, simply calculating the average value of w / h may be inappropriate. Some characters, such as punctuation marks and symbols in mathematical formulas, are originally small in size and have unstable w / h values. In addition, adjacent characters may be extracted when the rectangle is extracted, resulting in a character circumscribing rectangle A having an extremely large w. When obtaining the feature amount, it is necessary to exclude such special characters or extremely large w in advance. Therefore, in the subsequent step S104, a threshold value is determined in advance, and the character circumscribed rectangle A whose h value is smaller than the threshold value is excluded in advance, and a threshold value regarding the ratio of w / h is determined in advance. Character circumscribing rectangle A whose value is larger than the threshold is also excluded in advance. For example, the circumscribed rectangle A that is shaded in FIG. 40 is excluded in advance.
In the subsequent step S105, after eliminating the extreme character circumscribing rectangle A as described above, the average value of w / h of the character circumscribing rectangle A in each strip region C is obtained. FIG. 41 shows an example of the average value of w / h of the circumscribed rectangle A in each strip area C. Note that a strip region C2 in FIG. 41 is a strip region including a page binding portion.
Subsequently, it is determined whether or not the character circumscribing rectangle A exists in the strip area C2 including the page binding portion (step S106). This is because, as shown in FIG. 40, generally, there are many cases where the character circumscribed rectangle A does not exist near the page binding portion. If the character circumscribing rectangle A exists in the strip area C2 including the page binding portion (Y in step S106), the feature amount is calculated using the character circumscribing rectangle A, and the process directly proceeds to step S108.
On the other hand, when the character circumscribing rectangle A does not exist in the strip region C2 including the page binding portion (N in step S106), the process proceeds to step S107, and the feature amount of the strip region C2 including the page binding portion is obtained. The strip region C2 including the page binding portion is identified by, for example, obtaining a background density change of a scanned image (for example, a monochrome multi-valued image) for each strip region C, and having the lightest density value in the strip region C. This is realized by obtaining

FIG. 42 shows an example of obtaining the background density change, and the strip area having the highest background density is regarded as the strip area C2 including the page binding portion.
Note that when the scanned image is a color multi-valued image, the strip region C2 including the page binding portion is identified by focusing on one of the RGB components (for example, the G component) and using the background density of the G component. To identify them. Alternatively, RGB may be color-converted to be divided into a luminance component and a color difference component, and the strip region C2 including the page binding portion may be identified using the luminance component.
The feature amount of the strip region C2 including the page binding portion is determined as follows. Here, there is a character circumscribing rectangle A that can be a statistical feature quantity calculation target, and a predetermined constant value is set for the feature quantity of the strip area C that is the nearest to the strip area C2 including the page binding portion. A value calculated by multiplication is regarded as a feature amount in the strip region C2 including the page binding portion. That is, in the example shown in FIG. 41, since the character circumscribed rectangle A exists in any of the strip regions C3 and C4 on the left and right sides of the strip region C2 including the page binding portion, the appropriate feature amount is selected ( Here, the circle on the right side) is multiplied by a predetermined constant value (0.5 in this case), and this is used as the feature amount of the strip region C2 including the page binding portion.
In the subsequent step S108, an appropriate filtering process for the feature amount of each strip area C, for example, a process for obtaining a moving average in the direction of change of the position of the strip area C (ie, the sub-scanning direction) is performed. The change in the feature amount (in the sub-scanning direction) with respect to the change in the position is made gentle. Here, however, special processing is required in the vicinity of the page binding portion. This is because if the filtering is performed using windows whose lengths are all equal in the sub-scanning direction, the sharpness of the feature amount change near the page binding portion is lost.

Here, FIG. 43 shows the result of performing the filtering process on the feature amount of each strip region C shown in FIG. 41 using a window whose length is all five. As shown in FIG. 43, when the filtering process is performed using a window whose length is all 5, the change in the feature amount (w / h) near the page binding portion becomes too gentle. In such a case, appropriate image correction near the page binding portion becomes impossible.
Therefore, in the present embodiment, during the filtering process, the window length is set near the page binding portion so that the filter window does not straddle the strip regions C3 and C4 on both sides of the strip region C2 including the page binding portion. Adjust. Here, FIG. 44 is a graph showing the result of performing the filtering process by adjusting the window length in the vicinity of the page binding portion. As shown in FIG. 44, when the window length is adjusted in the vicinity of the page binding portion, the change in the feature amount (w / h) in the vicinity of the page binding portion can be appropriately expressed, so that favorable image correction can be realized.
In subsequent step S109, an estimated distortion amount of each strip region C is calculated. The calculation method of the estimated distortion amount of each strip region C is as follows.
First, a strip area (reference strip area) is defined as a reference for calculating the distortion amount of the strip area. Here, the strip region C that is considered to have the smallest distortion, for example, the strip region C having the maximum feature (w / h) is set as the reference strip region. This process may be performed in common on the left and right pages, but the reference strip area may be determined independently on the left and right. FIG. 44 shows an example in which the reference strip area is defined independently on the left and right sides. The strip area C marked with a circle is the reference strip area, the reference feature on the left is “Lw0 / Lh0”, and the reference reference on the right The feature amounts are indicated by “Rw0 / Rh0”, respectively.

Next, the feature amount w0 / h0 of the reference strip region is set as the reference feature amount of the entire scanned image,
(Feature amount of each strip area) / (reference feature amount) = (w / h) / (w0 / h0)
Is calculated as an estimated amount of distortion in each strip region.
If the strip area C near the outside of the page outside the page binding portion is used as the reference strip region, there may be a case where a font or type size difference is large from the vicinity of the page binding portion, and an appropriate estimated distortion amount cannot be calculated. It is done. When such an image is targeted, it is effective to limit the search range of the reference strip region to the vicinity of the page binding portion in advance. In order to realize this, the reference line 1 or the reference line 2 may be performed only for the position in the sub-scanning direction where the reference line 1 is a curve.
Finally, enlargement processing in the short side direction (sub-scanning direction) of the strip region C is performed on the scanned image to correct distortion near the page binding portion (step S110). In this case, the enlargement ratio (distortion correction ratio (expansion ratio) in the sub-scanning direction) is the reciprocal of the estimated distortion amount calculated in step S109, that is,
(Reference feature value) / (Feature value of each strip area) = (w0 / h0) / (w / h)
And Here, if the above-mentioned standard strip area is defined to be common to the left and right, this enlargement ratio is also calculated based on the standard feature quantity common to the left and right. To do. FIG. 45 shows the corrected enlargement ratio calculated based on the feature amount shown in FIG.
Here, the function of the sub-scanning direction distortion correction rate calculation means and the function of the sub-scanning direction distortion correction means are executed.
In this case as well, since the strip area C far from the vicinity of the page binding portion is likely to be an area without distortion of the image, there is a case where it is better not to make it an object of enlargement processing. This is because an unnatural distortion may occur due to the enlargement process. In order to prevent this, it is only necessary to enlarge only the position in the sub-scanning direction of the portion where the reference line 1 or the reference line 2 is a curve. That is, when the reference line 1 or the reference line 2 is a straight line portion, the estimated distortion amount is “1”.
Further, when a common correction magnification ratio is applied in the strip area C, the correction magnification ratio becomes discontinuous at the boundary between adjacent strip areas C, and the correction image becomes unnatural. Therefore, the correction magnification rate is corrected in advance so that the correction magnification rate at the boundary between adjacent strip regions C changes continuously. For example, the correction magnification rate of the central portion of the strip region C shown in FIG. 45 is plotted as a point indicating the reciprocal of the estimated distortion amount, and these points are connected by a line segment to complement a straight line, This can be realized by setting the correction magnification. Through the above processing, the correction magnification rate in the sub-scanning direction of the scanned image is determined.

The above processing is executed independently for each of the left and right pages of the scanned image. Further, information about the correction magnification ratio in a certain spread left and right page may be held and used for correcting other spread left and right pages. For example, when scanning a continuous spread page, it is assumed that the spread left and right pages are one set, and the correction enlargement ratio is calculated only once in five sets. In four sets, the information may be used as it is. This saves time for calculating the corrected enlargement ratio, thereby improving the overall processing speed.
As described above, the sub-scanning direction distortion correction processing (step S55) is completed by the processing in steps S101 to S110, and the distortion correction processing for the scanned image shown in FIG. Here, FIG. 46 is a plan view showing an image with distortion corrected. According to the above processing, the distortion of the scanned image that has occurred in the vicinity of the page binding portion as shown in FIG. 8 is corrected as shown in FIG.
As described above, when the scanner parameters are unknown, the correction in the sub-scanning direction is basically performed using the character circumscribed rectangle after the correction in the main scanning direction is completed for all lines.

It is a longitudinal cross-sectional view which shows the structure of the scanner part of this invention. It is a perspective view which shows the upper part of the digital copying machine carrying a scanner part. It is a block diagram which shows the electrical connection of the control system of a scanner part. It is a block diagram which shows the basic internal structure of the image processing part of this invention. It is a block diagram which shows the electrical connection of a main control part. It is a flowchart which shows roughly the flow of the distortion correction process of a scanned image. It is a perspective view showing a state when reading a book document. It is a top view which shows an example of the input image. It is a figure explaining the part which distortion arises in a scan image. It is a figure which shows the black pixel histogram of the binding part boundary line left side of the image shown in FIG. It is a flowchart which shows roughly the flow of the extraction process of a page external shape / ruled line / character line. It is explanatory drawing which shows an example of the scanning image which has a page external shape in an upper end. It is a figure which shows the black pixel histogram of the binding part boundary line left side of the scan image shown in FIG. It is explanatory drawing which shows an example of the scanning image in which a long ruled line exists. It is a figure which shows the black pixel histogram of the binding part boundary line left side of the scan image shown in FIG. It is a figure explaining judgment whether it is a ruled line based on ruled line continuity. It is a figure explaining the coordinate detection of a ruled line. It is explanatory drawing which shows the case where one ruled line is selected from each of the left and right pages. It is explanatory drawing which shows the case where a left-right page is further divided | segmented into an up-and-down part and one ruled line is selected in each of 4 blocks. It is explanatory drawing which shows the determination of the coordinate value of the extended optimal ruled line. It is explanatory drawing which shows the example of an inappropriate approximate ruled line. FIG. 9 is a black and white inversion number histogram of the image shown in FIG. 8 in the sub-scanning direction. It is a figure which shows the black-and-white inversion number histogram of the main scanning direction of the image shown in FIG. It is a figure which shows an example of the result of the character circumscribed rectangle extraction process of a scan image, and a character line extraction process. It is explanatory drawing which shows selection of the optimal horizontal writing character line. It is explanatory drawing which shows extraction of the horizontal writing character line from a vertical writing character line. It is explanatory drawing which shows the determination of the coordinate value of the external shape of a vertical writing character line. It is a flowchart which shows the flow of an image distortion correction process roughly. It is a figure explaining a scanner parameter. It is explanatory drawing which shows the reference line before and behind correction | amendment of the main scanning direction, and a reference line. It is explanatory drawing which shows the linear interpolation of the pixel value corresponding to an integer position. It is explanatory drawing which shows the case where a page outline / ruled line / character line does not exist above the right page. It is explanatory drawing which shows the approximation of the shape of the curved page. It is a figure which shows the position of the corresponding point of an original image and a correction image. It is explanatory drawing which shows the linear interpolation of the pixel value corresponding to an integer position. It is explanatory drawing which shows the reference line before and behind correction | amendment of the main scanning direction in case a scanner parameter is known, and a reference line. It is a flowchart which shows roughly the flow of a subscanning direction distortion correction process. It is explanatory drawing which shows the circumscribed rectangle of the character based on a scanning image. It is a figure which shows the state which divided | segmented the scanned image into the several strip-shaped area | region of the direction parallel to the page binding part of a book original. It is explanatory drawing which shows the character circumscribing rectangle to exclude. It is a figure which shows an example of the average value of the feature-value of the circumscribed rectangle in each strip area | region. It is a figure which shows an example of the background density | concentration change for every strip area | region. It is a figure which shows the result of having performed the filtering process with respect to the feature-value of each strip area | region shown in FIG. It is a figure which shows the result of having adjusted the window length in the page binding part vicinity, and having performed the filtering process. It is a figure which shows the correction | amendment expansion rate computed based on the feature-value shown in FIG. It is a top view which shows the image which correct | amended distortion. FIG. 6 is a front view showing a state in which a page binding portion of a book document is lifted from a contact glass.

Explanation of symbols

1 scanner unit, 2 contact glass, 16 image forming apparatus, 20 image processing unit, 29 image distortion correcting unit, 40 book document, 41 page binding unit

Claims (8)

Distortion correction is performed by extending a scan image obtained by reading a book original image placed on the scan surface in the main scanning direction and the sub-scanning direction so that the page binding portion is substantially parallel to the main scanning direction. In the image distortion correction apparatus, a lift amount calculation unit that obtains a distance between a book document and a scan surface from the scanned image and optical axis position information of a scanner lens that has read the scan image, and a distance calculated by the lift amount calculation unit. Sub-scanning direction distortion correction means for correcting distortion of the scanned image based on,
The sub-scanning direction distortion correction unit stores the distance between the book document and the scan surface calculated by the lift amount calculation unit, and stores the stored distance for a predetermined number of pages scanned continuously. An image distortion correction apparatus , wherein distortion correction in the sub-scanning direction is used .   The sub-scanning direction distortion correction unit approximates the cross-sectional curve of the book document based on a distance from the book document and the scan surface, and corrects the accumulated value to be the length of the corrected image in the sub-scanning direction. The image distortion correction apparatus according to claim 1, wherein the correction is performed by obtaining front and rear corresponding points. The sub-scanning direction distortion correction means, the image distortion correction apparatus according to claim 1 or 2, characterized in that corrected left and right for each page of the scanned image. An image reading unit that reads a book document image placed on a scan surface and corrects the scanned image read by the image reading unit so that the page binding portion is substantially parallel to the main scanning direction. image reading apparatus is characterized in that an image distortion correction apparatus according to any one of 3. An image reading unit that reads a book document image placed on a scan surface and corrects the scanned image read by the image reading unit so that the page binding portion is substantially parallel to the main scanning direction. an image distortion correction apparatus according to any one of 3, an image forming apparatus, characterized in that said image distortion correction apparatus is an image printing apparatus for printing on a sheet an image based on the image data corrected . A program for causing a computer to perform distortion correction by extending a scan image obtained by reading a book document image placed on a scan surface in the sub-scan direction so that the page binding portion is substantially parallel to the main scan direction. The lift amount calculating means for obtaining the distance between the book document and the scan surface from the scanned image and the optical axis position information of the scanner lens that has read the scan image on the computer, and the distance calculated by the lift amount calculating means. Based on this, the distortion of the scanned image is corrected , the distance between the book document and the scan surface calculated by the lifting amount calculation unit is stored, and the stored number of pages is scanned for a predetermined number of pages to be scanned continuously. A program for functioning as sub-scanning direction distortion correcting means for correcting distortion in the sub-scanning direction using a distance . The sub-scanning direction distortion correction unit approximates the cross-sectional curve of the book document based on a distance from the book document and the scan surface, and corrects the accumulated value to be the length of the corrected image in the sub-scanning direction. 7. The program according to claim 6, wherein correction is performed by obtaining corresponding points before and after. The program according to claim 6 or 7 , wherein the sub-scanning direction distortion correction unit performs correction for each of right and left pages of the scanned image.

Images