JP4145256B2 - Image distortion correction apparatus, program, and storage medium - Google Patents

Description

  The present invention relates to an image distortion correction apparatus, a program, and a storage medium for correcting distortion of a scanned image obtained by reading a book original image by an image reading unit.

  Many originals read using a flatbed scanner are sheet-like originals, and an openable / closable pressure plate is provided on the contact glass. After placing the original on the contact glass, the pressure plate is closed and the original is scanned. Yes. However, the manuscript is not limited to a sheet, and book manuscripts (books, booklets, etc.) may be handled as manuscripts. In such cases, the manuscript is placed on the contact glass and the manuscript is scanned. Will do.

  However, when a book manuscript is used as the manuscript, the page binding portion 101 of the book manuscript 100 is lifted from the contact glass 102 as shown in FIG. As described above, when the page binding portion 101 of the book document 100 is lifted from the contact glass 102, the page binding portion 101 is separated from the focal plane. , Image degradation such as blurred characters occurs. The page binding portion 101 of the deteriorated image is difficult to read, and the recognition rate when performing character recognition processing by OCR is remarkably reduced. In particular, the ratio is high in thick bookbinding, and when the page binding portion 101 of the book document 100 is pressed so as not to leave the focal plane, the book document 100 itself may be damaged.

  In order to solve such a problem, a method of correcting image distortion using a method of estimating the three-dimensional shape of an object from image density information has been proposed. As a method for estimating the three-dimensional shape of an object from such image density information, a method called Shape from Shading described in Non-Patent Document 1 is a typical example.

  Patent Document 1 proposes a method of correcting the distortion by measuring the shape of a book by a triangulation method.

  Further, Patent Document 2 proposes a method for estimating the three-dimensional shape of the book surface using the shape of the page outline of the scanned read image.

  However, according to the method called “Shape from Shading” described above, the amount of calculation is large, and the calculation time of the distortion correction processing is long.

  Further, according to the method described in Patent Document 1, a special shape measuring device for measuring the shape of a book by the triangulation method is required, and therefore, it is not appropriate.

  Furthermore, according to the method described in Patent Document 2, distortion correction can be performed with a small amount of calculation, but effective correction when the page outline is not completely contained in the image and is cut off in the middle. I can't.

  In view of this, the present applicant has disclosed an image distortion correction apparatus capable of effectively correcting the distortion with a small amount of calculation even in a scanned scan image in which the page outline is cut off in the middle of Japanese Patent Application No. 2002-247643. I have proposed. According to Japanese Patent Application No. 2002-247643, not only the page outline but also the character line information and ruled line information are used to correct image distortion, and the inherent parameters of the scanner (image reading means) (lens focal plane distance, scanning light) Since the axis position (address) is not used, the output image of any scanner can be corrected.

T. Wada, H. Uchida and T. Matsuyama, “Shape from Shading with Interreflections under a Proximal Light Source: Distortion-Free Copying of an Unfolded Book”, International Journal Computer Vision 24 (2), 125-135 (1997) Japanese Patent Laid-Open No. 5161002 Japanese Patent Laid-Open No. 11-41455

  However, according to the image distortion correction apparatus that can effectively correct the distortion with a small amount of calculation even in a scanned scan image in which the page outline as described above is cut off in the middle, the page outline is not Although image distortion is corrected using character line information and ruled line information, there is still a problem that is not complete and must be solved.

  For example, in a conventional method using a character line as a clue for correcting distortion, rectangles in character units are extracted, and an approximate curve is obtained from the rectangles in correction in the main scanning direction. However, as shown in FIG. 63, when an English image is written in alphabetical characters unlike an image written in Japanese, the aspect ratio of characters is various, and the interval between characters is narrow. There is also. Therefore, when rectangle extraction is performed on an English sentence image, a plurality of characters adjacent in the horizontal direction may come into contact with each other and be extracted as one rectangle. If distortion correction is performed based on a rectangle extracted as a single rectangle by contacting a plurality of characters in this way, distortion correction in the main scanning direction may be adversely affected.

  More specifically, in distortion correction in the main scanning direction, after extracting rectangles, the rows are integrated, an approximate curve is generated from the rectangles in the rows, and correction is performed based on the approximate curves. Therefore, when trying to generate a line from a rectangle in which characters are in contact with each other, as shown in FIG. 64, a plurality of lines are likely to be erroneously recognized as one line, and an inappropriate line may be generated. Then, the number of rectangles is reduced from the number of characters that are actually present in one line. Therefore, there is a problem that the approximate curve generated based on the center coordinates of the rectangle in one line becomes unstable.

  An object of the present invention is to provide an image distortion correction apparatus, a program, and a storage medium that can improve accuracy in distortion correction in the main scanning direction of an English image.

The image distortion correction apparatus according to the first aspect of the present invention is the image reading unit that reads a book document image that is in contact with the upper or lower side of the scanning surface so that the page binding portion is substantially parallel to the main scanning direction of the image reading unit. In the image distortion correction apparatus for correcting the distortion of the scanned image read by the method based on the shape of the character line in the scanned image, a predetermined second threshold value that is equal to or larger than a first threshold value in the main scanning direction from the scanned image. Means for extracting only a black pixel run in which the following number of black pixels are connected, and means for extracting, as a vertical component rectangle, only a vertical component rectangle that is a vertical component of a character, based on the extracted black pixel run. Means for extracting a character line based on the distance between the extracted vertical component rectangles in the main scanning direction and the sub-scanning direction, and a sentence used for correcting distortion of the scanned image from the extracted character line And means for determining a line, and means for calculating an approximate curve based on the center coordinates of each longitudinal component rectangle present in the determined character line, the.

  Therefore, by extracting a rectangle of only the vertical component of the character, contact between adjacent characters is prevented. By obtaining an approximate curve from such a vertical component rectangle, it is possible to improve the accuracy of distortion correction in the main scanning direction of an English image.

  According to a second aspect of the present invention, in the image distortion correction apparatus according to the first aspect, when the length of the extracted vertical component rectangle in the main scanning direction is equal to or greater than a predetermined threshold value, the length of the vertical component rectangle is set to the threshold value. Shrink to the length of

  Therefore, by reducing the length of the vertical component rectangle in the main scanning direction, it is possible to prevent the rectangle from coming into contact with the main scanning direction, so that a situation in which a plurality of lines are likely to be erroneously recognized as one line is avoided. And enables stable row integration.

  In the means for determining a character line to be used for distortion correction of the scanned image from the extracted character lines, a rectangle that bites most into the vicinity of the page binding portion as in the invention of claim 3. Or the ruled line included in the rectangle with the longest length in the sub-scanning direction as in the invention of claim 4. 6. A ruled line that is determined to be a ruled line to be used for distortion correction of the scanned image, or a ruled line included in a rectangle closest to a page outline of the scanned image, as in the invention of claim 5, is corrected for distortion of the scanned image. Determine the ruled line to use. According to a sixth aspect of the present invention, the ruled line used for distortion correction of the scanned image is a ruled line included in a rectangle that penetrates most to the vicinity of the page binding portion, and has the longest length in the sub-scanning direction. The ruled line included in the long rectangle and the ruled line included in the rectangle closest to the page outline of the scanned image may be determined in this order.

According to a seventh aspect of the invention, the image reading unit reads a book document image that is in contact with the upper or lower side of the scan surface so that the page binding portion is substantially parallel to the main scanning direction of the image reading unit. A program for causing a computer to execute a process of correcting distortion of a scanned image based on the shape of a character line in the scanned image, wherein the computer causes the computer to execute a process that is greater than or equal to a first threshold value that is predetermined in the main scanning direction from the scanned image A function that extracts only black pixel runs in which a number of black pixels equal to or less than a predetermined second threshold value are connected , and a vertical component that is a vertical component that is a vertical component of a character, based on the extracted black pixel runs. The function of extracting as a rectangle, the function of extracting a character line based on the distance between the extracted vertical component rectangles in the main scanning direction and the sub-scanning direction, A function of determining the character line to be used for distortion correction of the catcher down image, a function of calculating an approximate curve center coordinates based on each longitudinal component rectangle present in the determined character line, is executed.

  Therefore, by extracting a rectangle of only the vertical component of the character, contact between adjacent characters is prevented. By obtaining an approximate curve from such a vertical component rectangle, it is possible to improve the accuracy of distortion correction in the main scanning direction of an English image.

  According to an eighth aspect of the present invention, in the program according to the seventh aspect, when the length of the extracted vertical component rectangle in the main scanning direction is equal to or greater than a predetermined threshold, the length of the vertical component rectangle is set to the length of the threshold. Shrink to

  Therefore, by reducing the length of the vertical component rectangle in the main scanning direction, it is possible to prevent the rectangle from coming into contact with the main scanning direction, so that a situation in which a plurality of lines are likely to be erroneously recognized as one line is avoided. And enables stable row integration.

  In the function for determining a character line to be used for distortion correction of the scanned image from the extracted character lines, a rectangle that bites most into the vicinity of the page binding portion as in the invention according to claim 9. Or the ruled line included in the rectangle having the longest length in the sub-scanning direction is determined as the ruled line for use in correcting the distortion of the scanned image. 12. A ruled line that is determined as a ruled line to be used for distortion correction of the scan image, or a ruled line included in a rectangle closest to a page outline of the scan image, as in the invention of claim 11, is corrected for distortion of the scan image. Determine the ruled line to use. According to a twelfth aspect of the present invention, the ruled line used for distortion correction of the scanned image is a ruled line included in a rectangle that penetrates most to the vicinity of the page binding portion, and has the longest length in the sub-scanning direction. The ruled line included in the long rectangle and the ruled line included in the rectangle closest to the page outline of the scanned image may be determined in this order.

  A storage medium according to a thirteenth aspect stores the program according to any one of the seventh to twelfth aspects.

  Therefore, the program according to any one of claims 7 to 12 can be handled in the form of a tangible object.

  According to the present invention, by extracting a rectangle having only a vertical component of a character, contact between adjacent characters is prevented, and an approximate curve is obtained from such a vertical component rectangle, thereby correcting distortion in the main scanning direction of an English image. It is possible to improve the accuracy of the.

  An embodiment of the present invention will be described with reference to FIGS. The image distortion correction apparatus according to the present embodiment is provided in a digital copying machine as an image forming apparatus, and a scanner unit of the digital copying machine is applied as the image reading apparatus.

  Here, FIG. 1 is a longitudinal front view showing the configuration of the scanner unit 1. As shown in FIG. 1, the scanner unit 1 includes a contact glass 2 on which a document is placed, a first traveling body 5 including an exposure lamp 3 for exposing a document and a first reflection mirror 4, and a second reflection mirror 6. And a second traveling body 8 composed of the third reflecting mirror 7, a CCD (Charge Coupled Device) 9 as an imaging device for reading an image of the document, a lens unit 10 for forming an image on the CCD 9, and a document placed thereon. A document scale 11 that prevents the contact glass 2 from shifting and coming off, a white reference plate 12 for shading correction installed under the document scale 11, and a frame 14 are provided. The CCD 9 is formed on the sensor board 13.

  During scanning of a document, the first traveling body 5 and the second traveling body 8 are moved in the sub-scanning direction by a stepping motor 24 (see FIG. 3). That is, the first traveling body 5 and the second traveling body 8 travel under the contact glass 2, and the original is exposed and scanned by the exposure lamp 3. The light is reflected by the third reflecting mirror 7 and imaged on the CCD 9 through the lens unit 10. Here, an image reading means is realized.

  Such a scanner unit 1 is a printer unit (not shown) that is an image printing apparatus that forms an image on a sheet by, for example, electrophotography, in accordance with image data based on an image of a document read by the scanner unit 1. ). FIG. 2 is a perspective view showing an upper portion of the digital copying machine 16 on which the scanner unit 1 is mounted. As shown in FIG. 2, the scanner unit 1 is provided with a pressure plate 17 that can be opened and closed with respect to the contact glass 2, and an open / close sensor 18 that detects opening and closing of the pressure plate 17. As the printer provided in the digital copying machine 16, various printing methods such as an ink jet method, a sublimation type thermal transfer method, a silver salt photography method, a direct thermal recording method, and a melt type thermal transfer method are applied in addition to the electrophotographic method. be able to. Since the specific configuration is well known, detailed description is omitted.

  FIG. 3 is a block diagram showing the electrical connection of the control system of the scanner unit 1. As shown in FIG. 3, the control system includes an image processing unit 20 that is a circuit that performs various image processing on image data read by the CCD 9, and a first control unit 19 that controls the entire scanner unit 1. An image read by the CCD 9 and an operation panel 22 for accepting various operations to the digital copying machine 16 and receiving various operations on the digital copying machine 16, which is a circuit for controlling the traveling body 5 and the second traveling body 8. A memory 23 for storing data, predetermined data, and the like is connected. The operation panel 22 is provided with a copy start key for declaring the start of copying. Further, in the traveling body control unit 21, the exposure lamp 3, the stepping motor 24 that drives the first traveling body 5 and the second traveling body 8, and the first traveling body 5 and the second traveling body 8 are in the home position. A scanner home position sensor (HP sensor) 25 for detecting whether or not and an open / close sensor 18 are connected.

  Here, FIG. 4 is a block diagram showing a basic internal configuration of the image processing unit 20. As shown in FIG. 4, the image processing unit 20 includes an analog video processing unit 26 that performs analog image signal amplification processing and digital conversion processing when a document is read by the CCD 9, a shading correction processing unit 27 that performs shading correction processing, and shading. The digital image signal after the correction process includes an image data processing unit 28 that performs various image data processes such as MTF correction, scaling process, and γ correction to generate a scan image. The digital image signal after the image processing as described above is transmitted to the printer unit via the main control unit 19 and used for image formation.

  As shown in FIG. 5, the main control unit 19 includes a CPU (Central Processing Unit) 31 that centrally controls each unit. The CPU 31 includes a ROM (Read Only Memory) that stores BIOS and the like. Only memory (RAM) 32 and a RAM (Random Access Memory) 33 that stores various data in a rewritable manner and functions as a work area of the CPU 31 are connected by a bus 34 to constitute a microcomputer. Further, an HDD 35 in which a control program is stored, a CD-ROM drive 36 that reads a CD (Compact Disc) -ROM 37, and an interface (I / F) 38 that controls communication with a printer unit and the like are connected to the bus 34. ing.

  A CD-ROM 37 shown in FIG. 5 implements the storage medium of the present invention, and stores a predetermined control program. The CPU 31 reads the control program stored in the CD-ROM 37 with the CD-ROM drive 36 and installs it in the HDD 35. As a result, the main control unit 19 is in a state in which various processes as described later can be performed.

  As the storage medium, not only the CD-ROM 37 but also various types of media such as various optical disks such as DVD, various magnetic disks such as various magneto-optical disks and flexible disks, and semiconductor memory can be used. Alternatively, the program may be downloaded from a network such as the Internet and installed in the HDD 35. In this case, the storage device storing the program in the server on the transmission side is also a storage medium of the present invention. Note that the program may operate on a predetermined OS (Operating System), in which case the OS may execute a part of various processes described later, or a word processor. It may be included as part of a group of program files that constitute predetermined application software such as software or an OS.

  Next, the contents of various processes executed by the CPU 31 provided in the main control unit 19 based on the control program will be described. Here, only the scan image distortion correction processing that realizes the scan image distortion correction function, which is a characteristic function of the scanner unit 1 of the present embodiment, among the processes executed by the CPU 31 will be described. That is, the main control unit 19 functions as an image distortion correction device.

  FIG. 6 is a flowchart schematically showing the flow of a distortion correction process for a scanned image. Here, as shown in FIG. 7, the book document 40 is placed on the contact glass 2 so that the page binding portion 41 and the main scanning direction of image reading of the scanner portion 1 are parallel to each other. The case will be described.

  First, in step S1, a scanned image of the book document 40 placed on the contact glass 2 output from the image data processing unit 28 is input. Here, FIG. 8 shows an example of the input image. As shown in FIG. 9, the input scanned image of the book document 40 is distorted in the vicinity of the page binding portion 41.

  Next, optimal binarization processing of a scanned image (for example, a monochrome multi-valued image) of the book original 40 is executed (step S2), and a black pixel in the sub-scanning direction (a density value among pixels of the scanned image is determined in advance). A histogram with the number of pixels darker than the density value is obtained (step S3). FIG. 10 is a black pixel histogram on the left side of the binding portion boundary line of the image shown in FIG. The horizontal axis in FIG. 10 indicates the position of the black pixel in the main scanning direction (the pixel whose density value is darker than the predetermined density value among the pixels of the scanned image), and the vertical axis in FIG. This indicates the number of black pixels for each position. Note that the position in the sub-scanning direction where a pixel whose density value is originally dark among the pixels in the scan image is selected as the binding portion boundary line that is the scan image of the page binding portion 41.

  Note that the binarization process when the scanned image is a color multi-valued image focuses on, for example, any one of the RGB components (for example, the G component), and black pixels that are larger than the predetermined density threshold of the G component are black pixels. And a pixel smaller than a predetermined density threshold of the G component may be a white pixel. Alternatively, RGB may be color-converted to be divided into a luminance component and a color difference component, and threshold processing may be performed with the luminance component.

  In the subsequent step S4, page outline / ruled line / character line extraction processing is executed. FIG. 11 is a flowchart schematically showing the flow of page outline / ruled line / character line extraction processing.

[Extract page outline from scanned image]
First, the process of extracting the page outline from the scanned image in step S41 will be described. Here, FIG. 12 is an explanatory diagram showing an example of a scan image having a page outline at the upper end, and FIG. 13 is a black pixel histogram on the left side of the binding boundary line of the scan image shown in FIG. The x-axis of the histogram shown in FIG. 13 indicates the main scanning direction (vertical direction in FIG. 12) of the scan image, and the upper end of the scan image is associated with the left end of the histogram. Note that in the case of a scanned image having a page outline at the lower end, the lower end of the scanned image is associated with the right end of the histogram. Accordingly, when a page outline exists at the upper end of the scanned image as shown in FIG. 12, a black band appears at the upper portion of the scanned image, and thus a high vertical bar appears at the left end of the histogram shown in FIG. In this embodiment, it is determined whether or not a page outline exists in the scanned image using such characteristics.

算出された比率ｋが、予め定められた閾値よりも大きい場合に、スキャン画像にページ外形が存在すると判断する。 More specifically, as shown in FIG. 13, the distance AO from the binding boundary line to the left end of the scan image (left end in FIG. 12), the height BO of the histogram vertical bar, and the ratio is expressed by the following formula ( 1)

When the calculated ratio k is larger than a predetermined threshold, it is determined that a page outline exists in the scanned image.

  In addition, when page outlines exist above and below the scanned image, high vertical bars appear at the left and right ends of the histogram. In such a case, scanning is performed based on the high vertical bars at the left and right ends of the histogram. A determination is made as to whether or not a page outline exists in the image.

  If it is determined by the above processing that the page outline exists in the scanned image, the page outline is extracted together with information on which of the upper and lower sides of the left and right pages the page outline exists, and temporarily stored in the RAM 33. Remember.

  Note that the process of determining whether or not a page outline exists in the scanned image is executed for each of the left and right pages with the binding line boundary line of the scanned image as a boundary.

[Extract ruled lines from scanned images]
In the subsequent step S42, a ruled line extraction process from the scanned image is executed. The ruled line extraction process from the scanned image in step S42 will be described.

[Detection of ruled line candidates]
Here, FIG. 14 is an explanatory diagram showing an example of a scanned image having ruled lines. In the present embodiment, ruled line rectangle extraction is introduced, and ruled lines existing in a scanned image as shown in FIG. 14 are extracted as one rectangle. Although details will be described later, there may be cases where a ruled line cannot be extracted solely by simply performing rectangle extraction. Therefore, in this embodiment, rectangle extraction with restrictions on run registration is performed.

  FIG. 15 is an explanatory diagram illustrating a result of performing rectangular extraction on a binarized image. As shown in FIG. 15, a portion where black pixels are connected is extracted as one rectangle. As shown in FIG. 15, if a ruled line exists, it is extracted as a long and narrow rectangle in the sub-scanning direction. Therefore, based on the presence or absence of the long and narrow rectangle and the extracted rectangular shape (length / aspect ratio) and position. The presence / absence of ruled lines is determined.

  However, there are cases where a ruled line cannot be extracted by simply performing rectangle extraction. As shown in FIG. 16, when the ruled line is in contact with noise, a rectangle including noise is extracted. In the case of an image including a table as shown in FIG. 17, the ruled line in the sub-scanning direction intersects the ruled line in the main scanning direction, so that the entire table is extracted as one rectangle and the ruled line cannot be extracted alone.

[Rectangle extraction with restrictions on run registration]
Therefore, in order to extract the ruled lines alone, rectangle extraction with restrictions on run registration is performed. For an image in which noise is in contact with the ruled line as shown in FIG. 16, if only a run having a predetermined value in the main scanning direction (vertical direction) is registered and a rectangle is extracted, the ruled line is formed as shown in FIG. 18. The black pixel to be registered is a run to be registered, but the black pixel constituting the noise is not registered as a run. Since the rectangle extraction is performed on black pixels constituting the ruled line, the ruled line can be extracted independently.

  In addition, when extracting a ruled line in a rectangle, there is a method of extracting a rectangle only for a run that is long in the sub-scanning direction (horizontal direction), but in this method, a distorted portion near the binding portion is not included in the rectangle. However, by using the method of the present embodiment, a distorted portion near the ruled line binding portion can be included in the rectangle, and a more accurate ruled line position and length can be detected.

[Rectangle integration]
If rectangle extraction is performed on an image such as a table where a ruled line in the sub-scanning direction (horizontal direction) and a ruled line in the main scanning direction (vertical direction) intersect, the ruled line in the main scanning direction is not registered as a run. Rectangles are extracted in the direction. Then, as shown in FIG. 19, although there is a long ruled line in the sub-scanning direction, the ruled line is not extracted as one rectangle, but is extracted as a plurality of finely divided rectangles.

  Therefore, rectangle integration is performed. The rectangles whose distances in the sub-scanning direction are equal to or less than a certain value are integrated. FIG. 20 is an example in which rectangular integration is performed. In the rectangle integration, as shown in FIG. 20, the rectangles that have been cut into pieces are integrated into one rectangle, and a ruled line rectangle is extracted. This integration of rectangles is an effective method because the entire ruled line is extracted as one rectangle even if it is applied to a blurred ruled line or a dotted ruled line.

[Select optimal ruled line]
Next, rectangle extraction in which only runs less than a certain value are registered is performed, and the presence or absence of ruled lines is determined based on the presence or absence of a long and narrow rectangle in the sub-scanning direction. Such determination of the presence or absence of ruled lines is performed at each of the four locations of the upper left, lower left, upper right, and lower right of the image. For example, in the case of the image shown in FIG. 21, no ruled line exists only in the upper left. If there are a plurality of ruled lines at a certain location, the ruled lines used for correction are determined in the following priority order.
1. For example, in the case of the lower right side of the image shown in FIG. 21, a ruled line that penetrates to the vicinity of the binding portion is used for correction.
2. For example, in the case of the upper right side of the image shown in FIG. 21, the ruled line with the longer length is used for correction because both the ruled lines have penetrated to the vicinity of the binding portion.
3. For example, in the case of the lower left side of the image shown in FIG. 21, both ruled lines cut into the vicinity of the binding portion and the lengths are almost the same, so that the ruled line located outside the image is used for correction. The

[Coordinate value detection of optimal ruled line]
After selecting the optimum ruled line as described above, the coordinate value of each ruled line is detected. The position coordinates of the ruled line can be obtained from the extracted rectangular coordinates. As a special example, when the position of a rectangle elongated in the sub-scanning direction is in contact with the upper end or the lower end of the image, the rectangle is not regarded as a ruled line in consideration of the possibility that the rectangle is noise. In addition, when elongated rectangles are extracted from the left and right pages (for example, upper left and upper right, lower left and lower right), depending on the image, the rectangles of the left and right pages may be integrated across the binding portion. Then, since a long and narrow rectangle is extracted from the entire horizontal image, when such a feature is found in the extracted rectangle, the rectangle is divided at the binding position.

  If it is determined by the above processing that a ruled line exists in the scanned image, the ruled line is extracted together with information on which position of the ruled line exists on each of the left and right pages, and temporarily stored in the RAM 33.

[Extract text lines from scanned images]
In a succeeding step S43, a character line extraction process from the scanned image is executed. The character line extraction process from the scanned image in step S43 will be described. In the present embodiment, first, it is determined whether the character line in the scanned image is a vertically written character line or a horizontally written character line.

[Determination of character line]
A method for determining whether a character line in a scanned image is a vertically written character line or a horizontally written character line will be described. Here, FIG. 22 is a black-and-white inversion number histogram in the sub-scanning direction of the image shown in FIG. The horizontal axis in FIG. 22 is in the main scanning direction of black pixels in the sub-scanning direction (left-right direction) (pixels whose density value is darker than a predetermined density value among pixels obtained by inverting the scan image in black and white). The vertical axis in FIG. 22 indicates the number of black pixels for each position. FIG. 23 is a black-and-white inversion number histogram of the image shown in FIG. 8 in the main scanning direction. The horizontal axis in FIG. 23 represents the black pixel in the main scanning direction (vertical direction) (the pixel whose density value is darker than the predetermined density value among the pixels obtained by inverting the scan image in black and white) in the sub-scanning direction. The vertical axis in FIG. 23 indicates the number of black pixels for each position. When the characters in the image are horizontal scans as shown in FIG. 8, the histogram in the sub-scan direction as shown in FIG. 22 changes drastically, but the change in the histogram in the main scan direction as shown in FIG. Few. Although not particularly illustrated, when the character line in the scanned image is a vertically written character line, the histogram in the main scanning direction changes drastically, but the change in the histogram in the sub-scanning direction is small.

主走査方向ｙの位置でのヒストグラム値Pnt（ｙ）の平均値meanＨが算出される。ここで、heightは画像の高さである。
そして、下記に示す式（３）により、

副走査方向のヒストグラムの主走査方向に関する分散σＨが得られる。
同様に、下記に示す式（４）により、

副走査方向ｘの位置でのヒストグラム値Pnt（ｘ）の平均値meanＶが算出される。ここで、widthは画像の幅である。
そして、下記に示す式（５）により、

主走査方向のヒストグラムの副走査方向に関する分散σｖが得られる。 Specifically, the discrimination method as described above is realized by the following equations. First, according to the following equation (2):

An average value meanH of the histogram values Pnt (y) at the position in the main scanning direction y is calculated. Here, height is the height of the image.
And according to the equation (3) shown below,

A variance σH in the main scanning direction of the histogram in the sub-scanning direction is obtained.
Similarly, according to equation (4) below:

An average value meanV of the histogram values Pnt (x) at the position in the sub-scanning direction x is calculated. Here, width is the width of the image.
And according to the equation (5) shown below,

A variance σv in the sub-scanning direction of the histogram in the main scanning direction is obtained.

  As described above, when the character line in the scanned image is a horizontally written character line, the variance σH in the main scanning direction of the histogram in the sub-scanning direction is larger than the variance σv in the sub-scanning direction of the histogram in the main scanning direction. Conversely, when the character line in the scanned image is a vertically written character line, the variance σv in the sub-scanning direction of the histogram in the main scanning direction is larger than the variance σH in the main scanning direction of the histogram in the sub-scanning direction. That is, by comparing the variance σH and the variance σv, it is possible to determine whether the character line in the scanned image is a vertically written character line or a horizontally written character line.

  The reason why the black and white inversion number histogram is used to determine whether the character line in the scanned image is a vertically written character line or a horizontally written character line is to avoid confusion between the character line and the photograph portion. This is because, in general, when the values of the black pixel histogram are approximately the same, the value of the black / white inversion number histogram is larger in the character region than in the photo region.

[Detection of horizontal text line candidates]
If it is determined as a horizontally written character line as described above, a horizontally written character line candidate is detected. In the conventional method, circumscribed rectangles in character units are extracted to detect horizontal writing character line candidates, but when extracting rectangles, adjacent characters come into contact with each other and multiple characters become one rectangle. There was a point. Therefore, in the present embodiment, by limiting the length of the run and extracting a rectangle having only the vertical component of the character, an inappropriate run is eliminated and adjacent characters are not touched. Here, the vertical component is a vertical component of a character. The reason for extracting the rectangle of only the vertical component of the character in this way is that the aspect ratio of the alphabet (az) character is not constant, but the pitch of the vertical component included in the character is close to constant.

  Specifically, a run that is a set portion of black pixels that are considered to be characters is searched, and when the run is found, the length of the start coordinate of the run is greater than or equal to a predetermined first threshold in the main scanning direction. If it is below the second threshold, it is registered as a run. Rectangle extraction of only the vertical component is performed based on the run registered in this way. Here, FIG. 24 shows an example of the result of detection of horizontal writing character line candidates.

  In addition, in the present embodiment, after extracting the vertical component rectangle, the length of the vertical component rectangle is reduced to a certain length in the main scanning direction. The reason why the length of the vertical component rectangle is reduced in the main scanning direction is that the conventional method has a problem that a plurality of lines are easily recognized as one line. That is, by reducing the length of the vertical component rectangle in the main scanning direction, it is possible to prevent the rectangle from coming into contact with the main scanning direction and to perform stable row integration. Here, FIG. 25 is an explanatory diagram showing a method of reducing the vertical component rectangle. As shown in FIG. 25, assuming that the start coordinate in the main scanning direction of the vertical component rectangle is Xs, the end coordinate is Xe, and the center coordinate is Z, the upper and lower portions are contracted based on Z, and fixed in the main scanning direction. The vertical component rectangle is shortened by making it equal to or less than the length. Here, FIG. 26 shows an example of the result of detection of a horizontal character line candidate in which the vertical component rectangle is shortened. FIG. 26A is before the vertical component rectangle is contracted, and FIG. 26B is after the vertical component rectangle is contracted.

  In addition, when lines are merged from vertical component rectangles, noise lines may be merged. Approximate curves generated from vertical component rectangles in the noise row are not suitable for correction. Therefore, a line having a certain length or less is regarded as noise and is not used for correction. When these methods are performed, the possibility of obtaining a stable approximate curve increases.

  Next, an approximate curve is generated based on the center coordinates of the vertical component rectangle in one line. Here, FIG. 27A is an explanatory diagram showing generation of an approximate curve by a vertical component rectangle, and FIG. 27B is an explanatory diagram showing an enlarged row.

[Select optimum horizontal text line]
Next, a horizontal character line optimal for distortion correction is selected from the extracted horizontal character lines. When a plurality of horizontally written character lines are detected, it is necessary to select which horizontally written character line is used for distortion correction. An example of the optimum horizontal writing character line selection criterion is basically the same as the optimal ruled line selection criterion described above, and the horizontal writing character row used for correction is determined in the following priority order.
1. For example, in the case of the lower right side of the image shown in FIG. 28, a horizontally written character line that digs into the vicinity of the binding portion is used for correction.
2. For example, in the case of the upper right side of the image shown in FIG. 28, since both horizontal writing lines are biting into the vicinity of the binding portion, the horizontal writing line having the longer length is used for correction. The
3. For example, in the case of the lower left side of the image shown in FIG. 28, both the horizontal character lines are pierced to the vicinity of the binding portion, and the lengths are almost the same, so the horizontal characters that are located outside the image. Rows are used for correction.

[Determine the coordinate value of the optimal horizontal text line]
When the optimum horizontal writing character line is selected, the coordinate value (in the main scanning direction) of the horizontal writing character line is determined. The coordinate value (in the main scanning direction) of the horizontal writing line is obtained by connecting the center points of the vertical component rectangular main scanning directions in the horizontal writing line and approximating the straight line portion and the curved portion to extract the horizontal writing character line. The coordinate value (in the main scanning direction) is determined. More specifically, D shown in FIG. 28 is a binding boundary line, a coordinate value (in the main scanning direction) is estimated with a polynomial approximation curve between BDs, and an approximation between A and B at the left end is approximated. A coordinate value (in the main scanning direction) is estimated with a straight line value.

  If it is determined by the above processing that there is a horizontally written character line in the scanned image, the horizontally written character line is extracted together with information on which position of the horizontally written character line exists on each of the left and right pages and stored in the RAM 33. Memorize temporarily.

[Extract horizontal text lines based on vertical text lines]
Next, a horizontal writing character line is extracted from each vertical writing character line.

  FIG. 29 is a flowchart schematically showing the flow of processing for extracting horizontally written character lines from each vertically written character line. As shown in FIG. 29, first, a line cut-out rectangle of a vertically written character line is extracted (step S201). In addition, since the extraction process of the line cut-out rectangle of a vertically written character line can use the well-known technique generally used by OCR etc., the description is abbreviate | omitted. FIG. 30 is an explanatory diagram exemplarily showing the extracted row cutout rectangle.

  Next, a vertical writing character line having the maximum (or minimum) y coordinate at the beginning (or the end) of the vertical writing character line is extracted, and further, the vertical (with the leading (or tail)) existing within a predetermined distance range therefrom. A written character line is extracted (step S202). More specifically, in the example shown in FIG. 30, as shown in FIG. 31, the vertical writing character line having the maximum y coordinate of the first character of the vertical writing character row is the vertical writing character line indicated by A. . A line head character existing within a predetermined distance range h from the head position is a character indicated by a black circle “●” in FIG. That is, only the vertically written character lines including the character indicated by the black circle “●” are extracted, and the other vertically written character lines B and C are excluded. Note that h is a constant determined by the resolution of the scanned image.

  Next, a histogram is constructed for the y-coordinate at the beginning (or the end) of the extracted vertical writing line (step S203). In FIG. 32, a vertically written character line D close to the left end of the page is set as a reference line, and the top y coordinate (yD) is set as a reference coordinate. Hereinafter, the number of vertically written character lines having a head within a range of a constant width d (for example, ½ of the average width of the extracted vertically written character lines) with respect to yD is set as a histogram value related to yD. In FIG. 32, a vertically written character line whose head exists within the range of a dotted line sandwiching a straight line indicating yD vertically is the target. Therefore, the vertical writing character line E on the right side of the vertical writing character line D near the left end of the page is out of the range. In this way, when a vertically written character line that does not include the beginning of the target range of the existing reference coordinates appears, the vertically written character line is set as a new reference line, and the beginning coordinate is set as a new reference coordinate (here, YE). Also, since the line start coordinate of the vertical writing character line F adjacent to the right of the vertical writing character line E is included in the target range of yD, the value of the histogram relating to yD is counted up by 1 without providing a new reference coordinate. .

  Thereafter, the same processing is continued toward the binding portion boundary line. As a result, in the example shown in FIG. 32, the vertical writing character lines included in the target range of yD are surrounded by a hatched rectangle, and the vertical writing character lines included in the target range of yE are shaded. (The reference line, the reference coordinates, and the target range are also determined for vertically written character lines surrounded by other rectangles, but are omitted in FIG. 32). In addition, although the vertical writing character line G which should be irrelevant is included in the target range of yD, this is excluded in the next step S204.

  Subsequently, among the vertical writing character lines included in the target range of the reference line corresponding to the maximum value in the histogram configured in step S203, the vertical writing character line at the left end (or right end) of the page most. Using (reference line) as a start line, a vertically written character line having a leading (or trailing) y coordinate close to the binding boundary line is extracted (step S204). In FIG. 32, the maximum number of character lines included in the target range of the reference coordinate yD is seven, so the leftmost vertical writing character line D is the starting line, and binding is started from the starting line (vertical writing character line D). A vertically written character line with the leading y coordinate close to the partial boundary line is extracted.

  By the way, when extracting a vertically written character line whose leading y coordinate is close to the bounding line from the start line (vertically written character line D), a distortion occurs in the portion where the image is not distorted. The processing content is switched with the part that is present.

First, processing in a portion where image distortion has not occurred will be described with reference to FIG. In a portion where the image is not distorted, a vertically written character line that satisfies the following two conditions is extracted with reference to the target line H.
1. With respect to the positive direction of the y-coordinate (the upward direction in FIG. 33), there is a head of a vertically written character line that is extracted within a certain range b1 (for example, 1/2 of the average character line width) from the head position of the target line H. 1. What to do With respect to the negative direction of the y coordinate (the downward direction in FIG. 33), a predetermined angle (here) with respect to the positive direction of the x coordinate (the direction toward the binding boundary) viewed from the head position of the target row H Then, the head of the vertical writing character line to be extracted exists within the range of the angle of the straight line (b2 / a1). That is, the head of the vertical writing character line I next to the target row H is the above Since it is out of the range, it will be excluded, but the beginning of the next vertically written character line J will be extracted because it exists within the range. Hereinafter, the same processing is continued with the vertically written character line J as a new target line.

Next, processing in a portion where image distortion has occurred will be described with reference to FIG. In a portion where the image is distorted, a vertically written character line that satisfies the following two conditions is extracted with the target line L as a reference.
1. With respect to the negative direction of the y-coordinate (the downward direction in FIG. 34), a predetermined angle (here) with respect to the positive direction of the x-coordinate (the direction toward the binding boundary) viewed from the head position of the target row L The angle is represented by the slope of the straight line (b3 / a2). However, in consideration of the fact that the beginning of the character line intrudes into the inside of the page at the part where the distortion occurs, b2 / a1 < 1. The head of the vertically written character line to be extracted exists within the range of (b3 / a2). The slope (b4 / a2) of the straight line connecting the head position of the target line L and the head position of the vertical writing character line to be extracted is the slope of the straight line connecting the head position of the target line L and the head position of the previous extracted line K (b5). / A3) larger than a value obtained by subtracting a constant value α. That is, “b4 / a2> b5 / a3-α” is satisfied (basically “b4 / a2> b5 / a3” may be satisfied, but a constant value α is introduced in consideration of an error. α is a predetermined value)
That is, the beginning of the next vertically written character line M of the line of interest L is excluded because it is outside this condition, but the beginning of the next vertically written character line N is extracted because it satisfies the condition. Hereinafter, the same processing is continued with the vertically written character line N as a new target line.

  The problem here is how to identify the non-distorted portion and the distorted portion as follows. That is, if yC at the beginning of the target line and the next extracted line are yC and yN, respectively, if “yN−yC” is a certain value (for example, ¼ of the average character line width) or more, It is assumed that the part is distorted.

  The vertically written character lines extracted from FIG. 32 by the above method are shown surrounded by a hatched rectangle in FIG.

  Finally, an approximate curve polynomial relating to the position coordinates of the beginning (or the end) of the extracted vertically written character line is calculated (step S205). In the case of forming the outer shape by concatenating the heads of the row cutout rectangles of the extracted vertical writing character lines, as shown in FIG. An approximate curve polynomial is calculated regarding the position coordinates of the beginning of the extracted vertical character line. Further, in the case of forming the outline by connecting the end of the line cut rectangle of each extracted vertical character line, as shown in FIG. 36, the lower side center point of the line cut rectangle of each vertical character line to be connected is formed. Based on this, an approximate curve polynomial is calculated regarding the position coordinate at the end of the extracted vertical writing line.

  Finally, an inappropriate vertical character line outline is eliminated. This is because, when the coordinate value is estimated by polynomial approximation as described above, if the shape of the estimated curve by polynomial approximation is inappropriate, the distortion may increase at the time of correction. This eliminates the outline of vertical character lines. Examples of inappropriate approximate curve shapes are the same as in the case of the ruled lines and horizontal text lines described above, and although not particularly shown, when the curve goes to the outside of the book or greatly beyond the center line It is a case where it digs into.

  If the outline of the vertically written character line is excluded because the shape of the estimated curve is inappropriate, it means that there is no outline of the vertically written character line for distortion correction.

  If it is determined by the above processing that the outline of the vertical text line exists in the scanned image, the vertical text will be displayed along with information on where the vertical text line outline exists on the left and right pages. The outline of the row is extracted and temporarily stored in the RAM 33.

  In the following, it is assumed that the outer shape of a horizontal character line and a vertical character line is treated as a character line.

  As described above, the page outline / ruled line / character line extraction process (step S4) is completed by the processes of steps S41 to S43.

  In a subsequent step S5 (see FIG. 6), an image distortion correction process is executed. As shown in FIG. 37, the image distortion correction processing is roughly performed by using a page outline / position positioned near the upper side (or lower side) of the scanned image as a reference line (reference line) for distortion correction (expansion). Processing for selecting either a ruled line / character line (step S51: reference line selection process), which corresponds to the reference line and serves as a reference line for calculating a correction rate (expansion rate) ) (Page S52: reference line selection process) for selecting one of the page outline / ruled line / character line located near the image line, and if the reference line is a ruled line or character line, image information below the reference line is missing. Processing for calculating a virtual page outline for minimizing the image (step S53: virtual page outline calculation process), and performing a decompression process on the scanned image based on the virtual page outline to distort in the main scanning direction Processing for correcting (step S54: distortion correction processing in the main scanning direction), processing for extending the scanning image based on the character circumscribed rectangle of the corrected image and correcting distortion in the sub scanning direction (step S55: distortion correction in the sub scanning direction) Processing).

Here, in the reference line selection process (step S51) and the reference line selection process (step S52), the page outline / ruled line / character line positioned near the upper side (or lower side) of the scanned image as the reference line or reference line. However, the priority of selection of the page outline, ruled line, and character line in the present embodiment is as follows:
Page outline> ruled line> character line. This selection priority is used because character lines have lower extraction accuracy than page outlines and ruled lines, and more accurate distortion correction rates can be obtained by using page outlines outside the image. This is because it can. However, in this embodiment, the page outline generally spans the left and right pages, but ruled lines and character lines are not necessarily on both pages, so the priority order of the reference line selection from the upper or lower side of the scanned image is As shown below.

1. "Page outline" (Page outline always exists on both left and right pages)
2. "Rules" on both left and right pages
3. One page is "ruled line", the other page is "text line"
4). `` Character line '' on both left and right pages
5. "Rules" on one page only, "No clues" on the other page
6). "Character line" on one page only, "No clue" on the other page
Here, “no clue” means a case where none of the page outline, ruled line, and character line could be extracted.

  Next, examples of the virtual page outer shape calculation process (step S53) and the main scanning direction distortion correction process (step S54) will be described in order based on the priority order.

1. Case where “Page Outline” Exists on Both Upper and Lower Sides of Scanned Image Here, a case where “page outline” exists on both the upper and lower sides of the scanned image will be described. As shown in FIG. 38, when “page outline” exists on both the upper side and the lower side of the scanned image, any of them may be used as a reference line or a reference line. In the present embodiment, it is assumed that “page outline”, “ruled line”, and “character line” selected as the reference line are positioned on the lower side. In this case, since the reference line and the virtual page outline coincide with each other, the virtual page outline calculation process (step S53) does not have to be particularly performed.

Next, the main scanning direction distortion correction process (step S54) will be described. Here, x0 is a boundary point between the straight line portion and the curved portion of the “page outline”. In the main scanning direction distortion correction process, first, a distortion correction factor is calculated based on the distance h0 (see FIG. 39) between the reference line and the reference line at the boundary point x0. The distortion correction rate is for making the distance between the reference line and the reference line equal at all positions in the sub-scanning direction when the image is expanded and corrected in the main scanning direction. That is, when the distance between the reference line and the reference line at the position x is h (see FIG. 39), the distortion correction rate at the position x is
h0 / h
Can be expressed as In other words, the distortion correction rate refers to each x so that the distance between the reference line and the reference line becomes equal at all positions x in the sub-scanning direction when the image is expanded and corrected in the main scanning direction. It is a calculated value.

  Next, when the correction is actually performed, first, each pixel is shifted in the main scanning direction so that the virtual page outline (here, the reference line) becomes a straight line (the lowermost side) as shown in FIG. Thereafter, at all positions x in the sub-scanning direction, if the image is expanded at h0 / h toward the upper side in the main scanning direction, the image is corrected so that the “page outline” E on the upper side in FIG. 39 becomes RE. .

2. When the “page outline” exists on one of the upper and lower sides of the scanned image and “ruled lines” and “character lines” exist on the other side Here, either the upper or lower side of the scanned image A case where “page outline” exists and the page outline is cut off on the other side, but “ruled lines” and “character lines” exist will be described. As shown in FIG. 40, when the “page outline” exists on one of the upper side and the lower side of the scanned image and “ruled line” and “character line” exist on the other side, “page outline” Is set as a reference line and “character line” and “ruled line” are used as reference lines (in the example shown in FIG. 40, the left page is “character line” and the right page is “ruled line”). Even in this case, since the reference line and the virtual page outline coincide with each other, the virtual page outline calculation process (step S53) does not have to be performed.

Next, the main scanning direction distortion correction process (step S54) will be described. Here, x0 is a boundary point between the straight line portion and the curved portion of the “page outline”. In the main scanning direction distortion correction process, first, a distortion correction factor is calculated based on the distance h0 (see FIG. 41) between the reference line and the reference line at the boundary point x0. When the distance between the reference line and the reference line at the position x is h (see FIG. 41), the distortion correction rate at the position x is
h0 / h
Can be expressed as

  Next, when the correction is actually performed, first, each pixel is shifted in the main scanning direction so that the virtual page outline (here, the reference line) becomes a straight line (the lowermost side) as shown in FIG. Thereafter, if the image is expanded at h0 / h to the upper side in the main scanning direction at all the positions x in the sub-scanning direction, on the right page in FIG. 41, the image is displayed so that the “ruled line” LU on the upper side becomes the RLU. It is corrected. The same applies to the “character line” on the upper side of the left page.

3. When there is a “ruled line” on one of the upper and lower sides of both the left and right pages of the scanned image, and “ruled line” and “character line” are present on the other side A case will be described in which a “ruled line” exists on one of the upper side and the lower side, and a “ruled line” and a “character line” exist on the other side. When a “ruled line” exists on either the upper side or the lower side of both the left and right pages of the scanned image and a “ruled line” and a “character line” exist on the other side, as shown in FIG. The side where the “ruled line” exists on both pages is positioned on the lower side, the two “ruled lines” are used as reference lines, and the “character line” and “ruled line” positioned on the other side are used as reference lines (see FIG. In the example shown in 42, the left page is a “character line” and the right page is a “ruled line”). By the way, when the reference line is not “page outline” as described above, the reference line should not be regarded as a virtual page outline as it is. This is because, if they are matched, when the virtual page outline (reference line) is pixel-shifted to the lowermost side in the subsequent main scanning direction distortion correction processing, all image information below the reference line is lost. .

Therefore, when the reference line is not “page outline”, a virtual page outline calculation process (step S53) for obtaining a virtual page outline VE as indicated by a dotted line in FIG. 42 is executed. Here, x0 is a boundary point between the straight line portion and the curved portion of the “ruled line” LD on the lower side. The virtual page outline calculation processing is based on the distance a0 from the “ruled line” LD at the boundary point x0 to the lowermost side of the scanned image, and the distance b0 from the “ruled line” LD to the upper “ruled line” LU at the boundary point x0. The virtual page outline VE is calculated. That is, if the distance b from the position x on the “ruled line” LD to the “ruled line” LU on the upper side is solved,
a / b = a0 / b0
Therefore, the distance a from the position x on the “ruled line” LD to the virtual page outline VE can be calculated. Therefore, the virtual page outline VE can be obtained by calculating the distance from the position on the “ruled line” LD to the virtual page outline VE at all the positions x in the sub-scanning direction. Such processing is performed independently for the left and right pages.

Next, the main scanning direction distortion correction process (step S54) will be described. In the main scanning direction distortion correction processing, first, the distance a0 from the “ruled line” LD at the boundary point x0 to the lowermost side of the scanned image, and the “ruled line” LD at the boundary point x0 to the “ruled line” LU on the upper side. A distortion correction factor is calculated based on a distance h0 (h0 = a0 + b0) obtained by adding the distance b0. When the distance from the “ruled line” LU on the upper side at the position x to the lowest side of the scanned image is h (h = a + b), the distortion correction rate at the position x is
h0 / h
Can be expressed as

  Next, when actually correcting, first, each pixel is shifted in the main scanning direction so that the virtual page outline VE becomes a straight line (the lowermost side) as shown in FIG. After that, if the image is expanded at h0 / h to the upper side in the main scanning direction at all positions x in the sub-scanning direction, the “ruled line” LU on the upper side becomes the RLU (strictly, xLU on the right page in FIG. 37). Is not coincident with the boundary between the straight portion and the curved portion of the “ruled line” LU, but here, the image is corrected so that the boundary of the “ruled line” LD is used as it is. The same applies to the “character line” on the upper side of the left page.

  In this way, by using the virtual page outline VE, it is possible to minimize image loss due to shift.

4). When there are “ruled lines” and “text lines” on either the upper or lower side of both the left and right pages of the scanned image, and “ruled lines” only on one page of the other side. A case will be described in which “ruled lines” and “character lines” exist on either the upper side or the lower side of both the left and right pages, and “ruled lines” exist only on one page of the other side. When there are “ruled lines” and “character lines” on either the upper side or the lower side of both the left and right pages of the scanned image, and “ruled lines” exist only on one page on the other side (the other page 44), as shown in FIG. 44, the side where the “ruled line” and the “character line” are located is positioned on the lower side, the “ruled line” and the “character line” are used as the reference line, and the other A “ruled line” located on the side and a center line C penetrating the scanned image in the sub-scanning direction are used as reference lines (in the example shown in FIG. 44, the left page is “ruled line” and the right page is “no clue”). . The center line C is a line that crosses the center of the scanned image in the main scanning direction in the sub-scanning direction, and is not the center line of the book document 40.

When the reference line is not “page outline” as described above, the virtual page outline calculation process (step S53) for obtaining the virtual page outline VE as shown by the dotted line in FIG. 44 is executed as described above. Here, x0 is a boundary point between the straight line portion and the curved portion of the “ruled line” LD on the lower side. The virtual page outline calculation processing is based on the distance a0 from the “ruled line” LD at the boundary point x0 to the lowermost side of the scanned image and the distance b0 from the “ruled line” LD to the center line C at the boundary point x0. VE is calculated. That is, if the distance b from the position x on the “ruled line” LD to the center line C is solved,
a / b = a0 / b0
Therefore, the distance a from the position x on the “ruled line” LD to the virtual page outline VE can be calculated. Therefore, the virtual page outline VE can be obtained by calculating the distance from the position on the “ruled line” LD to the virtual page outline VE at all the positions x in the sub-scanning direction. Such processing is performed independently for the left and right pages. When a “ruled line” exists on the upper side as in the right page, instead of the center line C, the “ruled line” on the upper side is used as described above.

Next, the main scanning direction distortion correction process (step S54) will be described. In the main scanning direction distortion correction processing, first, the distance a0 from the “ruled line” LD at the boundary point x0 to the lowermost side of the scanned image, and the distance b0 from the “ruled line” LD to the center line C at the boundary point x0. The distortion correction rate is calculated based on the distance h0 (h0 = a0 + b0) obtained by adding. When the distance from the center line C at the position x to the lowest side of the scanned image is h (h = a + b), the distortion correction rate at the position x is
h0 / h
Can be expressed as

  Next, when the correction is actually performed, first, each pixel is shifted in the main scanning direction so that the virtual page outline VE becomes a straight line (lowermost side) as shown in FIG. Thereafter, at all the positions x in the sub-scanning direction, the image is corrected by extending it to the upper side in the main scanning direction at h0 / h. The same applies to the “ruled line” on the upper side of the left page. As for the right page, the center line C once distorted by the shift by the correction returns to the original straight line RC, but the correction of the entire image is incomplete except when C matches the optical axis of the scanner unit 1. .

5. When there is a “ruled line” on either the top or bottom side of the left or right page of the scanned image, and a “text line” exists only on the other page of the other side. A case where a “ruled line” exists on either the upper side or the lower side of the page and a “character line” exists only on the other page of the other side will be described. There is a “ruled line” on either the upper or lower side of the scanned image on either the left or right page (the other page is “no clue”), and there is a “character line” only on the other page on the other side. In the case (one page is “no clue”), as shown in FIG. 46, the side where the “ruled line” exists is positioned on the lower side, the “ruled line” is set as the reference line, and the “ruled line” does not exist For a page (the page on which “a character line” exists), a curve SL obtained by moving the character line L to a line-symmetrical position with the center line C in between is used as a reference line. As for the reference line, the center line C is used as the reference line for the page where the “ruled line” exists, and the “character line” is used as the reference line for the page where the “character line” exists.

  Note that the calculation of the distortion correction rate, pixel shift, and correction (expansion) processing are the same as those in the previous examples, and thus description thereof is omitted.

  Finally, sub-scanning direction distortion correction processing (step S55) will be described. First, the sub-scanning direction distortion correction processing when the intrinsic parameters (lens focal plane distance, scan optical axis position (address)) of the scanner unit 1 (image reading means) are unknown will be described.

  FIG. 47 is a flowchart schematically showing the flow of sub-scanning direction distortion correction processing when the intrinsic parameter is unknown. As shown in FIG. 47, in step S101, a circumscribed rectangle A (see FIG. 48) of the character is extracted based on the scanned image corrected for distortion in the main scanning direction. Here, since the character recognition process is a well-known technique, the description thereof is omitted. The reason why the circumscribed rectangle A of the character is extracted in this way is to correct the distortion in the sub-scanning direction based on the change in the shape of the circumscribed rectangle A of the character. Here, as shown in FIG. 48, the horizontal side length w, the vertical side length h, and the character center B of the character circumscribing rectangle A are defined. Here, the center B of the character is the intersection of the diagonal lines of the circumscribed rectangle A.

  Subsequently, as shown in FIG. 49, after the scanned image is divided into a plurality of strip-shaped areas C in a direction parallel to the page binding portion 41 of the book document 40 (step S102), A feature amount relating to the included character circumscribing rectangle A is obtained (step S103). Here, the circumscribed rectangle A included in a strip area C is a circumscribed rectangle A whose center is included in the strip area C. For example, the circumscribed rectangle A included in the strip area C1 of FIG. 49 is a shaded rectangle in the drawing.

Now, the feature amount related to the character circumscribed rectangle A is as follows.
(Length of horizontal side of character) / (Length of vertical side of character) = w / h
Required based on That is, for each strip region C, the average value of w / h values of all the character circumscribed rectangles A included therein is used as the feature amount of the strip region C.

  However, simply calculating the average value of w / h may be inappropriate. Some characters, such as punctuation marks and symbols in mathematical formulas, are originally small in size and have unstable w / h values. In addition, adjacent characters may be extracted when the rectangle is extracted, resulting in a character circumscribing rectangle A having an extremely large w. When obtaining the feature amount, it is necessary to exclude such special characters or extremely large w in advance. Therefore, in the subsequent step S104, a threshold value is determined in advance, and the character circumscribed rectangle A whose h value is smaller than the threshold value is excluded in advance, and a threshold value regarding the ratio of w / h is determined in advance. Character circumscribing rectangle A whose value is larger than the threshold is also excluded in advance. For example, the circumscribed rectangle A that is shaded in FIG. 50 is excluded in advance.

  In the subsequent step S105, after eliminating the extreme character circumscribing rectangle A as described above, the average value of w / h of the character circumscribing rectangle A in each strip region C is obtained. FIG. 51 shows an example of the average value of w / h of the circumscribed rectangle A in each strip area C. Note that the strip region C2 in FIG. 51 is a strip region including the page binding portion 41 of the book document 40.

  Subsequently, it is determined whether or not the character circumscribing rectangle A exists in the strip area C2 including the page binding portion 41 of the book document 40 (step S106). This is because, as shown in FIG. 50, generally, there are many cases where the character circumscribing rectangle A does not exist near the page binding portion 41 of the book document 40. If the character circumscribing rectangle A exists in the strip area C2 including the page binding portion 41 of the book document 40 (Y in step S106), the feature amount is calculated using the character circumscribing rectangle A, and thus the step S108 is performed as it is. Proceed to

  On the other hand, if the character circumscribing rectangle A does not exist in the strip area C2 including the page binding portion 41 of the book document 40 (N in step S106), the process proceeds to step S107, and the strip area C2 including the page binding portion 41 of the book document 40 is displayed. Find the feature quantity. The strip region C2 including the page binding portion 41 of the book document 40 is identified by, for example, obtaining a background density change of a scanned image (for example, a monochrome multi-value image) for each strip region C, and obtaining the most in the strip region C. This is realized by obtaining a low density value. FIG. 52 shows an example in which the background density change is obtained. The strip area with the highest background density is regarded as the strip area C2 including the page binding portion 41 of the book document 40.

  When the scanned image is a color multi-valued image, the identification of the strip region C2 including the page binding portion 41 of the book document 40 is focused on, for example, one of the RGB components (for example, the G component), and the G component The background density may be used for identification. Alternatively, RGB may be color-converted into a luminance component and a color difference component, and the strip region C2 including the page binding portion 41 of the book document 40 may be identified using the luminance component.

  The feature amount of the strip area C2 including the page binding portion 41 of the book document 40 is determined as follows. Here, there is a character circumscribing rectangle A that can be a statistical feature quantity calculation target, and a constant value determined in advance for the feature quantity of the strip area C that is the closest to the strip area C2 including the page binding portion 41. The value calculated by multiplying by is regarded as the feature amount in the strip area C2 including the page binding portion 41 of the book document 40. That is, in the example shown in FIG. 51, the character circumscribing rectangle A exists in any of the left and right strip regions C3 and C4 of the strip region C2 including the page binding portion 41 of the book document 40. The amount is selected (here, the circle on the right side), and is multiplied by a predetermined constant value (here, 0.5), and this is multiplied by the feature amount of the strip region C2 including the page binding portion 41 of the book document 40. It is said.

  In the subsequent step S108, an appropriate filtering process for the feature amount of each strip area C, for example, a process for obtaining a moving average in the direction of change of the position of the strip area C (ie, the sub-scanning direction) is performed. The change in the feature amount (in the sub-scanning direction) with respect to the change in the position is made gentle. However, also here, special processing is required near the page binding portion 41 of the book document 40. This is because if the filtering is performed using windows whose lengths are all equal in the sub-scanning direction, the sharpness of the feature amount change in the vicinity of the page binding portion 41 of the book document 40 is lost.

  Here, FIG. 53 shows the result of performing the filtering process on the feature amount of each strip region C shown in FIG. 51 using a window whose length is all five. As shown in FIG. 53, when the filtering process is performed using a window whose length is all 5, the change in the characteristic amount (w / h) in the vicinity of the page binding portion 41 of the book document 40 becomes gentle. It will pass. In such a case, appropriate image correction near the page binding portion 41 of the book document 40 becomes impossible.

  Therefore, in the present embodiment, during the filtering process, the page binding is performed so that the filter window does not extend over the strip regions C3 and C4 on both sides of the strip region C2 including the page binding portion 41 of the book document 40. The window length is adjusted near the portion 41. Here, FIG. 54 is a graph showing the result of performing the filtering process by adjusting the window length in the vicinity of the page binding portion 41. As shown in FIG. 54, when the window length is adjusted in the vicinity of the page binding portion 41, a change in the feature amount (w / h) in the vicinity of the page binding portion 41 can be appropriately expressed, so that a good image correction is realized. it can.

  In subsequent step S109, an estimated distortion amount of each strip region C is calculated. The calculation method of the estimated distortion amount of each strip region C is as follows.

  First, a strip area (reference strip area) is defined as a reference for calculating the distortion amount of the strip area. Here, the strip region C that is considered to have the smallest distortion, for example, the strip region C having the maximum feature (w / h) is set as the reference strip region. This process may be performed in common on the left and right pages, but the reference strip area may be determined independently on the left and right. FIG. 54 shows an example in which the reference strip area is determined independently on the left and right sides. The strip area C marked with a circle is the reference strip area, the reference feature value on the left is “Lw0 / Lh0”, and the reference reference on the right The feature amounts are indicated by “Rw0 / Rh0”, respectively.

Next, the feature amount w0 / h0 of the reference strip region is set as the reference feature amount of the entire scanned image,
(Feature amount of each strip area) / (reference feature amount) = (w / h) / (w0 / h0)
Is calculated as an estimated amount of distortion in each strip region.

  If the strip region C near the outside of the page outside the page binding portion 41 of the book document 40 is set as the reference strip region, the difference in font and type size is large from the vicinity of the page binding portion 41 of the book document 40. It is also conceivable that a large estimated distortion amount cannot be calculated. When such an image is the target, it is effective to limit the search range of the reference strip region to the vicinity of the page binding portion 41 of the book document 40 in advance. In order to realize this, the reference strip region may be obtained only from the strip region C whose background density is higher than a predetermined density.

Finally, enlargement processing in the short side direction (sub-scanning direction) of the strip region C is performed on the scanned image to correct distortion in the vicinity of the page binding portion 41 of the book document 40 (step S110). The enlargement ratio in that case is the reciprocal of the estimated distortion amount calculated in step S109, that is,
(Reference feature value) / (Feature value of each strip area) = (w0 / h0) / (w / h)
And Here, if the above-mentioned standard strip area is defined to be common to the left and right, this enlargement ratio is also calculated based on the standard feature quantity common to the left and right. To do. FIG. 55 shows the corrected enlargement ratio calculated based on the feature amount shown in FIG.

  In this case as well, the strip area C away from the vicinity of the page binding portion 41 of the book document 40 is likely to be an area without distortion of the image from the beginning, so it may be better not to make it an object of enlargement processing. This is because an unnatural distortion may occur due to the enlargement process. In order to prevent this, the estimated distortion amount is set to “1” for the strip region C whose background density is thinner than a predetermined density.

  Further, when a common correction magnification ratio is applied in the strip area C, the correction magnification ratio becomes discontinuous at the boundary between adjacent strip areas C, and the correction image becomes unnatural. Therefore, the correction magnification rate is corrected in advance so that the correction magnification rate at the boundary between adjacent strip regions C changes continuously. For example, the correction magnification rate of the central portion of the strip region C shown in FIG. 55 is plotted as a point indicating the reciprocal of the estimated distortion amount, and these points are connected by a line segment to complement a straight line, This can be realized by setting the correction magnification. Through the above processing, the correction magnification rate in the sub-scanning direction of the scanned image is determined.

  Note that the image enlargement process is executed using, for example, a cubic function convolution method often used as a copy scaling function.

  Next, the sub-scanning direction distortion correction processing in the case where the intrinsic parameters (lens focal plane distance, scan optical axis position (address)) of the scanner unit 1 (image reading means) are known will be described.

  When the main scanning direction of scanning and the boundary line of the page binding portion 41 of the book document 40 are parallel, the image forming system has the following characteristics when the book document 40 is read by a scanner lens such as the lens unit 10. The main scanning direction is center projection, and the sub-scanning direction is projection. Here, three-dimensional shape restoration is performed using these characteristics.

  In the case of central projection, as shown in FIGS. 56 and 57, the surface of the book document 40 floats from the surface of the contact glass 2 and the image formation distance becomes longer, and the magnification of the image becomes smaller. Curves inward. If the shrinkage amount AB in FIG. 57 is measured, the height h of the surface of the book floating in FIG. 56 can be calculated. Therefore, if the amount of distortion inward of the straight line is measured, the three-dimensional shape (the amount of lifting of the book original 40 from the contact glass 2) can be restored.

Here, it is assumed that the book original 40 is placed horizontally on the contact glass 2. Then, the three-dimensional shape becomes two-dimensional. The imaging relationship of the scanner lens is shown in FIG. OO ′ is the optical axis of the lens, and 0 is the center of the lens. F is the distance from the center 0 of the lens to the scanner surface (contact glass 2), which is called the focal length of the scanner. The point B on the scanner surface forms an image on D on the image plane. A point C on the surface of the floating bookbinding (book original 40) forms an image on E. The distance from the center 0 of each imaging plane is y ′ and y (the center 0 also moves as the first traveling body 5 and the second traveling body 8 move in the sub-scanning direction. This will be called the imaging center line). The following relational expressions (6) and (7) are obtained by the triangular similarity.
h / F = AB / AO ′ (6)
AB / AO ′ = (y′−y) / y (7)
Further, the amount h of lifting of the book document 40 is obtained by the following equation (8) from the equations (6) and (7).
h = F × ((y′−y) / y) (8)
It can be seen from this equation (8) that a three-dimensional shape is obtained from the two-dimensional strain amount. Here, the distortion amount (y′−y) and the distance y are obtained from the image. In the present embodiment, this is obtained from the distance between the straight line component of the character line or ruled line and the curved portion. The focal length F of the lens is a known amount determined by the scanner unit 1, and a setting value of the scanner unit 1 or a calibration value of the lens is used.

  Therefore, when the intrinsic parameters (lens focal plane distance, scan optical axis position (address)) of the scanner unit 1 (image reading means) are known, the surface of the book is determined from the degree of curvature of the page outline / ruled line / character line. As shown in FIG. 59, the height h of the floating from the scanning surface is obtained, and one pixel in the sub-scanning direction is expanded by m times and added, whereby the distortion in the sub-scanning direction can be corrected. .

  Thus, the sub-scanning direction distortion correction process (step S55) ends, and the scan image distortion correction process shown in FIG. 6 ends. Here, FIG. 60 is a plan view showing an image in which distortion is corrected. According to the above processing, the distortion of the scan image that has occurred in the vicinity of the page binding portion 41 of the book document 40 as shown in FIG. 8 is corrected as shown in FIG.

  Here, by extracting a rectangle of only the vertical component of the character, contact between adjacent characters is prevented. By obtaining an approximate curve from such a vertical component rectangle, it is possible to improve the accuracy of distortion correction in the main scanning direction of an English image. In addition, when the length of the extracted vertical component rectangle in the main scanning direction is equal to or larger than a predetermined threshold, the rectangle comes into contact with the main scanning direction by reducing the length of the vertical component rectangle to the threshold length. Therefore, it is possible to avoid a situation in which a plurality of lines are likely to be erroneously recognized as one line, and to enable stable line integration.

  In the present embodiment, the scanner unit 1 of the digital copying machine is applied as the image reading apparatus. However, the present invention is not limited to this. For example, the scanner unit 1 may be applied to a scanner equipped with an automatic page turning function. .

  In the present embodiment, the book original 40 is placed on the contact glass 2 so that the page binding portion 41 of the book original 40 and the main scanning direction of image reading of the scanner unit 1 are parallel to each other. However, the present invention is not limited to this. For example, as shown in FIG. 61, an upward book original 40 may be brought into contact with the contact glass 2 from below the contact glass 2.

  Furthermore, in the present embodiment, the image distortion correction apparatus is provided in the digital copying machine 16 that is an image forming apparatus, and image distortion correction processing is performed on the scanned image read by the scanner unit 1 of the digital copying machine 16. However, it is not limited to this. For example, an image scanner having an image reading unit for reading a document image is connected to a personal computer, and a program stored in a CD-ROM 37 which is a storage medium is installed in the HDD of the personal computer, whereby an image distortion correction apparatus is installed. Even if it comprises, the effect similar to the various effect mentioned above can be acquired. Further, an image distortion correction apparatus is configured by installing a program stored in a CD-ROM 37 as a storage medium in the HDD of a personal computer, and distortion correction processing is performed on a scanned image read in advance by an image reading means. You may do it.

It is a vertical front view which shows the structure of the scanner part of one Embodiment of this invention. It is a perspective view which shows the upper part of the digital copying machine carrying a scanner part. It is a block diagram which shows the electrical connection of the control system of a scanner part. It is a block diagram which shows the basic internal structure of an image process part. It is a block diagram which shows the electrical connection of a main control part. It is a flowchart which shows roughly the flow of the distortion correction process of a scanned image. It is a perspective view which shows the state which mounted the book original on the contact glass of a scanner part. It is a top view which shows an example of the input image. It is explanatory drawing which shows the distortion of the vicinity of the page binding part of a scan image. 9 is a black pixel histogram of the image shown in FIG. 8. It is a flowchart which shows roughly the flow of the extraction process of a page external shape / ruled line / character line. It is explanatory drawing which shows an example of the scanning image which has a page external shape in an upper end. 13 is a black pixel histogram on the left side of the binding portion boundary line of the scanned image shown in FIG. 12. It is explanatory drawing which shows an example of the scanning image in which a long ruled line exists. It is explanatory drawing which shows the result of having performed rectangular extraction to the binarized image. It is explanatory drawing which shows the case where a ruled line is contacting with noise. It is explanatory drawing which shows the image containing a table | surface. It is explanatory drawing which shows the result of having registered only the run below a fixed value and extracting the rectangle. It is explanatory drawing which shows the case where a rectangle will be extracted to a thin piece in the subscanning direction. It is explanatory drawing which shows the example which performed rectangle integration. It is explanatory drawing which shows the result of having performed rectangle extraction. 9 is a black / white inversion number histogram in the sub-scanning direction of the image shown in FIG. 8. 9 is a black / white inversion number histogram of the image shown in FIG. 8 in the main scanning direction. It is explanatory drawing which shows an example of the result of a detection of a horizontal writing character line candidate. It is explanatory drawing which shows the method of shrinking | reducing a vertical component rectangle. It is explanatory drawing which shows an example of the result of the detection of the horizontal writing character line candidate which shortened the vertical component rectangle. (A) is explanatory drawing which shows the production | generation of the approximated curve by a vertical component rectangle, (b) It is explanatory drawing which expands and shows a line. It is explanatory drawing which shows the result of having performed rectangle extraction. It is a flowchart which shows roughly the flow of the extraction process of the horizontal writing character line from each vertical writing character line. It is explanatory drawing which shows the extracted line cut-out rectangle as an example. It is explanatory drawing which shows illustratively the vertical writing character line in which the head exists within the predetermined distance range. It is explanatory drawing which shows the state which comprises a histogram regarding the y coordinate of the head of the extracted vertical writing character line. It is explanatory drawing which shows the process in the part which has not produced distortion of an image. It is explanatory drawing which shows the process in the part which has produced the distortion of an image. It is explanatory drawing which shows the extracted vertical writing character line. It is explanatory drawing which shows the line cut-out rectangle of a vertical writing character line. It is a flowchart which shows the flow of an image distortion correction process roughly. It is explanatory drawing which shows the case where a "page outer shape" exists in both the upper side and lower side of a scan image. It is explanatory drawing which shows the state which shifted the pixel of FIG. It is explanatory drawing which shows the case where "page outline" exists in any one of the upper side of a scan image, and a lower side, and a "ruled line" and a "character line" exist in the other side. It is explanatory drawing which shows the state which shifted the pixel of FIG. It is explanatory drawing which shows the case where a "ruled line" exists in either one of the upper side and the lower side of both the left and right pages of a scanned image, and a "ruled line" and a "character line" exist on the other side. It is explanatory drawing which shows the state which shifted the pixel of FIG. It is explanatory drawing which shows the case where a "ruled line" and a "character line" exist in any one of the upper side and the lower side of both the left and right pages of a scanned image, and a "ruled line" exists only in one page of the other side. . It is explanatory drawing which shows the state which shifted the pixel of FIG. It is explanatory drawing which shows the case where a "ruled line" exists in any one of the upper side and the lower side of a left or right page of a scanned image, and a "character line" exists only in the other page of the other side. It is a flowchart which shows roughly the flow of a subscanning direction distortion correction process. It is explanatory drawing which shows the extracted character circumscribed rectangle. It is explanatory drawing which shows the state which divided | segmented the binarized image into the several strip-shaped area | region of the direction parallel to the page binding part of a book original. It is explanatory drawing which shows the character circumscribing rectangle excluded beforehand. It is a graph which shows an example of the average value of the feature-value of the circumscribed rectangle in each strip area. It is a graph which shows an example of the background density change in each strip area. It is a graph which shows the result of having performed the filtering process using the window whose length is all 5 with respect to the feature-value of each strip area | region shown in FIG. FIG. 52 is a graph showing a result of performing filtering processing by adjusting the window length in the vicinity of the page binding portion with respect to the feature amount of each strip area shown in FIG. 51. FIG. 55 is a graph showing a corrected enlargement ratio calculated based on the feature amount shown in FIG. 54. FIG. It is explanatory drawing which shows the height which the surface of the book floated. It is explanatory drawing which shows the quantity which an image shrinks. It is explanatory drawing which shows the imaging relationship of a scanner lens. It is explanatory drawing which shows a subscanning direction distortion correction process in case an intrinsic | native parameter is known. It is a top view which shows the image which correct | amended distortion. It is a front view which shows the state which made the book original document contact the contact glass. It is a front view which shows the state which mounted the book original on the contact glass. The rectangle extraction of a character unit is shown, (a) is Japanese rectangle extraction, (b) is explanatory drawing which shows the rectangle extraction of an English sentence. It is explanatory drawing which shows the case where several lines are misrecognized as one line.

Explanation of symbols

19 Image distortion correction device 37 Storage medium 40 Book manuscript 41 Page binding part

Claims (13)

The distortion of the scanned image obtained by reading the book document image read by the image reading unit so that the page binding portion is substantially parallel to the main scanning direction of the image reading unit is scanned in the scanned image. In an image distortion correction apparatus that corrects based on the shape of a character line,
Means for extracting only black pixel runs in which a number of black pixels not less than a predetermined first threshold and not more than a predetermined second threshold are connected in the main scanning direction from the scanned image;
Based on the extracted black pixel run, means for extracting a vertical component rectangle that is a vertical component of a character as a vertical component rectangle;
Means for extracting a character line based on the distance between the extracted vertical component rectangles in the main scanning direction and the sub-scanning direction;
Means for determining a character line to be used for distortion correction of the scanned image from the extracted character line;
An image distortion correction apparatus comprising: means for calculating an approximate curve based on the center coordinates of each vertical component rectangle existing in the determined character line.   2. The image distortion correction according to claim 1, wherein when the length of the extracted vertical component rectangle in the main scanning direction is equal to or greater than a predetermined threshold, the length of the vertical component rectangle is reduced to the threshold length. apparatus.   The image distortion correction apparatus according to claim 1, wherein a character line that penetrates to the vicinity of a page binding portion is determined as a character line to be used for distortion correction of the scanned image.   3. The image distortion correction apparatus according to claim 1, wherein a character line having the longest length in the sub-scanning direction is determined as a character line to be used for distortion correction of the scanned image.   3. The image distortion correction apparatus according to claim 1, wherein a character line closest to a page outline of the scan image is determined as a character line to be used for distortion correction of the scan image.   The character line used for correcting the distortion of the scanned image is the character line that has penetrated most to the vicinity of the page binding part, the character line having the longest length in the sub-scanning direction, and the character line closest to the page outline of the scanned image The image distortion correction apparatus according to claim 1, wherein the image distortion correction apparatus is determined in the following order. The distortion of the scanned image obtained by reading the book document image read by the image reading unit so that the page binding portion is substantially parallel to the main scanning direction of the image reading unit is scanned in the scanned image. A program for causing a computer to execute a process for correcting based on the shape of a character line, wherein the computer
A function of extracting only black pixel runs in which a number of black pixels not less than a predetermined first threshold and not more than a predetermined second threshold are connected in the main scanning direction from the scanned image;
Based on the extracted black pixel run, a function that extracts a vertical component rectangle that is a vertical component of a character as a vertical component rectangle;
A function of extracting a character line based on the distance between the extracted vertical component rectangles in the main scanning direction and the sub-scanning direction;
A function for determining a character line to be used for distortion correction of the scanned image from the extracted character line;
And a function for calculating an approximate curve based on the center coordinates of each vertical component rectangle existing in the determined character line.   8. The program according to claim 7, wherein if the length of the extracted vertical component rectangle in the main scanning direction is equal to or greater than a predetermined threshold value, the length of the vertical component rectangle is reduced to the threshold value.   The program according to claim 7 or 8, wherein a character line that penetrates to the vicinity of a page binding portion is determined as a character line to be used for distortion correction of the scanned image.   9. The program according to claim 7, wherein a character line having the longest length in the sub-scanning direction is determined as a character line to be used for correcting distortion of the scanned image.   9. The program according to claim 7, wherein a character line closest to a page outline of the scanned image is determined as a character line to be used for correcting distortion of the scanned image.   The character line used for correcting the distortion of the scanned image is the character line that has penetrated most to the vicinity of the page binding part, the character line having the longest length in the sub-scanning direction, and the character line closest to the page outline of the scanned image 9. The program according to claim 7, wherein the program is determined in the following order. A storage medium storing the program according to any one of claims 7 to 12.

Images