JP4577844B2 - Image processing apparatus, image processing method, program, and storage medium storing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, a program, and a storage medium for processing a background or shadow of a scanned image obtained by reading a book document.

  Many originals read using a flatbed scanner are sheet-like originals, and an openable / closable pressure plate is provided on the contact glass. After placing the original on the contact glass, the pressure plate is closed and the original is scanned. Yes. However, the original is not limited to a sheet, and book originals (books, booklets, etc.) may be handled as originals. In such cases, the book original is placed on the contact glass and the original is scanned. Will do.

  However, when a book document is used as the document, the binding portion of the book document is lifted from the contact glass. FIG. 50 shows an example of a scanned image of a book document. As shown in FIG. 50, when the binding part is lifted from the contact glass, the binding part is separated from the focal plane, so that the scanned image of the lifted part has image degradation such as image distortion, shadow, and character blurring. Will occur. The binding portion of the deteriorated image is difficult to read, and the recognition rate when performing character recognition processing by OCR (Optical Character Reader) is significantly reduced. In particular, in the case of thick bookbinding, the binding part is severely deteriorated, and when the bookbinding part is pressed so as not to leave the focal plane, the book original itself may be damaged. In view of this, an image reading apparatus that corrects the distortion of the binding portion using the page outline, character line information, and ruled line information has been proposed.

  In addition, in order to correct the brightness of the binding portion, the scanned image is divided into a plurality of blocks, and the brightness value of the pixel having the highest brightness among the pixels included in each block is set as the background value of each block. A background correction apparatus that performs background correction processing based on the background value of each block has been proposed (see, for example, Patent Document 1).

In addition, a background value in the scanned image is specified according to a desired processing mode selected from a plurality of processing modes related to the background correction processing for the scanned image, and the background for the scanned image is specified based on the specified background value. Background correction apparatuses that perform correction processing have been proposed (see, for example, Patent Document 2). In the background correction apparatus described in Patent Document 2, since the position on the image from which the background value to be used as a reference for the background correction processing is extracted differs depending on the type of the image reading means, the user is most suitable for the image reading means to be used. The background correction process for the scanned image can be executed by selecting the processing mode.
JP 2003-69824 A JP 2003-198845 A

  However, the conventional background correction apparatus cannot sufficiently correct the shadows at both ends of the binding portion (the top and the vicinity of the binding portion), and shadows remain at both ends of the binding portion (the shadow is image data). Is the same as lightness, but it is called a shadow to distinguish it). This is presumably because the intensity of the illumination light becomes weaker as it is closer to both ends of the book document because the length of the light source (fluorescent lamp) of the image scanner is finite. Moreover, since the reflected light (mutual reflection light) from the opposite side is also included as illumination on the left and right book surfaces, it is considered that the influence of the binding also affects the shadow of the binding portion.

  In view of the above problems, an object of the present invention is to provide an image processing apparatus, an image processing method, a program, and a storage medium in which the program is stored, in which shading can be corrected near both ends of a binding portion of a book document.

In view of the above problems, according to the present invention, the image reading unit reads the book document placed on the contact glass, and the first unit determines whether the book document is flat from the pixel value of the scanned image of the book document. Identifying a portion, and normalizing the lightness of the scanned image with respect to a position y in a direction orthogonal to a one-dimensional image in a direction connecting both ends of the binding portion with reference to the pixel value of the flat portion to generate a normalized lightness profile A background correction unit correcting the background near the binding portion of the scanned image using the normalized brightness profile; and a second unit is a lightness distribution with respect to a position x in a direction parallel to the one-dimensional image. On the other hand, the position where the brightness is approximately the median of the brightness distribution of the one-dimensional image is x0, the difference between the approximately median and the asymptotically constant brightness is a, and the gradient of the brightness with respect to the position x passing through the approximately median. The b, and substantially median brightness distribution c as, the lightness
And defining a brightness distribution of the one-dimensional image by assigning a predetermined value to x0 and substituting the lightness at a certain x of the one-dimensional image into the formula, And a step of correcting shadows in the vicinity of both ends of the binding portion of the scanned image using the lightness distribution .

  According to the present invention, it is possible to correct the shadows generated at both ends of the binding portion of the book document by correcting the shadows near the both ends of the binding portion of the scanned image. The pixel value is RGB or the like. For example, the pixel value may be a value obtained by processing the pixel value like the brightness calculated from the RGB value.

  As one form of the present invention, the second means performs a filter operation on the pixel value of the one-dimensional image, estimates the coordinates xl and xr at both ends of the scan image, and is outside the coordinates xl or more than the coordinates xr. The outside position is set to the predetermined value of x0.

In one aspect of the present invention, the shadow correction means determines the brightness, saturation, and hue from the RGB values for each pixel, determines whether each pixel of the book document is a chromatic color or an achromatic color, In the case of chromatic color, the background correction by the normalized lightness profile is performed for both the saturation and the lightness, and in the case of achromatic color, the background correction by the normalized lightness profile is performed only for the lightness. Obtaining RGB values from saturation and hue .

In one aspect of the present invention, the shadow correction unit obtains the brightness for each pixel, and performs the background correction by the normalized brightness profile for each of the R value, the G value, and the B value of the pixel value. It is characterized by having.

In one form of this invention, the said shading correction | amendment means calculates | requires each said R value, G value, and B value of said normalized brightness profile of R value, G value, and B value, Performing background correction based on the normalized brightness profile of each of the value, G value, and B value .

In one form of this invention, the said shadow correction means produces | generates the brightness profile which shows the brightness distribution of the said scan image corresponding to the position y of the direction orthogonal to a one-dimensional image, and the binding part of the said scan image And a step of specifying a predetermined region centered on a position y where the lightness profile shows a downward peak, and a step of correcting shadows near both ends of the binding portion of only the region of the scanned image using the lightness distribution. It is characterized by having.

In one aspect of the present invention, the shading correction unit divides the scan image of the region into a plurality of regions parallel to a one-dimensional image, and the second unit includes a boundary between the divided regions. Obtaining the lightness distribution of the one-dimensional image only for the one-dimensional image, complementing the lightness distribution in a direction orthogonal to the one-dimensional image, and using the lightness distribution, shadows near both ends of the binding portion of the scanned image are obtained. And a step of correcting .

In one aspect of the present invention, the shadow correction unit divides the scan image into a plurality of regions in a direction orthogonal to the one-dimensional image, and uses the brightness distribution for each of the divided regions. Correcting shadows near both ends of the binding portion of the image .

In one embodiment of the present invention, a center portion of the scan image in a direction orthogonal to the one-dimensional image is specified based on a shape in which an outer edge of the scan image enters inside, and is divided into two regions from the center portion. A step of correcting the background for each of the divided areas; and a step of correcting the shadow near both ends of the binding portion of the scan image for each of the divided areas; It is characterized by having.

  It is possible to provide an image processing apparatus, an image processing method, a program, and a storage medium storing the program that can correct shadows near both ends of the binding portion of the book document.

  The best mode for carrying out the present invention will be described below with reference to the drawings. The image processing apparatus according to the present embodiment is applied to a digital copying machine that is an image forming apparatus, and includes a scanner unit of the digital copying machine for reading an image. In other words, the image processing apparatus can be applied to an image forming apparatus, a scanner apparatus, a facsimile apparatus, and a multifunction peripheral MFP (Multi Function Printer).

  Further, not only a scan image by a one-dimensional image sensor (sometimes referred to as image data in some cases) as in a scanner unit of a digital copying machine, but also image data by a two-dimensional image sensor as in a digital camera. This image processing apparatus can perform the same processing. In the case of image data photographed with a scanner device or a digital camera, a personal computer (hereinafter simply referred to as a PC) executes a program for performing image processing according to the present embodiment, and background correction, shadow correction, and distortion correction are performed.

  FIG. 1 is a longitudinal front view showing a configuration of a scanner unit 1 of a digital copying machine. As shown in FIG. 1, the scanner unit 1 includes a first traveling body 5 including a contact glass 2 on which an original is placed, an exposure lamp (hereinafter referred to as a linear light source) 3 for exposing the original, and a first reflecting mirror 4. A second traveling body 8 comprising a second reflecting mirror 6 and a third reflecting mirror 7, a CCD (Charge Coupled Device) 9 as an image pickup device for reading an image of a document, and a lens unit for forming an image on the CCD 9. 10, a document scale 11 that serves as a reference for placing the document and prevents the contact glass 2 from being displaced or detached, a white reference plate 12 for shading correction installed under the document scale 11, and a frame 14. And. The CCD 9 is formed on the sensor board 13.

  When scanning the document, the first traveling body 5 and the second traveling body 8 are moved in the sub-scanning direction by the motor. That is, the first traveling body 5 and the second traveling body 8 travel under the contact glass 2, the original is exposed and scanned by the linear light source 3, and the reflected light is reflected on the first reflecting mirror 4, the second reflecting mirror 6, and the like. The light is reflected by the third reflecting mirror 7 and imaged on the CCD 9 through the lens unit 10. Thereby, an image reading means is realized.

  The scanner unit 1 includes a printer unit (not shown) that is an image printing apparatus that forms an image on a sheet by, for example, electrophotography, in accordance with image data based on a scanned image of a document read by the scanner unit 1. It is mounted on the digital copying machine 16.

  FIG. 2A is a perspective view showing an upper portion of the digital copying machine 16 on which the scanner unit 1 is mounted. As shown in FIG. 2A, the scanner unit 1 is provided with a pressure plate 17 that can be opened and closed with respect to the contact glass 2, and an open / close sensor 18 that detects opening and closing of the pressure plate 17. In addition to the electrophotographic system, various printing systems such as an ink jet system, a sublimation type thermal transfer system, a silver salt photographic system, and a melt type thermal transfer system can be applied as a printer provided in the digital copying machine 16.

  FIG. 2B is a system diagram in the case where image data captured by a digital camera or a scanner device is subjected to image processing by a PC. A PC 101 and a scanner device 102 and a PC 101 and a digital camera 103 are connected via a network 100. Image data captured by the scanner device 102 or the digital camera 101 is transmitted to the PC 101, and background correction, shadow correction, and distortion correction are performed by a program described later.

  FIG. 3 is a block diagram showing the electrical connection of the control system of the scanner unit 1. As shown in FIG. 3, the control system includes an image processing unit 20 that is a circuit that performs various image processing on image data read by the CCD 9, and a first control unit 19 that controls the entire scanner unit 1. An image read by the CCD 9 and an operation panel 22 for accepting various operations to the digital copying machine 16 and receiving various operations on the digital copying machine 16, which is a circuit for controlling the traveling body 5 and the second traveling body 8. A memory 23 for storing data, predetermined data, and the like is connected. The operation panel 22 is provided with a copy start key for declaring the start of copying.

  Further, in the traveling body control unit 21, the line light source 3, the stepping motor 24 that drives the first traveling body 5 and the second traveling body 8, and the first traveling body 5 and the second traveling body 8 are in the home position. A scanner home position sensor (HP sensor) 25 for detecting whether or not and an open / close sensor 18 are connected.

  FIG. 4 is a block diagram illustrating a basic internal configuration of the image processing unit 20. As shown in FIG. 4, the image processing unit 20 includes an analog video processing unit 26 that performs analog image signal amplification processing and digital conversion processing when a document is read by the CCD 9, a shading correction processing unit 27 that performs shading correction processing, and shading. The digital image signal after the correction processing is configured by an image data processing unit 28 that performs various image data processing such as MTF (Modulation Transfer Function) correction, scaling processing, and γ correction to generate a scan image.

  The image data processing unit 28 includes a background correction unit and a shadow correction unit, and realizes an image processing method described later. The background correction unit and the shadow correction unit will be described in detail through the embodiment. The background correction unit obtains a flat portion of the book original from the pixel value of the scan image of the book original, and converts the background image into a scanned image based on the pixel value of the flat portion. Apply background correction processing. The shadow correction unit corrects shadows near both ends of the binding portion of the scan image based on the pixel value of the flat portion.

  The digital image signal after the image processing as described above is sent via the main control unit 19 to the printer unit when printing, to the facsimile unit when transmitting by facsimile, or to a predetermined storage device when performing OCR processing. To be used for each process.

  As shown in FIG. 5A, the main control unit 19 includes a CPU (Central Processing Unit) 31 that centrally controls each unit. The CPU 31 is a read-only memory that stores a BIOS and the like. A ROM (Read Only Memory) 32 and a RAM (Random Access Memory) 33 that stores various data in a rewritable manner and functions as a work area of the CPU 31 are connected by a bus 34 to constitute a microcomputer. Further, the bus 34 has an interface (I / F) for communication with an HDD 35 storing programs for image processing and control, a CD-ROM drive 36 for reading a CD (Compact Disc) -ROM 37, and a printer unit. ) 38, a NIC (Network Interface Card) for connecting to a LAN (Local Area Network) is connected.

  FIG. 5B shows a hardware configuration diagram of the PC 101. In FIG. 5B, the same functional components as those in FIG. 5A are denoted by the same reference numerals, and the description thereof is omitted. The input / output device 30 is an interface with a display device such as a keyboard or mouse operation or a disk play.

  The CD-ROM 37 shown in FIG. 5A or 5B corresponds to a storage medium in the scope of claims, and stores a program in the scope of claims. That is, the CD-ROM 37 stores a program for causing the PC 101 or the main control unit 19 to execute the background correction step and the shadow correction step.

  The CPU 31 reads the control program stored in the CD-ROM 37 with the CD-ROM drive 36 and installs it in the HDD 35. When the CPU 31 executes programs for performing various image processing described later, the main control unit 19 controls the image processing unit 20 and the image processing unit 20 performs various processes as described later.

  As the storage medium, not only the CD-ROM 37 but also various types of media such as various optical disks such as DVD, various magnetic disks such as floppy disk (registered trademark), semiconductor memory, and the like are used. it can. Alternatively, the program may be downloaded from a network such as the Internet and installed in the HDD 35. In this case, the storage device storing the program in the server on the transmission side is also a storage medium of the present invention. Note that the program may operate on a predetermined OS (Operating System), and in that case, the OS may take over execution of some of the various processes described below, or a word processor. It may be included as part of a group of program files that constitute predetermined application software such as software or an OS.

  Subsequently, the shadow correction in the present embodiment will be described. As shown in FIG. 6, the book document 40 is placed on the contact glass 2 so that its page binding portion (hereinafter simply referred to as binding portion) 41 and the main scanning direction of image reading of the scanner unit 1 are parallel to each other. Has been. That is, part of the book document 40 is in contact with the contact glass 2 as a flat portion, but the binding portion 40 is separated from the contact glass 2 (distance d in FIG. 6).

  When the book document 40 is scanned in the state shown in FIG. 6, distortion occurs in the vicinity of the binding portion 41 as shown in FIG. In addition, the brightness of the both end portions 41A and 41B of the binding portion is not sufficient compared to the vicinity of the center, resulting in a low image. The image processing apparatus according to the present embodiment corrects the background of both end portions 41A and B of the binding portion with high accuracy.

  Since the linear light source 3 of the scanner unit 1 has a finite length, an optical model in which a one-dimensional light source is finitely arranged will be considered. FIG. 8 shows a configuration diagram of a light source and a book document when the book document is scanned. In FIG. 8, the X direction indicates the main scanning direction, the Y direction indicates the sub-scanning direction, and the Z direction indicates the thickness direction of the book document 40. In FIG. 8, only one side of the spread book document 40 is shown.

  In FIG. 8, a point p (xp, yp, zp) is a three-dimensional coordinate of a predetermined point on the paper surface of the book document 40, (nx, ny, nz) is a normal vector at that point, and (dy, dz) is The position of the line light source 3 in the YZ plane is shown. The line light source 3 is a finite light source arranged in parallel with the X axis and is the exposure lamp of FIG. x1 and x2 indicate the position of the line light source 3 in the X-axis direction.

  The illumination light intensity of the linear light source 3 having a finite length as shown in FIG. 8 can be modeled as if the point light sources are arranged in a line. The illumination light intensity of the point light source attenuates in inverse proportion to the square of the distance from the light source. Therefore, if the position of the point light source is x and the illumination light intensity from the point light source is integrated from x1 to x2, the illumination light intensity at the predetermined point p can be obtained. The reflected light intensity P (xp, yp, zp) from the line light source 3 at a predetermined point p 1 on the surface of the book original 40 can be configured as shown in Equation (1).

Here, a and Δ are the photoelectric conversion parameters inside the CCD, ρ is the reflectance at the point p, and α is the intensity parameter of the point light source.

  Since the cross-sectional shape of the book document 40 is constant in the X-axis direction (no skew distortion), the x component (nx) of the normal vector can be set to 0. When the integral of the expression (1) is solved under this condition, the expression (2) is obtained.

However, mutual reflection occurs on the surface of the actual book original 40, and the mutual reflection intensity can be similarly modeled in view of the fact that this mutual reflection becomes a number of linear light sources having a finite length. . Therefore, actual P (xp, yp, zp) takes the form of the sum of similar expressions with different coefficients (parameters).

  From the above, the optical model Px (xp) of the lightness distribution in a certain one-dimensional image along the main scanning direction on the paper surface of the book document 40 can be generally expressed as Expression (3).

Subsequently, Equation (3) is approximated according to the optical model of the scanner unit 1. In equation (3), if (x1 <x2), near x = x1, (x−x2) ^ 2 → ∞, and the second term in parentheses in equation (3) is approximately 1. Similarly, in the vicinity of x = x2, the first term may be considered to be approximately 1. That is, if the lightness distribution in the one-dimensional image is divided into two on the x1 side and the x2 side in the X-axis direction, the lightness distribution on one side is expressed as in Expression (4).

As shown in FIG. 9, the illumination light intensity according to the equation (4) basically shows a distribution similar to arc tangent (inverse function of tan θ). A part of this distribution represents a change in lightness (one side) in the one-dimensional image.

Arctan has a graph shape having asymptotic lines up and down with respect to θ. In FIG. 9, a center line parallel to the upper and lower asymptotic lines is provided, and parameter a is newly defined by Expression (4) (Expression (1) ( 2) different from a). The contents of each coefficient in Equation (4) are as follows.
a: Distance from the center line to the asymptotic line b: Defines the slope of the curve (when b ≧ 0).

In FIG. 9, the slope that fluctuates to the value b is indicated by a dotted line.
(When b <0, x = x0 is an asymptote, and the outline diverges to ± ∞.)
c: distance from y (brightness) = 0 to the center line x0: x coordinate of a point intersecting the center line a> 0 and x0 is small, corresponding to the brightness distribution on the left side of FIG. 9 in the one-dimensional image, When a <0 and x0 is large, it corresponds to the lightness distribution on the right side of FIG.

  When skew distortion occurs in the actual scanned image or the “C” shape is placed, the one-dimensional brightness distribution is asymmetrical on the left and right, so it is more effective to handle it as divided as shown in FIG. It is.

  Actually, since the brightness does not become negative, the brightness distribution in the one-dimensional image can be normalized to a value of 0 to 1. In this case, a = 1.0 (or -1.0), c = 0.0 can be set in the coefficient of the equation (4). Note that brightness normalization will be described in the brightness profile described later.

  Therefore, if the remaining coefficients b and x0 in the equation (4) can be estimated, the normalized brightness distribution in the one-dimensional image including the end portion of the book document 40 can be obtained, thereby correcting the shadow of the end portion. Is possible.

  As shown in FIG. 7, the scan image of the book document 40 generates a distortion of the shape and a lightness distribution. Therefore, it is preferable to perform distortion correction. Therefore, as shown in FIG. 10, the image processing of the book document 40 is constituted by processing procedures of background correction processing (S1) including shadow correction by equation (4) and distortion shape correction processing (S2). In the present embodiment, the background correction and the shadow correction may be performed at the same time. However, in the following, the process using Equation (4) is particularly referred to as the shadow correction. Further, the distortion shape correction processing will be described after the embodiment.

  In order to estimate the coefficients b and x0, first, a brightness profile for performing background correction processing is obtained.

  First, the background correction process will be described. As it moves away from the contact glass 2 of the scanning unit 1 (closer to the vicinity of the binding unit), the lightness and saturation decrease, while the hue hardly changes. Therefore, the color correction of the binding portion of the scanned image can be performed by enhancing the lowered brightness and saturation so as to be approximately the same as the flat portion of the scanned image.

  As for how much lightness and saturation should be enhanced for each pixel color in the input image, a method of detecting the background color in a flat part and correcting it to match the lightness and saturation is considered. It is done. However, since the background color is generally a low-saturation color such as white or cream, this method cannot sufficiently enhance the saturation of the curved surface portion.

  Therefore, in the present embodiment, the concept of “normalized brightness profile” is introduced, and a background correction method using the concept is proposed. In the present embodiment, the following three background correction processes using the normalized brightness profile are provided.

[Background correction processing 1]
In the background correction process 1, the following processes 1 to 7 are sequentially executed as shown in the flowchart of FIG.
1. Obtain the brightness (Value), saturation (Saturation), and hue (Hue) of the scanned image.
Using the red, green, and blue components of the input image, values of lightness (Value), saturation (Saturation), and hue (Hue) at each pixel are obtained.
The red, green, and blue components at the coordinates (x, y) of each pixel are the R (x, y), G (x, y), B (x, y), brightness, saturation, and hue values, respectively. Let it be V (x, y), S (x, y), H (x, y). V, S, and H can be expressed as follows using R, G, and B.
V (x, y) = 0.3 * R (x, y) + 0.59 * G (x, y) + 0.11 * B (x, y)
C1 (x, y) = R (x, y) -V (x, y)
C2 (x, y) = B (x, y) -V (x, y)
H (x, y) = Tan ^ (-1) (C1 (x, y) / C2 (x, y))
S (x, y) = √ (C1 (x, y) ^ 2 + C2 (x, y) ^ 2)
2. Each pixel is classified into a chromatic color or an achromatic color by using a chromatic / achromatic color determination S (x, y) and an appropriate threshold value St (for example, threshold value St = 15).
If S (x, y) ≤ St, achromatic color S (x, y)> St, if chromatic, 3. Create lightness profile V (x, y) along the direction perpendicular to the binding A brightness profile V (y) is created. In particular,
A histogram is obtained for the one-dimensional image V (x) of V (x, y) at each y, and a brightness range (V1, v2) in which Vt (or more) pixels exist from the brighter side is obtained. The value of Vt is, for example, Vt = (number of pixels in the width of the image) × 0.1.

FIG. 12 shows an example of a histogram of the one-dimensional image brightness V (x). In FIG. 12, the X axis is lightness and the Y axis is the number of pixels. And
-Find the average value of brightness for the range from v1 to v2, and let it be v (y). This is calculated for each line (for each y).

4. Smoothing the brightness profile Smooth the brightness profile v (y) to remove noise.
For each y, the average value of v (y + n) from v (y−n) centered on y is set to the value of v (y). Repeat this several times. For example, repeat 3 to 10 times.

5. Specifically, the lightness of the flat part of the scanned image of the book document 40 is obtained from the lightness profile. Specifically, the lightness of the flat part is calculated from the lightness profile v (y).
FIG. 13 shows an example of the relationship between the pixels of the lightness profile v (y) and the lightness. As shown in FIG. 13, the portion with low brightness is the binding portion of the scanned image.
First,
Create a histogram for the value (brightness) of v (y).
FIG. 14 shows an example of a histogram of v (y). In FIG. 14, the X axis is lightness and the Y axis is the number of pixels. And
Since it is considered that the lightness with the highest frequency corresponds to the lightness of the flat portion (the left and right flat portions in FIG. 13), the average value of the lightness is obtained for the range of ± Vm around the lightness, and is obtained as the flat portion. The brightness is Vflat. For example, Vm is Vm = 2.

6. Calculate the normalized brightness profile vn (y) for obtaining the normalized brightness profile by the following formula.
The other lightness is expressed by a value of 1 or less (expressed as a ratio) so that the value of the flat portion becomes 1.0, and is multiplied by the entire lightness profile and normalized to a range of 0 to 1.
vn (y) = v (y) / Vflat
7. Background correction For each pixel (x, y)
・ If the pixel is chromatic,
S ′ (x, y) = S (x, y) / vn (y)
V ′ (x, y) = V (x, y) / vn (y)
Then, saturation and lightness are corrected, and values of R, G, and B are obtained from H (x, y), S ′ (x, y), and V ′ (x, y).
・ If the pixel is achromatic,
V ′ (x, y) = V (x, y) / vn (y)
As described above, only the brightness is corrected, and the values of R, G, and B are obtained from H (x, y), S (x, y), and V ′ (x, y).

[Background correction processing 2]
In the background correction process 2, as shown in the flowchart of FIG. 15, the following processes 1 to 6 are sequentially executed. Unlike the background correction process 1, the background correction process 2 directly corrects the R, G, and B values without separating them into chromatic colors / achromatic colors.

1. Obtaining the lightness (Value) Using the red, green and blue components of the input image, the lightness (Value) value at each pixel is obtained. The red, green, and blue components at the coordinates (x, y) are R (x, y), G (x, y), B (x, y), and the brightness value is V (x, y).
The brightness value V is, for example,
V (x, y) = 0.3 * R (x, y) + 0.59 * G (x, y) + 0.11 * B (x, y)
It becomes. In the case of a gray scale image, the pixel value itself is treated as V (x, y) for processing.

2. Creation of brightness profile V (x, y) is used to create a brightness profile v (y) along the direction (y direction) perpendicular to the binding portion. In particular,
A histogram is obtained for a one-dimensional image V (x) of V (x, y) at each y, and a brightness range (V1, v2) in which Vt (or more) pixels exist from the brighter side is obtained (see FIG. 12). For example, Vt is Vt = (number of pixels in the width of the image) × 0.1.
-An average value of brightness is obtained for the range from v1 to v2, and this is defined as v (y).

3. Smoothing the brightness profile The brightness profile v (y) is smoothed to remove noise.
For each y, the average value of v (y + n) from v (y−n) around y is set to the value of v (y). Repeat this several times. For example, repeat 3 to 10 times.

4. The flat partial lightness is calculated from the lightness profile v (y) for obtaining the flat partial lightness from the lightness profile (see FIG. 13).
Create a histogram for the value (brightness) of v (y).
Since the most frequent lightness corresponds to the lightness of the flat part, the lightness average value is obtained for the range of ± Vm centering on the lightness, and this is used as the lightness Vflat of the flat part (see FIG. 14). For example, Vm = 2.

5. The normalized brightness profile vn (y) for obtaining the normalized brightness profile is calculated by the following formula. Multiply the entire lightness profile by a ratio with a flat portion value of 1.0 and normalize to a range of 0-1.
vn (y) = v (y) / Vflat
6. Background correction The values of R, G, and B are directly corrected for each pixel (x, y).
R ′ (x, y) = R (x, y) / vn (y)
G ′ (x, y) = G (x, y) / vn (y)
B ′ (x, y) = B (x, y) / vn (y)
In the background correction processing 2, unlike the background correction processing 1, only V (x, y) needs to be obtained, and chromatic and achromatic color classification processing is not necessary. Therefore, the processing speed is improved and the memory capacity required for the processing is increased. Can be reduced.

[Background correction processing 3]
In the background correction process 3, the following processes 1 to 5 are sequentially executed as shown in the flowchart of FIG. Unlike the background correction processes 1 and 2, the background correction process 3 directly obtains a normalized (R, G, B) profile for each image of the pixel values R, G, B, and uses it to use the corresponding R, G, G and B images are corrected.

1. Profile creation For each pixel value of R, G, B, find the maximum pixel value in each one-dimensional image (determine the histogram, and calculate the average value of the p% pixel values from the larger one). Are r (y), g (y), and b (y). In the case of a grayscale image, the pixel value itself is treated as g (y) and the same processing is performed.

2. Smoothing of profiles The profiles r (y), g (y), b (y) are smoothed to remove noise. For each y, the average value of r (y−n) to r (y + n), g (y−n) to g (y + n), and b (y−n) to b (y + n) around y is expressed as r (y The values are y), g (y), and b (y). Repeat this several times. For example, repeat 3 to 10 times. 3. Calculate the flat portion from the profiles r (y), g (y), and b (y) for obtaining the flat portion lightness from the profile.
Create a histogram for r (y), g (y), b (y).
Since the most frequent value corresponds to the flat part, the average value is obtained for the range of ± m around that value, and the reference values of the flat part are defined as rflat, gflat, bflat. For example, m = 2.

4. Calculate normalized brightness profiles pr (y), pg (y), and pb (y) to obtain a normalized profile by the following formula. The entire profile is multiplied by a ratio with a flat portion value of 1.0 and normalized to a range of 0-1.
pr (y) = r (y) / rflat
pg (y) = g (y) / gflat
pb (y) = b (y) / bflat
5. Background correction The values of R, G, and B are directly corrected for each pixel (x, y).
R ′ (x, y) = R (x, y) / pr (y)
G ′ (x, y) = G (x, y) / pg (y)
B ′ (x, y) = B (x, y) / pb (y)
In the background correction process 3, unlike the background correction processes 1 and 2, it is not necessary to obtain V (x, y), S (x, y), and H (x, y), and a chromatic color / achromatic color classification process is also required. Therefore, the processing speed can be improved and the memory capacity required for processing can be reduced.

  Next, a process for performing shadow correction on the scanned image of the book document 40 will be described. As described above, if the coefficients b and x0 of Expression (4) can be obtained, the normalized lightness distribution in the one-dimensional image can be estimated. If a normalized lightness distribution is obtained, the shadow can be corrected by multiplying the lightness (or pixel value) of the input image by the inverse of the distribution in the same manner as the background correction.

  Since the coefficients b and x0 have different values for each one-dimensional image (that is, with respect to the y coordinate), it is suitable to obtain using the normalized brightness distribution in the actual image. The desired brightness distribution is the background color. For example, since the end portion of a book document in a one-dimensional image is generally a blank portion and often has a background color, the normalized brightness of this portion can be used.

  However, if there is only one data, it is difficult to determine the values of both the two variables b and x0. Thus, it is clear that x0 is slightly outside the edge of the book document, and therefore an appropriate value may be set in advance. Then, the normalized brightness distribution in the one-dimensional image is estimated by determining the value of b from the normalized brightness of the edge portion of the book document.

  FIG. 17 is a flowchart showing a shading correction processing procedure in this embodiment. The basic shadow correction procedure for a one-dimensional image is as follows.

  First, a normalized brightness profile t (y) is obtained for the input scanned image of the book document by any one of the background correction processes 1 to 3 (S11).

  Next, a one-dimensional brightness distribution V (x) is obtained for a one-dimensional image (R (x, y), G (x, y), B (x, y)) at a certain y coordinate (S12).

  Next, the x-coordinates xl and xr at both ends of the book document are estimated for the one-dimensional image for which the one-dimensional brightness distribution V (x) is obtained (13). Specifically, a convolution operation is performed using the one-dimensional difference filter shown in FIG. 18, and two locations having large absolute values are defined as xl and xr. The x coordinate at the center of xl and xr is assumed to be xc.

  Then, t (y) is used to normalize V (x) to v (x) (S14).

  Next, a normalized lightness distribution on the left side (xl to xc) of the one-dimensional image is estimated (S15). Specifically, the search is performed from xl to xc, and the x coordinate xp satisfying v (x)> thres (for example, 0.8) and the value pix of v (x) at that time are obtained. The coefficient x0 is X0 = xl− (xr−xl) × Δ. Δ is, for example, 0.1. Then, if xp, pix, x0 are input to the equation (4), the value of b can be calculated. In Expression (4), xp is “x”, and pix is lightness.

  When the value of b is calculated, the lightness distribution p (x) of xl to xc is obtained by setting the values of x0 and b in the equation (4).

  Similarly, the lightness distribution p (x) for xr to xc is calculated (S16).

Next, the shadow is corrected using p (x). That is, in addition to the background correction described above, the shadow due to insufficient illumination at both ends is corrected. For example,
R ′ (x, y) = R (x, y) / (rn (y) * p (x))
G ′ (x, y) = G (x, y) / (gn (y) * p (x))
B ′ (x, y) = B (x, y) / (bn (y) * p (x))
It becomes.

  As described above, according to the present embodiment, when the book document 40 is scanned, it is possible to correct the background based on the brightness of the flat portion and to correct the shadow due to insufficient illumination at both ends.

In the first embodiment, the background correction and the shadow correction are performed over the entire area in the sub-scanning direction. However, in this embodiment, a case where the image processing speed is increased by limiting the range in which the background correction and the shadow correction are performed will be described.
The correction range can be limited by detecting the position at which the book document starts to be bent.

  The position at which the book document begins to be bent is detected by detecting the range of the curved surface portion of the book document using the normalized lightness profile, and only that portion is corrected. Note that the range of the curved surface portion may be detected using a page outer shape detected by processing to be described later.

  For the curved part of the book manuscript, both sides of the spread page in the scanned image of the book manuscript should detect the position of the “start of bending” that changes from the flat part to the curved part, so that the curved part is between them. Can do. FIG. 19 is a flowchart showing a procedure for detecting a position at the beginning of bending.

  1. First, the brightness profile in the y-axis direction and the brightness of the flat portion are obtained. FIG. 20 shows an example of a brightness profile at a certain position in the y direction. The vicinity of the center where the lightness indicates a concave portion is a binding portion.

  2. Next, the barycentric position of the distribution of “flat portion lightness-lightness profile” in FIG. 20 is obtained. The gravity center position is, for example, the gravity center position when the concave portion is approximated to a triangle.

  3. Next, from the y coordinate of the center of gravity position, the value of the brightness profile is searched to the left and right, and two y coordinates (y1 and y2 in the figure) having the brightness of “flat portion brightness−δ (for example, δ = 10)” are obtained. . Let this y coordinate be the starting position of the turn.

  If the background correction and shadow correction of the first embodiment are performed only between two y coordinates, the image processing can be speeded up.

  Incidentally, it is not efficient to perform background correction and shadow correction in all main scanning directions between two y coordinates (for all curved surface portions between two y coordinates). Therefore, as shown in FIG. 21, the curved surface portion between two y coordinates is divided into an appropriate number of regions, and a normalized brightness distribution is obtained only for a one-dimensional image on the boundary line. And about the one-dimensional image in the meantime, the corresponding normalized brightness distribution is calculated | required by linear interpolation. By this method, it is possible to perform correction processing at higher speed.

  In addition, the color correction is performed at the same time as the shadow correction near both ends of the binding portion so far, but the shadow correction near both ends of the binding portion can be applied after the background correction is performed. Even in this case, the correction procedure is not changed. FIG. 22 is a flowchart of a correction procedure for correcting shadows near both ends of the binding portion after the background correction is performed.

  First, the coordinates of both end points of the binding portion are obtained by processing such as detecting the outer shape of the book document 40 (S21). The detection of the outer shape of the book document 40 will be described later.

  Next, a brightness image is obtained from the scanned image, and a brightness profile is calculated (S22). Similar to FIG. 20, the position at the beginning of bending is detected.

  Next, as shown in FIG. 21, the curved surface portion sandwiched between the starting positions of the bending is divided into M pieces in the y-axis direction (S23). Then, the positions of both end points of the book document on the boundary line are obtained from the scanned image.

Next, the following correction processing is performed for each single color image of red, green, and blue (S24).
-The input image is divided into two parts vertically from the coordinates of the end points of the binding part.-Each of the upper and lower images is divided into N parts in the horizontal direction and the background correction is performed.-The corrected upper and lower images are combined. The brightness distribution of the one-dimensional image on the boundary line is obtained, and shading correction at both ends of the binding portion is performed only for the curved surface portion (S25).

  In this embodiment, image processing for performing background correction processing and shadow correction by dividing a scanned image into a plurality of regions in a direction perpendicular to the binding portion will be described.

  First, as shown in FIG. 23, the scanned image is vertically divided into the binding portion 41 of the book document 40 and divided into a plurality of regions L in the X-axis direction (that is, with a line having the Y-axis direction as the length direction). To divide). Then, any one of the background correction processes 1 to 3 is performed on each area L, and the shading correction at both ends is performed. In the example of FIG. 23, the X-axis direction is divided into five.

  This embodiment is suitable when the scanned image is skewed or the book document is in the “C” arrangement. In such a scanned image, since the background color profile of the image changes in the X-axis direction, the entire scanned image is not corrected using the same profile, but the image is divided in the X-axis direction, A method of obtaining a profile in the region L and correcting the profile is effective.

  According to this embodiment, even when the profile of the pixel value changes in the direction parallel to the binding portion, the background color can be accurately corrected by obtaining the profile from each of the changing areas.

  In the present embodiment, image processing for performing background correction processing and shadow correction by dividing a scanned image into a plurality of regions in a direction parallel to the binding portion will be described.

  First, as shown in FIG. 24, the scanned image is divided into a plurality of regions L in a direction parallel to the length direction of the binding portion 41 of the book document 40 (that is, divided by a line having a length direction in the X-axis direction). To do). Any one of the background correction processes 1 to 3 is performed on each region L, and shading correction at both ends is performed. In the example of FIG. 24, the region is divided into two parts above and below the line of the binding portion 41.

In this processing, both spread pages are for the book document 40 having a different background color, and since the background color is different between the left and right or upper and lower pages, the background color profile changes in the Y-axis (vertical) direction of the image. Therefore, the correction is not performed using the same profile, but the image is divided in the Y-axis direction, the profile is obtained in each region L, and correction is performed. Specifically, the following 1.2. By processing.

  1. In the input scan image, the positions of both ends of the binding portion 41 are detected (a straight line of the binding portion 41 is obtained). The detection of this position may be detected based on the page outline of the book document 40 (a part where the outline enters the innermost side is detected) or may be detected based on the density at the center of the image. (The darkest part) or a change in brightness may be used for detection. The page outline detection process will be described later.

  The input image is divided vertically by the straight line obtained in 2.1, and background correction and shadow correction are performed.

  According to the present embodiment, even when both spread pages have different background colors, the brightness profile can be obtained from each page and the background color can be accurately corrected.

  This embodiment is a combination of the third and fourth embodiments. As shown in FIG. 25, the length direction (X-axis direction) of the binding portion 41 of the book document 40 and the length direction of the binding portion 41 are perpendicular to each other. Each of the regions L is divided into a plurality of regions L in the direction (Y-axis direction), and any one of background correction processes 1 to 3 is performed on each region L, and shadow correction is performed on both ends. In the example of FIG. 25, the division in the X-axis direction is divided into five, and the division in the Y-axis direction is divided into two above and below the line of the binding portion 41.

  The processing of the present embodiment is for a book document 40 that is skewed or in a “C” configuration and that has a background color that differs in the left and right or upper and lower pages. Since the background color of the book manuscript 40 is different in the X and Y axis directions of the image because the background color differs between the skew and the “C” shape and the upper and lower pages, the same profile is used. It is effective to divide an image in both directions of the X and Y axes, obtain a profile in each partial image, and perform correction instead of using the correction.

  As described above, the image processing apparatus according to the present embodiment can correct the background of the binding portion of the book document 40 and can also correct shadows near both ends of the binding portion.

[Distortion correction]
Subsequently, the distortion shape correction in step S2 of FIG. 10 will be described.

  FIG. 26 is a flowchart for explaining an outline of the distortion shape correction process in step S2. In step S2, page outline / ruled line / character line extraction processing is performed for the book document 40 in the scanned image (step S201), and image distortion correction processing for the scanned image of the book document 40 is performed (step S301).

First, in step S201, page outline / ruled line / character line extraction processing is executed. FIG. 27 is a flowchart schematically showing the flow of page outline / ruled line / character line extraction processing.

Extraction of page outline from scanned image (S211)
First, the process of extracting the page outline from the scanned image in step S211 will be described. FIG. 28 shows an example of a scanned image in which a page outline exists at the upper end of the scanned image. FIG. 29 is a black pixel histogram on the left side of the binding boundary line of the scan image shown in FIG.

  The X axis of the histogram shown in FIG. 29 indicates the main scanning direction (vertical direction in FIG. 28) of the scan image, and the upper end of the scan image is associated with the left end of the histogram. Note that in the case of a scanned image having a page outline at the lower end, the lower end of the scanned image is associated with the right end of the histogram. Therefore, when a page outline exists at the upper end of the scanned image as shown in FIG. 28, a black band appears at the upper part of the scanned image, so that a high vertical bar appears at the left end of the histogram shown in FIG. In this embodiment, it is determined whether or not a page outline exists in the scanned image using such characteristics.

  More specifically, as shown in FIG. 29, the distance AO from the binding boundary line to the left end of the scan image (left end in FIG. 29) and the height BO of the histogram vertical bar, and the ratios are expressed by the following formula ( When the calculated ratio k is larger than a predetermined threshold, it is determined that a page outline exists in the scanned image.

In addition, when page outlines exist above and below the scanned image, high vertical bars appear at the left and right ends of the histogram. In such a case, scanning is performed based on the high vertical bars at the left and right ends of the histogram. A determination is made as to whether or not a page outline exists in the image.

  If it is determined by the above processing that the page outline exists in the scanned image, the page outline is extracted together with information on which of the upper and lower sides of the left and right pages the page outline exists, and temporarily stored in the RAM 33. Remember.

  Note that the process of determining whether or not a page outline exists in the scanned image is executed for each of the left and right pages with the binding line boundary line of the scanned image as a boundary.

Extraction of ruled lines from scanned image (S212)
In a succeeding step S212, ruled line extraction processing from the scanned image is executed.

"Detection of ruled line candidates"
FIG. 30 is an explanatory diagram showing an example of a scanned image having ruled lines. In this embodiment, ruled line rectangle extraction is introduced, and ruled lines existing in a scan image as shown in FIG. 28 are extracted as one rectangle. Although details will be described later, there are cases where a ruled line cannot be extracted by simply performing rectangle extraction, and therefore rectangle extraction with restrictions on run registration is performed.

  FIG. 30 shows the result of performing rectangular extraction on the binarized image. As shown in FIG. 30, a portion where black pixels are connected is extracted as one rectangle. If a ruled line as shown in FIG. 30 exists, it is extracted as an elongated rectangle in the sub-scanning direction. Therefore, the ruled line is determined based on the presence or absence of the elongated rectangle and the shape (length / aspect ratio) and position of the extracted rectangle. The presence / absence is determined.

  However, there are cases where a ruled line cannot be extracted by simply performing rectangle extraction. As shown in FIG. 31, when the ruled line is in contact with noise, a rectangle including noise is extracted. In the case of a scanned image including a table as shown in FIG. 32, the ruled line in the sub-scanning direction intersects with the ruled line in the main scanning direction, so that the entire table is extracted as one rectangle and the ruled line cannot be extracted alone.

[Rectangle extraction with restrictions on run registration]
Therefore, in order to extract the ruled lines alone, rectangle extraction with restrictions on run registration is performed. For an image in which noise is in contact with a ruled line as shown in FIG. 31, if only a run with a predetermined value in the main scanning direction (vertical direction) is registered and a rectangle is extracted, a ruled line is formed as shown in FIG. The black pixel to be registered is a run to be registered, but the black pixel constituting the noise is not registered as a run. Since the rectangle extraction is performed on black pixels constituting the ruled line, the ruled line can be extracted independently.

  In addition, when extracting a ruled line in a rectangle, there is a method of extracting a rectangle only for a run that is long in the sub-scanning direction (horizontal direction), but in this method, a distorted portion near the binding portion is not included in the rectangle. However, by using the method of the present embodiment, a distorted portion near the ruled line binding portion can be included in the rectangle, and a more accurate ruled line position and length can be detected.

"Rectangle integration"
If rectangle extraction is performed on an image such as a table where a ruled line in the sub-scanning direction (horizontal direction) and a ruled line in the main scanning direction (vertical direction) intersect, the ruled line in the main scanning direction is not registered as a run. Rectangles are extracted in the direction. Then, as shown in FIG. 34, although there is a long ruled line in the sub-scanning direction, the ruled line is not extracted as one rectangle but is extracted as a plurality of finely divided rectangles.

  Therefore, rectangle integration is performed. The rectangles whose distances in the sub-scanning direction are equal to or less than a certain value are integrated. FIG. 35 shows an example in which rectangle integration is performed. In the rectangle integration, as shown in FIG. 35, the rectangles that have been cut into pieces are integrated into one rectangle, and a ruled line rectangle is extracted. This integration of rectangles is an effective method because the entire ruled line is extracted as one rectangle even if it is applied to a blurred ruled line or a dotted ruled line.

"Selecting the best ruled line"
Next, rectangle extraction in which only runs less than a certain value are registered is performed, and the presence or absence of ruled lines is determined based on the presence or absence of a long and narrow rectangle in the sub-scanning direction. Such a ruled line presence / absence determination is performed at each of the four locations of the upper left, lower left, upper right, and lower right of the image. For example, in the case of the image shown in FIG. 36, no ruled line exists only in the upper left. If there are a plurality of ruled lines at a certain location, the ruled lines used for correction are determined in the following priority order.

1. For example, in the case of the lower right side of the image shown in FIG. 36, a ruled line that penetrates to the vicinity of the binding portion is used for correction.

2. For example, in the case of the upper right side of the image shown in FIG. 36, the ruled line with the longer length is used for correction since both the ruled lines are biting into the vicinity of the binding portion.

3. For example, in the case of the lower left side of the image shown in FIG. 36, both ruled lines cut into the vicinity of the binding portion, and the lengths are almost the same, so the ruled line located outside the image is used for correction. The

"Coordinate value detection of optimal ruled line"
After selecting the optimum ruled line as described above, the coordinate value of each ruled line is detected. The position coordinates of the ruled line can be obtained from the extracted rectangular coordinates. As a special example, when the position of a rectangle elongated in the sub-scanning direction is in contact with the upper end or the lower end of the image, the rectangle is not regarded as a ruled line in consideration of the possibility that the rectangle is noise. In addition, when elongated rectangles are extracted from the left and right pages (for example, upper left and upper right, lower left and lower right), depending on the image, the rectangles of the left and right pages may be integrated across the binding portion. Then, since a long and narrow rectangle is extracted from the entire horizontal image, when such a feature is found in the extracted rectangle, the rectangle is divided at the binding position.

  If it is determined by the above processing that a ruled line exists in the scanned image, the ruled line is extracted together with information on which position of the ruled line exists on each of the left and right pages, and temporarily stored in the RAM 33.

Extraction of character lines from the scanned image (S213)
In the subsequent step S213, a character line extraction process from the scanned image is executed. In the present embodiment, first, it is determined whether the character line in the scanned image is a vertically written character line or a horizontally written character line.

"Determination of character lines"
A method for determining whether a character line in a scanned image is a vertically written character line or a horizontally written character line will be described. Here, FIG. 37 is a black-and-white inversion number histogram in the sub-scanning direction of the scan image shown in FIG. The horizontal axis in FIG. 37 represents the black pixel in the sub-scanning direction (left-right direction) (the pixel in which the density value is darker than the predetermined density value among the pixels obtained by inverting the scan image in black and white) in the main scanning direction. The position indicates the position, and the vertical axis in FIG. 37 indicates the number of black pixels for each position.

  FIG. 38 is a black-and-white inversion number histogram of the image shown in FIG. 50 in the main scanning direction. The horizontal axis of FIG. 38 shows the black pixel in the main scanning direction (vertical direction) (the pixel whose density value is darker than the predetermined density value among the pixels obtained by reversing the scan image in black and white) in the sub-scanning direction. The position is shown, and the vertical axis in FIG. 38 shows the number of black pixels for each position. When the characters in the image are horizontal scans as shown in FIG. 50, the sub-scanning histogram as shown in FIG. 37 changes drastically, but the main-scanning histogram as shown in FIG. Few. Although not particularly illustrated, when the character line in the scanned image is a vertically written character line, the histogram in the main scanning direction changes drastically, but the change in the histogram in the sub-scanning direction is small.

Specifically, the discrimination method as described above is realized by the following equations. First, an average value mean _H of the histogram values Pnt (y) at the position in the main scanning direction y is calculated by the following equation (6). Here, height is the height of the image.

An average value mean _H of the histogram values Pnt (y) at the position in the main scanning direction y is calculated. Here, height is the height of the image.

Then, the variance σ _{H in} the main scanning direction of the histogram in the sub-scanning direction is obtained by the following expression (7).

Similarly, the average value mean _V of the histogram values Pnt (x) at the position in the sub-scanning direction x is calculated by the following equation (8). Here, width is the width of the image.

Then, the variance σ _v regarding the sub-scanning direction of the histogram in the main scanning direction is obtained by the following equation (9).

As described above, when the character line in the scanned image is a horizontally written character line, the variance σ _H in the main scanning direction of the histogram in the sub-scanning direction is larger than the variance σ _{v in} the sub-scanning direction of the histogram in the main scanning direction. . Conversely, if the character line in the scanned image is a vertical text line, variance sigma _v in the sub-scanning direction of the main scanning direction of the histogram is larger than variance sigma _H in the main scanning direction in the sub-scanning direction histogram . That is, by comparing the variance σ _H and the variance σ _v , it is possible to determine whether the character line in the scanned image is a vertically written character line or a horizontally written character line.

  The reason why the black and white inversion number histogram is used to determine whether the character line in the scanned image is a vertically written character line or a horizontally written character line is to avoid confusion between the character line and the photograph portion. This is because, in general, when the values of the black pixel histogram are approximately the same, the value of the black / white inversion number histogram is larger in the character region than in the photo region.

"Coordinate detection of horizontal text lines"
After determining the character line as described above, first, the coordinates of each horizontal character line are detected. In detecting the coordinates of a horizontally written character line, a circumscribed rectangle extraction process for each character is performed, and a horizontally written character line extraction process is performed. Since the character recognition process is a well-known technique, its description is omitted. Here, FIG. 39 shows an example of the result of the character circumscribed rectangle extraction process and the character line extraction process of the scanned image. Then, the coordinates of the center point of the circumscribed rectangle of each character are regarded as the coordinates of the character, and the coordinates of the horizontal character line are detected.

"Selecting optimal horizontal text"
Next, a horizontal character line optimal for distortion correction is selected from the extracted horizontal character lines. When a plurality of horizontally written character lines are detected, it is necessary to select which horizontally written character line is used for distortion correction. An example of the optimum horizontal writing character line selection criterion is basically the same as the optimal ruled line selection criterion described above, and the horizontal writing character line length BC is set to a predetermined threshold as shown in FIG. On the condition that a part C of a horizontal writing line is covered within a long and constant width area (shaded area in FIG. 40) on either side of the binding boundary line, either upper or lower page Select the horizontal text line closest to the outline. Here, B is the center of the leftmost rectangle of the character line, and C is the center of the rightmost rectangle. The optimum horizontal writing character line may be selected by selecting the horizontal writing character line closest to the page outline from each of the left and right pages, or the left and right pages are further divided into upper and lower parts. A horizontally written character line that is closest to the page outline one by one in the block may be selected.

  Regarding the above two conditions (the length of the horizontal writing character line is longer than a predetermined threshold, and a part of the horizontal writing character line is applied within the left and right constant width regions across the binding portion boundary line). , But not both of them, only one of them may be satisfied. In the above example, “closest to the page outline” is used as the selection criterion. However, the selection criterion is not limited to this, and “horizontal writing character line has the largest curve” may be used. Here, “curve of horizontally written character line” is expressed by the difference in the coordinate values in the main scanning direction of the center coordinates of the character circumscribed rectangle at both ends of the horizontally written character line.

"Determining the coordinate value of the optimal horizontal text line"
When the optimum horizontal writing character line is selected, the coordinate value (in the main scanning direction) of the horizontal writing character line is determined. The coordinate value (in the main scanning direction) of the horizontal writing line is obtained by connecting the center points of the circumscribed rectangles in the horizontal writing line and approximating the straight line part and the curved part and extracting them (main scanning). The coordinate value of the direction will be determined. More specifically, D shown in FIG. 40 is a binding boundary line, a coordinate value (in the main scanning direction) is estimated with a polynomial approximation curve between BDs, and an approximation between A and B at the leftmost end A coordinate value (in the main scanning direction) is estimated with a straight line value.

"Eliminating inappropriate horizontal text"
Finally, remove inappropriate horizontal text lines. This is because, when the coordinate value is estimated by polynomial approximation as described above, if the shape of the estimated curve by polynomial approximation is inappropriate, the distortion may increase at the time of correction. This eliminates horizontal text lines. An example of an inappropriate approximate curve shape is the same as that of the ruled line described above, and although not particularly illustrated, the curve may go outwards from the book document, or it may bite inwardly beyond the center line. This is the case.

  If the horizontal writing character line is excluded because the shape of the estimated curve is inappropriate, the optimal horizontal writing character line is selected again, and the above processing is repeated.

  If it is determined by the above processing that there is a horizontally written character line in the scanned image, the horizontally written character line is extracted together with information indicating in which position the horizontally written character line exists on each of the left and right pages, and stored in the RAM 33. Memorize temporarily.

"Extracting horizontal text lines based on vertical text lines"
Next, a horizontal writing character line is extracted from each vertical writing character line.
FIG. 41 is a flowchart schematically showing the flow of processing for extracting horizontally written character lines from each vertically written character line. As shown in FIG. 41, first, a line cut-out rectangle of a vertically written character line is extracted (S221). In addition, since the extraction process of the line cut-out rectangle of a vertically written character line can use the well-known technique generally used by OCR etc., the description is abbreviate | omitted. FIG. 42 is an explanatory diagram exemplarily illustrating the extracted row cutout rectangle.

  Next, a vertical writing character line having the maximum (or minimum) y coordinate at the beginning (or the end) of the vertical writing character line is extracted, and further, the vertical (with the leading (or tail)) existing within a predetermined distance range therefrom. A writing character line is extracted (S222). More specifically, in the example shown in FIG. 42, as shown in FIG. 43, the vertical writing character line having the maximum y coordinate of the first character of the vertical writing character row is the vertical writing character line indicated by A. . A line head character existing within a predetermined distance range h from the head position is a character indicated by a black circle “●” in FIG. That is, only the vertically written character lines including the character indicated by the black circle “●” are extracted, and the other vertically written character lines B and C are excluded. Note that h is a constant determined by the resolution of the scanned image.

  Next, a histogram is constructed for the y-coordinate at the beginning (or end) of the extracted vertical writing character line (S223). In FIG. 44, a vertically written character line D close to the left end of the page is set as a reference line, and the first y coordinate (yD) is set as a reference coordinate. Hereinafter, the number of vertically written character lines having a head within a range of a constant width d (for example, ½ of the average width of the extracted vertically written character lines) with respect to yD is set as a histogram value related to yD. In FIG. 44, a vertically written character line whose head exists within the range of a dotted line sandwiching a straight line indicating yD vertically is the target. Therefore, the vertical writing character line E on the right side of the vertical writing character line D near the left end of the page is out of the range. In this way, when a vertically written character line that does not include the beginning of the target range of the existing reference coordinates appears, the vertically written character line is set as a new reference line, and the beginning coordinate is set as a new reference coordinate (here, YE). Also, since the line start coordinate of the vertical writing character line F adjacent to the right of the vertical writing character line E is included in the target range of yD, the value of the histogram relating to yD is counted up by 1 without providing a new reference coordinate. .

  Thereafter, the same processing is continued toward the binding portion boundary line. As a result, in the example shown in FIG. 44, the vertical writing character lines included in the target range of yD are surrounded by a hatched rectangle, and the vertical writing character lines included in the target range of yE are shaded. (The reference line, the reference coordinates, and the target range are also determined for vertically written character lines surrounded by other rectangles, but are omitted in FIG. 44). In addition, although the vertical writing character line G which should be irrelevant is included in the target range of yD, this is excluded in the next step S224.

  Subsequently, among the vertically written character lines included in the target range of the reference line corresponding to the maximum value in the histogram configured in step S223, the vertically written character line at the left end (or right end) of the page. Using (reference line) as a start line, a vertically written character line having a leading (or trailing) y coordinate close to the binding boundary line is extracted (step S224). In FIG. 44, the maximum number of character lines included in the target range of the reference coordinate yD is seven, so the leftmost vertical writing character line D is the start line, and binding is started from the starting line (vertical writing character line D). A vertically written character line with the leading y coordinate close to the partial boundary line is extracted.

  By the way, when extracting a vertically written character line whose leading y coordinate is close to the bounding line from the start line (vertically written character line D), a distortion occurs in the portion where the image is not distorted. The processing content is switched with the part that is present.

First, processing in a portion where image distortion has not occurred will be described with reference to FIG. In a portion where the image is not distorted, a vertically written character line that satisfies the following two conditions is extracted with reference to the target line H.
1. With respect to the positive direction of the y coordinate (the upward direction in FIG. 45), there is a head of a vertically written character line that is extracted within a certain range b1 (for example, 1/2 of the average character line width) from the head position of the target line H. 1. What to do With respect to the negative direction of the y coordinate (the downward direction in FIG. 45), a predetermined angle (here) with respect to the positive direction of the x coordinate (the direction toward the binding boundary) viewed from the head position of the target row H Then, the head of the vertical writing character line to be extracted exists within the range of the angle of the straight line (b2 / a1). That is, the head of the vertical writing character line I next to the target row H is the above Since it is out of the range, it will be excluded, but the beginning of the next vertically written character line J will be extracted because it exists within the range. Hereinafter, the same processing is continued with the vertically written character line J as a new target line.

Next, processing in a portion where image distortion has occurred will be described with reference to FIG. In a portion where the image is distorted, a vertically written character line that satisfies the following two conditions is extracted with the target line L as a reference.
1. With respect to the negative direction of the y coordinate (the downward direction in FIG. 46), a predetermined angle (here) with respect to the positive direction of the x coordinate (the direction toward the binding boundary) viewed from the head position of the row of interest L The angle is represented by the slope of the straight line (b3 / a2). However, in consideration of the fact that the beginning of the character line intrudes into the inside of the page at the part where the distortion occurs, b2 / a1 < 1. The head of the vertically written character line to be extracted exists within the range of (b3 / a2). The slope (b4 / a2) of the straight line connecting the head position of the target line L and the head position of the vertical writing character line to be extracted is the slope of the straight line connecting the head position of the target line L and the head position of the previous extracted line K (b5). / A3) larger than a value obtained by subtracting a constant value α. That is, “b4 / a2> b5 / a3-α” is satisfied (basically “b4 / a2> b5 / a3” may be satisfied, but a constant value α is introduced in consideration of an error. α is a predetermined value)
That is, the beginning of the next vertically written character line M of the line of interest L is excluded because it is outside this condition, but the beginning of the next vertically written character line N is extracted because it satisfies the condition. Hereinafter, the same processing is continued with the vertically written character line N as a new target line.

  The problem here is how to identify the non-distorted portion and the distorted portion as follows. That is, if yC at the beginning of the target line and the next extracted line are yC and yN, respectively, if “yN−yC” is a certain value (for example, ¼ of the average character line width) or more, It is assumed that the part is distorted.

  The vertically written character line extracted from FIG. 44 by the above method is shown by being surrounded by a hatched rectangle in FIG.

  Finally, an approximate curve polynomial relating to the position coordinates of the beginning (or end) of the extracted vertical writing character line is calculated (step S225). In the case of forming the outer shape by connecting the heads of the extracted line-cut rectangles of each vertical writing character line, as shown in FIG. 48, based on the center point of the upper side of the line-cutting rectangle of each vertical writing character line to be connected, An approximate curve polynomial is calculated regarding the position coordinates of the beginning of the extracted vertical character line. Also, in the case of forming the outer shape by connecting the end of the line cut rectangle of each extracted vertical character line, as shown in FIG. Based on this, an approximate curve polynomial is calculated regarding the position coordinate at the end of the extracted vertical writing line.

  Finally, an inappropriate vertical character line outline is eliminated. This is because, when the coordinate value is estimated by polynomial approximation as described above, if the shape of the estimated curve by polynomial approximation is inappropriate, the distortion may increase at the time of correction. This eliminates the outline of vertical character lines. An example of an inappropriate approximate curve shape is the same as in the case of the ruled lines and horizontal text lines described above, and although not particularly illustrated, the curve is directed to the outside of the book document, or larger than the center line. This is the case when it bites inward.

  If the outline of the vertically written character line is excluded because the shape of the estimated curve is inappropriate, this means that there is no outline of the vertically written character line for distortion correction.

  If it is determined by the above processing that the outline of the vertical text line exists in the scanned image, the vertical text will be displayed along with information on where the vertical text line outline exists on the left and right pages. The outline of the row is extracted and temporarily stored in the RAM 33.

  In the following, it is assumed that the outer shape of a horizontal character line and a vertical character line is treated as a character line.

  As described above, the page outline / ruled line / character line extraction process (step S201) in FIG. 26 is completed by the processes in steps S211 to S213.

  In a succeeding step S202, an image distortion correction process is executed. FIG. 49 is a flowchart showing an outline of the distortion correction supplement processing.

  In general, the distortion correction supplement processing is any one of page outline / ruled line / character line positioned near the upper side (or lower side) of the scanned image as a reference line (reference line) for distortion correction (expansion). (Step S301: reference line selection process), a page corresponding to the reference line and positioned in the vicinity of the upper side (or lower side) of the scanned image as a reference line for calculating the correction rate (expansion rate) Processing for selecting any one of outline / ruled line / character line (step S302: reference line selecting process), and when the reference line is a ruled line or character line, to minimize the loss of image information below the reference line A process for calculating a virtual page outline (step S303: virtual page outline calculation process), a process for performing a decompression process on the scanned image based on the virtual page outline to correct distortion in the main scanning direction ( (Step S304: main scanning direction distortion correction process), and a process of performing decompression processing on the scanned image based on the character circumscribed rectangle of the corrected image to correct distortion in the sub scanning direction (step S305: sub scanning direction distortion correction process). Has been. Since the process in step S202 is known, detailed description thereof is omitted (for details, see Japanese Patent Laid-Open No. 2003-69807).

  As described above, according to the image processing apparatus of the present embodiment, it is possible to correct shadows near both ends of the binding portion in a scanned image of a book document by modeling the illumination intensity of a finite line light source. it can. Moreover, the background of a binding part can be correct | amended using the brightness of a flat part by producing a brightness profile. Furthermore, it is possible to correct the distortion of the binding portion of the scanned image.

FIG. 2 is a longitudinal front view showing a configuration of a scanner unit of a digital copying machine. It is a perspective view which shows the upper part of the digital copying machine carrying a scanner part. It is a block diagram which shows the electrical connection of the control system of a scanner part. It is a block diagram which shows the basic internal structure of an image process part. It is a hardware block diagram of a main control part. It is a figure which shows the book original document mounted on the contact glass. A scan image in which distortion occurs in the vicinity of the page binding portion is shown. It is a block diagram of a light source and a book document when scanning a book document. It is a figure which shows an example of distribution of illumination light intensity. FIG. 6 is a flowchart showing a procedure for image processing of a book document. It is a flowchart figure which shows the procedure of the background correction process. It is an example of the histogram of one-dimensional image brightness V (x). It is a figure which shows an example of the relationship between the pixel of a one-dimensional image, and a brightness. It is an example of a histogram of brightness v (y). It is a flowchart figure which shows the procedure of the background correction process. It is a flowchart figure which shows the procedure of the background correction process. It is a flowchart figure which shows the process sequence of shadow correction. It is an example of a one-dimensional difference filter. It is a flowchart figure which shows the detection procedure of the position of the start of bending. It is an example of the brightness profile of a flat part and a binding part. It is a figure which shows the scan image which divided | segmented the curved surface part into the appropriate number of area | regions. It is a flowchart figure of the correction | amendment procedure which correct | amends the shadow of the binding part vicinity after performing background correction | amendment. It is a figure which shows the scan image divided | segmented into the some area | region L perpendicular | vertical to the binding part. It is a figure which shows the scan image divided | segmented into the several area | region L in parallel with the length direction of the binding part. It is a figure which shows the scan image divided | segmented perpendicularly | parallel to the binding part. It is a flowchart figure explaining the outline | summary of a distortion shape correction process. It is a flowchart which shows roughly the flow of the extraction process of a page external shape / ruled line / character line. It is explanatory drawing which shows an example of the scanning image which has a page external shape in an upper end. FIG. 51 is a black pixel histogram on the left side of the binding boundary line of the scan image shown in FIG. 50. FIG. It is explanatory drawing which shows the result of having performed rectangular extraction to the binarized image. It is explanatory drawing which shows the case where a ruled line is contacting with noise. It is explanatory drawing which shows the image containing a table | surface. It is explanatory drawing which shows the result of having registered only the run below a fixed value and extracting the rectangle. It is explanatory drawing which shows the case where a rectangle will be extracted to a thin piece in the subscanning direction. It is explanatory drawing which shows the example which performed rectangle integration. It is explanatory drawing which shows the result of having performed rectangle extraction. It is a black-and-white inversion number histogram of the image shown in FIG. 50 in the sub-scanning direction. It is a black-and-white inversion number histogram of the image shown in FIG. 50 in the main scanning direction. It is explanatory drawing which shows an example of the result of the character circumscribed rectangle extraction process of a scan image, and a character line extraction process. It is explanatory drawing which shows selection of the optimal horizontal writing character line. It is a flowchart which shows roughly the flow of the extraction process of the horizontal writing character line from each vertical writing character line. It is explanatory drawing which shows the extracted line cut-out rectangle as an example. It is explanatory drawing which shows illustratively the vertical writing character line in which the head exists within the predetermined distance range. It is explanatory drawing which shows the state which comprises a histogram regarding the y coordinate of the head of the extracted vertical writing character line. It is explanatory drawing which shows the process in the part which has not produced distortion of an image. It is explanatory drawing which shows the process in the part which has produced the distortion of an image. It is explanatory drawing which shows the extracted vertical writing character line. It is explanatory drawing which shows the line cut-out rectangle of a vertical writing character line. It is a flowchart which shows the flow of an image distortion correction process roughly. It is an example of a scanned image of a book document.

Explanation of symbols

1 Scanner unit 2 Contact glass 3 Line light source (exposure lamp)
19 Main control unit 20 Image processing unit 40 Book document 41 Binding unit 41A, B Near both ends of binding unit

Claims (20)

Image reading means for reading a book document placed on the contact glass;
The scan for a position y in a direction orthogonal to a one-dimensional image in a direction connecting both ends of the binding portion is specified based on a pixel value of a scan image of the book document and a pixel value of the book document is used as a reference. A first means of normalizing the brightness of the image to generate a normalized brightness profile;
A background correction means for correcting the background near the binding portion of the scanned image using the normalized brightness profile;
For the lightness distribution for the position x in the direction parallel to the one-dimensional image, the position where the lightness is approximately the median value of the lightness distribution of the one-dimensional image is x0, and the difference between the approximately median value and the lightness that is asymptotically constant a, the gradient of brightness with respect to a position x passing through the approximate median value is b, and the approximate median value of the brightness distribution is c, the brightness is
A second value for giving a predetermined value to x0, estimating b by substituting the lightness at x in the one-dimensional image into the equation, and obtaining the lightness distribution of the one-dimensional image;
Shadow correction means for correcting shadows near both ends of the binding portion of the scanned image using the brightness distribution ;
An image processing apparatus comprising: The second means performs a filter operation on the pixel value of the one-dimensional image, estimates the coordinates xl and xr at both ends of the scanned image, and sets a position outside the coordinate xl or outside the coordinate xr to the value of x0. A predetermined value ,
The image processing apparatus according to claim 1. The background correction means obtains brightness, saturation and hue from RGB values for each pixel,
Determine whether each pixel of the book manuscript is chromatic or achromatic,
In the case of a chromatic color, perform background correction by the normalized brightness profile for both saturation and brightness,
In the case of an achromatic color, the background correction by the normalized brightness profile is performed only for the brightness,
For each pixel, an RGB value is obtained from lightness, saturation, and hue .
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The background correction means obtains brightness for each pixel,
A background correction based on the normalized brightness profile is performed on each of the R value, the G value, and the B value of the pixel value .
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The background correction means
Obtaining the normalized brightness profile of each of the R value, G value, and B value of the pixel value;
For each of the R value, the G value, and the B value, background correction based on the normalized brightness profile of each of the R value, the G value, and the B value is performed.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The shadow correction means includes
In correspondence with the position y in the direction orthogonal to the one-dimensional image, a lightness profile indicating the lightness distribution of the scanned image is generated,
A predetermined area is identified around a position y where the brightness profile shows a downward peak in the binding portion of the scanned image,
Using the brightness distribution, only the region of the scanned image is corrected for shadows near both ends of the binding portion,
2. The image processing apparatus according to claim 1, wherein: The shadow correction means includes
Dividing the scanned image of the region into a plurality of regions parallel to the one-dimensional image;
The second means obtains the brightness distribution of the one-dimensional image only for the one-dimensional image of the boundary of the region after the division,
Complementing the lightness distribution in a direction orthogonal to the one-dimensional image, and correcting shadows near both ends of the binding portion of the scanned image using the lightness distribution,
The image processing apparatus according to claim 6. The shadow correction means includes
Dividing the scanned image into a plurality of regions in a direction orthogonal to the one-dimensional image;
For each divided region, the shadow distribution near both ends of the binding portion of the scanned image is corrected using the brightness distribution .
The image processing apparatus according to claim 1. Based on the shape in which the outer edge of the scanned image has entered inside, identify the central portion of the scanned image in the direction orthogonal to the one-dimensional image, and divide into two regions from the central portion,
The background correction means performs background correction for each divided area,
The shadow correction means corrects shadows near both ends of the binding portion of the scan image for each divided area .
The image processing apparatus according to claim 1. An image reading means for reading a book document placed on the contact glass;
A first means identifies a flat portion of the book document from pixel values of a scanned image of the book document, and a direction orthogonal to a one-dimensional image in a direction connecting both ends of the binding portion with reference to the pixel value of the flat portion Normalizing the brightness of the scanned image with respect to a position y to generate a normalized brightness profile;
A background correction unit correcting the background near the binding portion of the scanned image using the normalized brightness profile;
The second means sets the position where the lightness is approximately the median value of the lightness distribution of the one-dimensional image to x0, asymptotically constant with the approximate median, with respect to the lightness distribution with respect to the position x in the direction parallel to the one-dimensional image. The brightness is defined as a, where b is the slope of the brightness with respect to the position x passing through the approximate median, and c is the approximate median of the brightness distribution.
A predetermined value is given to x0, and b is estimated by substituting the lightness at a certain x of the one-dimensional image into the equation to obtain the lightness distribution of the one-dimensional image;
A shadow correcting means for correcting shadows near both ends of the binding portion of the scanned image using the brightness distribution ;
An image processing method comprising: The second means performs a filter operation on the pixel value of the one-dimensional image, estimates the coordinates xl and xr at both ends of the scanned image, and sets a position outside the coordinate xl or outside the coordinate xr to the value of x0. A predetermined value ,
The image processing method according to claim 10. The shading correction means for obtaining brightness, saturation and hue from RGB values for each pixel;
Determining whether each pixel of the book document is a chromatic color or an achromatic color;
Performing background correction with the normalized brightness profile for both saturation and brightness in the case of chromatic color, and performing background correction with the normalized brightness profile only for brightness in the case of achromatic color; and
Obtaining RGB values from lightness, saturation and hue for each pixel;
The image processing method according to claim 10 or 11, further comprising: The shading correction means obtains brightness for each pixel;
Performing background correction using the normalized brightness profile for each of the R value, G value, and B value of the pixel value;
The image processing method according to claim 10 or 11, further comprising: The shadow correction means is
Obtaining the normalized brightness profile of each of the R value, G value, and B value of the pixel value;
Performing background correction on each of the R value, the G value, and the B value by the normalized brightness profile of each of the R value, the G value, and the B value ;
The image processing method according to claim 10 or 11, further comprising: The shadow correction means includes
Generating a brightness profile indicating the brightness distribution of the scanned image in association with a position y in a direction orthogonal to the one-dimensional image;
Identifying a predetermined region around a position y where the lightness profile shows a downward peak in the binding portion of the scanned image;
Correcting only the region of the scanned image using the lightness distribution, and shadows near both ends of the binding portion ;
The image processing method according to claim 10, further comprising: The shadow correcting means dividing the scanned image of the region into a plurality of regions parallel to a one-dimensional image;
The second means determines the brightness distribution of the one-dimensional image only for the one-dimensional image of the boundary of the divided region;
Complementing the brightness distribution in a direction orthogonal to the one-dimensional image, and correcting shadows near both ends of the binding portion of the scanned image using the brightness distribution ;
16. The image processing method according to claim 15, further comprising: The shadow correcting means divides the scanned image into a plurality of regions in a direction orthogonal to the one-dimensional image;
For each divided region, correcting shadows near both ends of the binding portion of the scanned image using the brightness distribution;
The image processing method according to claim 10, wherein the having. Identifying a central portion of the scan image in a direction orthogonal to the one-dimensional image based on a shape in which an outer edge of the scan image is embedded inside, and dividing the region into two regions from the central portion;
The background correction means performing background correction for each divided area;
The shading correction means for correcting shading near both ends of the binding portion of the scanned image for each divided region;
The image processing method according to claim 10, wherein the having. A program for causing a computer to execute the image processing method according to any one of claims 10 to 18. A computer-readable storage medium storing the program according to claim 19.