JPH06217090A - Method for correcting picture data and picture reader - Google Patents

Abstract

PURPOSE:To perform correcting processing against the influence of three- dimensional information included in picture data. CONSTITUTION:A picture correcting section 106 is provided with a reflection rate distribution arithmetic section 201 obtaining the reflection rate distribution of a document surface based on the maximum luminance value at every line of picture data, a three-dimensional shape estimation section 202 estimating the three-dimensional shape of the document with the use of the reflection rate distribution obtained by the section 201 and the intensity of illumination with an illumination light intensity table 111a stored, a distortion correcting section 203 correcting the geometrical distortion of picture data caused by the inclination of the document surface based on the 3-dimensional shape estimated by the section 202, and shade eliminating section 204 eliminating the shade of the picture data caused by the inclination of the document surface based on the three-dimensional shape estimated by the section 202.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of correcting image data obtained by irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light, and a method of placing the image data on the scanning surface. The present invention relates to an image reading device which irradiates a document on which light is read and photoelectrically converts the reflected light to read image data.

[0002]

2. Description of the Related Art In an image reading device such as an image scanner, a document placed on a contact glass (scanning surface) is irradiated with light, and the reflected light is photoelectrically converted to read image data. As the manuscript read by the image reading device,
There are two types, sheet originals and book originals. Since the sheet original is a single sheet of paper, the original surface is flat and can be placed in close contact with the contact glass surface. Therefore, the distance between the document surface and the light source is constant, and the light source irradiates the document surface with light having a predetermined illumination light intensity, and the image data can be read well. On the other hand, when reading a book document, as shown in FIG. 14, the document surface is lifted from the contact glass (scanning surface) and tilted at the binding portion of the book document, and the distance between the document surface and the light source is increased. Since the light intensity is not constant, there was a problem that the illumination light intensity changed and the original image near the binding could not be read. Specifically, the document surface near the binding of the book document is tilted with respect to the contact glass, and the read image data is distorted or the illumination light intensity of the light source is insufficient and the read image data is shaded. Get out.

Therefore, in order to read the original image near the binding in a better state, the book original is pressed against the contact glass to make the original surface as flat as possible for reading. There is a risk of damage, and even if the book document is pressed against the contact glass, the binding area cannot be completely flattened, so there is a shadow on that part, and the read image data is output with a plotter or the like. In some cases, it becomes black,
There was an inconvenience that the appearance was bad.

Therefore, it has been desired to develop a method and an image reading apparatus for satisfactorily reading a document image near the binding of a book document.

[0005]

However, according to the conventional image data correction method and image reading apparatus, the image data obtained by scanning the vicinity of the binding of the book original due to the inclination of the original surface causes a geometrical distortion or image distortion. However, even if it is known that a shadow will be generated, since the image data is read as two-dimensional luminance data, the correction process cannot be performed for the influence of the three-dimensional information included in the image data. was there.

The present invention has been made in view of the above, and it is an object of the present invention to make it possible to perform correction processing for the influence of three-dimensional information included in image data.

[0007]

In order to achieve the above object, the present invention corrects image data obtained by irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light. In the method, a method for correcting image data is provided, which estimates a three-dimensional shape of a document from image data and corrects geometrical distortion caused by the inclination of the document surface based on the estimated three-dimensional shape.

In order to achieve the above object, the present invention provides a method for correcting image data obtained by irradiating an original placed on a scanning surface with light and photoelectrically converting the reflected light.
The estimated 3D shape of the original from the image data
Provided is a method of correcting image data for removing a shadow caused by the inclination of a document surface based on a three-dimensional shape.

In order to achieve the above object, the present invention provides a method for correcting image data obtained by irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light.
The estimated 3D shape of the original from the image data
Provided is a method for correcting image data, which corrects geometrical distortion caused by inclination of a document surface and removes shadows based on a three-dimensional shape.

Further, in order to achieve the above object, the present invention provides an image reading apparatus for irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light to read image data. A reflectance distribution calculating means for obtaining the reflectance distribution of the document surface based on the maximum luminance value for each line of data, and the illumination light intensity previously stored as a function of the distance between the scanning surface and the document surface are stored. The storage means, the reflectance distribution obtained by the reflectance distribution calculating means, and the illumination light intensity of the storage means are used to estimate the three-dimensional shape of the original, and the three-dimensional shape estimating means estimates the three-dimensional shape of the original. An image reading apparatus provided with a distortion correction means for correcting geometrical distortion of image data caused by the inclination of a document surface based on a three-dimensional shape.

Further, in order to achieve the above object, the present invention provides an image reading apparatus for irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light to read image data. A reflectance distribution calculating means for obtaining the reflectance distribution of the document surface based on the maximum luminance value for each line of data, and the illumination light intensity previously stored as a function of the distance between the scanning surface and the document surface are stored. The storage means, the reflectance distribution obtained by the reflectance distribution calculating means, and the illumination light intensity of the storage means are used to estimate the three-dimensional shape of the original, and the three-dimensional shape estimating means estimates the three-dimensional shape of the original. (EN) An image reading device provided with a shadow removing means for removing a shadow of image data caused by an inclination of a document surface based on a three-dimensional shape.

Further, in order to achieve the above object, the present invention provides an image reading apparatus which irradiates a document placed on a scanning surface with light and photoelectrically converts the reflected light to read image data. A reflectance distribution calculating means for obtaining the reflectance distribution of the document surface based on the maximum luminance value for each line of data, and the illumination light intensity previously stored as a function of the distance between the scanning surface and the document surface are stored. The storage means, the reflectance distribution obtained by the reflectance distribution calculating means, and the illumination light intensity of the storage means are used to estimate the three-dimensional shape of the original, and the three-dimensional shape estimating means estimates the three-dimensional shape of the original. Distortion correcting means for correcting geometric distortion of image data caused by inclination of the original surface based on the three-dimensional shape, and inclination caused by the original surface based on the three-dimensional shape estimated by the three-dimensional shape estimating means There is provided an image reading apparatus that includes a shadow removal means for removing the shading of the image data.

The above-described three-dimensional shape estimating means calculates the distance between the document surface and the scanning surface based on the reflectance distribution obtained by the reflectance distribution calculating means, and uses the calculated distance to determine the scanning surface and the scanning surface. The intensity of the illumination light, which is a function of the distance to the document surface, is specified, and the angle between the normal to the document surface and the light source direction is calculated based on the following equation:

[0014]

[Equation 2] It is desirable to obtain the normal vector (three-dimensional shape) of the original surface from the angle formed by the normal line of the original surface and the light source direction.

Further, the above-mentioned distortion correction means uses the inclination of the document surface as θ and the length of one pixel of the read image data as α.
Then, it is desirable to correct the distortion based on the equation β = α × 1 / cos θ (where β is the corrected data).

Further, the above-described shadow removing means multiplies the ratio of the luminance value of each pixel of the line to the maximum luminance value of each line by the luminance value of the background portion when the inclination of the document surface is 0 degree. ， It is desirable to remove shadows.

Further, it is desirable that the above-mentioned shadow removing means sets the image data of pixels in which the inclination of the document surface is within a predetermined value or more to the background level.

Further, it is desirable that the distortion correction means and / or the shadow removal means described above correct the image data only when the original is a book original.

Further, it is desirable that the above-mentioned reflectance distribution calculating means fetches image data at the time of prescanning and obtains the reflectance distribution of the document surface.

[0020]

The image data correction method and the image reading apparatus of the present invention estimate the three-dimensional shape of a document from the image data, and based on the estimated three-dimensional shape, the geometric distortion caused by the inclination of the document surface is estimated. to correct. Further, the shadow generated by the inclination of the document surface is removed based on the estimated three-dimensional shape.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following is an example of applying the image data correction method and image reading apparatus of the present invention to an image scanner, and the configuration of the scanner: Outline of three-dimensional shape estimation method Definition Definition of cross-sectional shape of book document Definition of reflection characteristics Three-dimensional shape estimation algorithm in this embodiment Determination of distribution of illumination light intensity and determination of reflection characteristics Detailed description will be given in the order of operation of this embodiment with reference to the drawings. .

Configuration of Scanner FIG. 1 shows the configuration of the scanner of the present embodiment, which is a CCD (solid-state image sensor) for reading image data by inputting reflected light of light applied to a document and converting it into an electric signal. 10
1, and a CCD driving unit 102 that drives the CCD 101,
An image processing unit 103 that inputs image data read by the CCD 101 and executes various image processes, a scan buffer 104 that stores the image data that has undergone the image processing, and a buffer that controls input / output of the image data with respect to the scan buffer 104. A controller 105, an image correction unit 106 that corrects geometric distortion and shading of image data caused by the inclination of the document surface, and an I / F (interface) for outputting the image data to an external device such as a host computer. Ys) controller 107, CPU 108 for controlling the entire apparatus, ROM 109 storing a control program executed by CPU 108 and various data
A RAM 110 used as a work area, a fluorescent lamp 112 as a light source for exposing a document, and a fluorescent lamp 1.
12, a fluorescent lamp driver 111 that drives 12, a stepping motor 114 that moves a traveling body (not shown), and a motor driver 113 that drives the stepping motor 114.
And an operation unit 115 for performing various operations. The ROM 111 stores the illumination light intensity table 111a (see FIG. 2) which is obtained in advance as a function of the distance between the scanning surface and the document surface.

FIG. 2 shows the configuration of the image correction unit 106.
The image correction unit 106 calculates the reflectance distribution of the document surface based on the maximum luminance value of each line of the image data, the reflectance distribution calculation unit 201, the reflectance distribution calculated by the reflectance distribution calculation unit 201, and the illumination. A three-dimensional shape estimation unit 202 that estimates the three-dimensional shape of a document using the illumination light intensity of the light intensity table 111a, and an inclination of the document surface based on the three-dimensional shape estimated by the three-dimensional shape estimation unit 202 A distortion correction unit 203 for correcting geometric distortion of image data,
Based on the three-dimensional shape estimated by the three-dimensional shape estimation unit 202, the shadow removal unit 204 removes the shadow of the image data caused by the inclination of the document surface.

FIGS. 3A and 3B show the structure of the optical system of the scanner. In the figure, 301 and 302 are fluorescent lamps 1.
The mirror and the lens for guiding the reflected light of the light emitted from 12 to the original to the CCD 101 are shown. Reference numeral 303 denotes a flat scanning surface (scanner surface), which is specifically a surface that comes into close contact with the original surface of the contact glass (not shown).

In this embodiment, an orthogonal coordinate system is defined as shown in the figure. The y axis is parallel to the moving direction of the CCD 101, and x
The axis is parallel to the CCD 101, and the z axis is the scanning plane 30.
It is arranged so as to be orthogonal to 3. Further, both the x and y axes are included in the scanning plane 303.

The mirror 301 is used to guide the reflected light from the document to the CCD 101 as described above. x-
The light rays on the y-plane and the vertical plane V are called scanning lines S. Fluorescent light 1
12 and the mirror 301, the lens 302 and the CCD 10
While keeping a constant distance from 1, the original is scanned while moving under the xy plane.

The CCD 101 captures a one-dimensional image sequence I (x) on the scanning line S. A two-dimensional image I (x, y) is obtained by arranging a series of one-dimensional images I (x) taken while moving the CCD 101. The one-dimensional image is an image projected by perspective transformation, but CCD
The projection along the moving direction of 101 is equivalent to the parallel projection.

Outline of Three-Dimensional Shape Estimation Method Next, an outline of the three-dimensional shape estimation method will be described prior to the description of the specific operation of this embodiment. The problem of restoring the original object shape from a two-dimensional image of a three-dimensional object is one of the basic problems in computer vision. In particular, the problem of restoring the three-dimensional shape of an object from the density information of an image is called Shape from Shading, which has been a subject of research for a long time in computer vision.

If the surface of the object is a perfect diffusion surface, Shape
from Shading is formulated as follows.

[Equation 3]

The problem of restoring the three-dimensional shape of an object based on the equation (1) is decomposed into the following two subproblems. Problem 1 Find φ from observed I _V Problem 2 Find normal vector of object surface from φ

To solve problem 1, the values of I _S and ρ must be given, but these parameters are generally unknown. Therefore, in many cases, φ is calculated assuming that the intensity of illumination light and the reflectance of the object surface are constant.

The problem 2 is also a problem that cannot be solved unless some kind of constraint or assumption is given. For this reason, conventionally, as a method of compensating for the lack of the condition number in the problem 2, the "illuminance difference stainless steel method" (R which restores the shape of an object from a plurality of images taken under illumination light from different directions J. Woodham, "Photomet
ric method for detail
g surface orientation from
multiple image ”, Optical
Engineering, 19, 1, 1977), and a method of introducing assumptions such as smoothness and constant cross-sectional shape with respect to an object shape.

As can be seen from the properties of the problems 1 and 2 described above, the general Shape from Shading problem has a shortage of condition numbers, and if a new assumption is not introduced to compensate for it, the problem can be solved. I can't. From this point of view, most of the conventional research on Shape from Shading can be grasped within the framework of specializing general problems by introducing assumptions and finding solutions.

The problem here is that the specialized and solvable problem contains an assumption that is not applicable in a practical situation, and solving the problem itself does not have any practical significance. There are often cases. Therefore, when the surface of a book document is photographed by a scanner and the three-dimensional shape of the Gook document is restored (estimated) from the obtained image as in the present embodiment, there is no method that can be applied as it is.

The scanner of this embodiment forms a two-dimensional image from a one-dimensional image sequence taken by the moving one-dimensional CCD 101 as described above. The fluorescent lamp 112 built in the scanner moves while keeping its relative position with respect to the CCD 101, and illuminates the surface of the book document.
It is assumed that the book document is arranged such that the center line of the bound portion is parallel to the CCD 101. Assumptions applicable to the problem handled in this embodiment are as follows.

Assumption 1: The cross-sectional shape of a book document orthogonal to binding is constant. Assumption 2: The surface of the book original is a paper surface printed with ink having a lower reflectance than a bright paper having a constant reflectance.

The above assumption 1 can be easily established by taking the placement direction into consideration when placing the book document on the scanner. Assumption 2 can be easily applied to the surface of a general book document.

From Assumption 1, if each φ formed by the light source direction and the normal line of the book document surface is found, it is guaranteed that the normal line of the book document surface is uniquely obtained. Further, from this assumption, it can be said that each one-dimensional image is an image of a portion with a constant inclination on the surface of the book document.

From Assumption 2, the intensity of the reflected light from the background paper having a constant reflectance ρω can be easily obtained by obtaining the pixel having the maximum brightness in each one-dimensional image. By using this intensity distribution, it is possible to treat the surface of a book original in the same manner as an object having a constant reflectance.

These assumptions 1 and 2 facilitate the problem of obtaining the three-dimensional shape of the book manuscript surface, but φ cannot be determined by this alone, and as is clear from the equation (1), the book manuscript surface is irradiated. that it is necessary to provide the illumination light intensity I _S.

The light emitted from the fluorescent lamp 112 is the scanning surface 3
Since the object which is in contact with 03 is illuminated most strongly and the other parts are narrowed down so as not to be sufficiently illuminated, the illumination is applied to the book document surface as the book document surface moves away from the scanning surface 303. Light intensity becomes weak. Therefore, the illumination light intensity is expressed as a function I _S (d) of the distance d between the scanning surface 303 and the book document surface. Where 3
As the third assumption, Assumption 3 is introduced. This assumption is I _S
This means that (d) is measured in advance.

Assumption 3: The illumination light intensity I _S (d) is given in advance as a characteristic peculiar to the scanner.

If the distance d is obtained from the above three assumptions, the value of I _S (d) is determined, and φ can be obtained from the equation (1). On the contrary, the distance d is obtained by integrating the tangent of the angle of the normal vector obtained from φ. That is, in order to obtain the inclination of a certain point on the surface of the book original, it is necessary to obtain all the inclination series of the surface of the book original from the portion where the surface of the book original contacts the scanning surface 303 to that point.

A large-scale optimization calculation is required to obtain the inclination of each point on the surface of the book manuscript at one time, and when the inclination of the surface is to be obtained sequentially, an appropriate method is used. If you do not perform, the correct solution may not be obtained due to the accumulation of errors. This problem is a technical problem to be overcome in order to achieve the object of the present invention. Therefore, in the present embodiment, the following two points are further taken into consideration, and both the perfect diffusion component and the regular reflection component on the object surface shown in the formulas (2) and (3) instead of the formula (1) are expressed. Phong that can be expressed at the same time
The reflection model of is adopted.

[0045]

[Equation 4]

By using this model, the reflection characteristic of the surface of the book original can be expressed with a small number of parameters.

Definition of Scanner Light Source Direction and Illumination Light Intensity In the fluorescent lamp 112 which is the light source of this embodiment, as shown in FIG. 4, when the intersection of the scanning line and the yz plane is S ₀ , y−V
It has an elongated cylindrical shape with a radius r having a center at the position of (-d ₁ , -d ₂ ) from S ₀ in the plane. Around the fluorescent lamp 112, there is a film 112aga that blocks light,
Light is emitted from the part without the film. That is, the direction of the illumination light is limited so that the strongest illumination light is applied in the direction connecting the center of the fluorescent lamp 112 and S ₀ . The part irradiated with light (called a slit) is in the range of angles ω ₁ to ω ₂ (ω ₁ <ω ₂ ).

Since the illumination light is set to be strongly emitted in a specific direction, if there is a gap between the scanning surface and the surface of the book original, the intensity of the illumination light on the surface of the book original scans from the scanning surface. It changes according to the distance d to the front surface of the book document.

Further, as is apparent from FIG. 4, the direction ψ of the light source when viewed from the surface of the book original also changes depending on the distance d. In the present embodiment, "the direction of the bisector of the angle connecting the both ends of the slit from the point on the surface of the book document" is defined as the light source direction. At this time, the light source direction ψ (d), which changes according to the height d, is expressed by Expression (4).

[0050]

[Equation 5]

Next, the definition of the illumination light intensity will be described. Generally, the intensity of illumination light from an infinitely long line light source is inversely proportional to the distance to the surface of the book document. Therefore, assuming that the line light source is arranged on the slit of the fluorescent lamp 112 shown in FIG. 4, the illumination light intensity on the surface of the book document can be calculated. However, in consideration of the fact that the length of the light source built into the scanner is finite and there is also the ambient light component from the background, in the present embodiment, the illumination light intensity I _S on the surface of the book document is actually taken into consideration. (D) is represented by the formula (5). However, α, β, γ, Δ ₁ , and Δ ₂ are parameters that determine the characteristics of the illumination light.

[0052]

[Equation 6]

Definition of Cross Section of Book Document As shown in FIG. 5, it is assumed that the book document on the scanning surface is arranged so that the center of binding is directly above the x axis. When the book document is arranged in this way, the positional relationship between the scanning plane and the surface of the book document becomes constant in the x-axis direction from the above-mentioned assumption 1. Therefore, the three-dimensional shape restoration problem of the book document surface is yz It can be treated as a two-dimensional shape estimation problem on a plane.

When viewed along the y-axis, the surface of the book document comes into contact with the scanning surface at a point of y = y ₀ . Therefore, the angle between the z-axis and the normal to the surface of the book original at point y is θ (y),
Assuming that the distance between the scanning surface and the surface of the book original is d (y), the relation of equation (6) is maintained when y ≧ y ₀ .

[0055]

[Equation 7] The distance d (y) in 0 ≦ y <y ₀ is expressed by the following equation (7).

[0056]

[Equation 8]

In FIG. 5, reference numerals (302) and (101) indicate the positions on the assumption that the lens 302 and the CCD 101 are on the same straight line as the vertical plane V.

Definition of Reflection Characteristics As described above, in the present embodiment, as the reflection characteristics of the surface of the book original, the perfect diffuse reflection component and the regular reflection component are simultaneously expressed as shown in the following equations (2) and (3). The model of Phong that can be expressed was adopted.

[0059]

[Equation 9]

Here, σ represents the ratio of the perfect diffuse reflection component and the regular reflection component, 0 ≦ σ ≦ 1, and δ is the angle formed by the direction of the regular reflection light and the direction of the line of sight. Further, n indicates the "sharpness" of the specular reflection component, and is known to take a value from 1 to 200.

In the case where the book document of the present embodiment shown in FIG. 5 is installed, φ = φ (d) −θ and δ = 2θ−φ (d). The reflection characteristic is rewritten as the following Expression (8).

[0062]

[Equation 10]

Three-Dimensional Shape Estimation Algorithm in this Embodiment From the equation (2) and assumption 2, the optical model in this embodiment is expressed by the following equation (9).

[Equation 11]

Here, Iω (y) and ρω respectively represent the intensity and the reflectance of the observation light of the ground portion having a large reflectance on the surface of the book document, and are calculated by the following equation (10).

[Equation 12]

When the expression (9) is discretely expressed, it is expressed by the expression (11) from the expressions (5) and (8).

[Equation 13]

Here, y ₀ > y ₁ >...> y _i . However, d (y _i ) is obtained by discretizing the equation (7) to obtain the equation (12).
It is represented by.

[Equation 14]

Based on equations (11) and (12), θ (y
There are two possible methods for obtaining _i ).

Method 1 ... By minimizing the squared error function G (θ (y _i )) between the observed light intensity calculated from the optical model of equations (11) and (12) and the measured value, θ
Calculate (y _i ). It is shown in Expression (13).

[Equation 15]

Method 2 Further, as a method for performing accurate estimation, the following evaluation function H is minimized θ (y _i ).
Compute the entire series of at once. It is shown in equation (14).

[Equation 16]

In method 1, each θ (y_i) Is calculated
Method. In this method, each y _iCalculation result in
Is y until then₀, Y₁, …… y_i-1Depends on the result in
Is included in the obtained solution as the calculation progresses
The error may increase.

On the other hand, in method 2, a multidimensional nonlinear minimization algorithm is used. In this case, each θ
In order to obtain (y _i ), it is necessary to set an optimum initial value in advance. Further, if the method 2 is applied to the entire range of 0 ≦ y _i ≦ y _max , there is a problem that the calculation cost becomes huge.

Therefore, in this embodiment, each θ (y _i ) is obtained by the following procedures (a) to (c) in consideration of the characteristics of the above two types of methods. (A) Formula (10) from the image of the surface of the book manuscript
Therefore, the observed light intensity Iω (y
_i ) is asked. (A) By the method 1, the condition shown in the equation (6) is changed to i =
It is used as the result when 0 and θ (y _i ) is sequentially obtained. (C) Of θ (y _i ) obtained by the method 1, G
The method (2) is used to re-determine θ (y _i ) only for the portion where the error in (θ (y _i )) is large. At this time,
The result of Method 1 is used as the initial value.

Determination of Distribution of Illumination Light Intensity and Determination of Reflection Specification Before estimating the cross-sectional shape of the surface of a book document, first, 12 parameters in total in equations (11) and (4) are calculated. I have to ask for it. Among these, the design values of the scanner are used for the parameters (d ₁ , d ₂ , τ) in the equation (4). For the rest of the parameters (α, β, γ, Δ ₁ , Δ ₂ , σ, n, ω ₁ , ω ₂ ), the image obtained by scanning a plain paper slope inclined by a known angle with a scanner The parameter was estimated by obtaining Iω from the above and applying the nonlinear least squares method. As a result, the estimated values of each parameter are as follows.

Α = 618.1, β = 772.9, γ
= -216.9 Δ ₁ = -2.463, Δ ₂ = -1.966 σ = 0.771, n = 1.0 ω ₁ = 0.0 [degree] ω ₂ = 63.0 [degree] ( Below, design values) d ₁ = 11 [mm], d ₂ = 16 [mm], τ
= 7 [mm]

As a result, the illumination light intensity I _S (d) on the surface of the book original which is attenuated by the distance d is as shown in FIG. 7 shows the estimated reflection characteristics on the surface of the book document. This figure shows the case where θ = 30 degrees. Reference numeral 701 in the figure denotes the book document surface.

FIG. 8 shows a graph of f (θ, d) with respect to θ, d expressed by the equation (8). From this graph,
As the value of θ gradually increases from 0, f
It can be seen that the value of (θ, d) tends to slightly increase and then decrease again. This means that the perfect diffusion component is θ = ψ
In the case of (d), the specular reflection component takes the maximum value when θ = ψ (d) / 2, so the maximum value of f (θ, d) is ψ (d) / 2.
It is also clear from the existence of <θ <ψ (d). Thus, the tendency that f (θ, d) once increases is particularly strong when the value of d is near zero. Also, if d is constant, then f
The cross-sectional shape of (θ, d) is a cosin curve similar to that of equation (1) when there is no specular reflection component on the surface of the book original, but in the case of FIG. It can be seen that the shape is different from the cosin curve.

Operation of this Embodiment The operation of the above configuration will be described. A book document is placed on the contact glass, and reading of the book document (hereinafter referred to as a book mode) is designated via the operation unit 115. At this time, the book document is placed so that its cross-sectional shape is constant as shown in FIG.

When a predetermined start key on the operation unit 115 is pressed, the CPU 108 controls the CCD driving unit 102, the fluorescent lamp driver 111, and the motor driver 113 to start reading the image data. The image data read by the CCD 101 is subjected to image processing such as amplification processing, A / D conversion processing, shading correction processing, and binarization processing in the image processing unit 103, and then the scan buffer 10
4 is stored.

The reflectance distribution calculation unit 20 of the image correction unit 106
1 reads the image data from the scan buffer 104, and based on the maximum luminance value of each line of the image data,
Obtain the reflectance distribution on the document surface. At this time, the reflectance of the white background of the flat surface of the book original is used as a reference value, and the reflectance of the maximum luminance value of each line is processed as being the same (constant) as the reference value of the white value. Determine the distribution.

The three-dimensional shape estimation unit 202 reads the image data from the scan buffer 104, and uses the reflectance distribution obtained by the reflectance distribution calculation unit 201 and the illumination light intensity of the illumination light intensity table 111a to obtain the three-dimensional shape of the original. Estimate the shape. In other words, the cross-sectional shape of the book manuscript is estimated. At this time, the three-dimensional shape estimation unit 202
The cross-sectional shape of the document is obtained by the method shown in the shape estimation algorithm. 12 existing in formula (4) and formula (11)
The above-mentioned estimated value and set value are used as the values of the individual parameters.

The distortion correction unit 203 includes a three-dimensional shape estimation unit 2
On the basis of the three-dimensional shape (inclination θ of the document) estimated in 02, the geometric distortion of the image data caused by the inclination of the document surface is corrected. Specifically, as shown in FIG. 9, the inclination of the document estimated by the three-dimensional shape estimating unit 202, the length of one pixel of the read image data is a, and A = a × 1 / cos θ (however, A: Corrected data) The distortion in the y-axis direction is corrected based on the following equation. That is, in the y-axis direction of the geometric distortion correction image generated by the tilt of the book document surface 1201 in FIG. 9, the tilted surface is projected in parallel, so the pattern on the surface is reduced and photographed. . The reduction ratio is cos θ.
Therefore, the distortion can be corrected by extending each pixel by 1 / cos θ pixel in the y-axis direction.

Further, the distortion correction unit 203 corrects the geometric distortion caused by perspective projection by the following method. Three-dimensional coordinates (x ₀ , y _i , d (y _i )) on the surface of the book document
The point of (x _d (x
₀ , y _i ), y _i ). Where the function x
_d is defined as follows.

[0083]

[Equation 17]

Here, F represents the distance from the focal point of the lens to the scanning surface (see FIG. 5). The correction may be performed by moving the projection point in perspective projection to the projection point in parallel projection. Therefore, the correction is performed by using the inverse function of the function x _d .

The shading removal unit 204 includes a three-dimensional shape estimation unit 2
Based on the three-dimensional shape (inclination θ of the document) estimated in 02, the shadow of the image data caused by the inclination of the document surface is removed. The inclination of the document surface also has an effect such as attenuation of illumination light. In this embodiment, as shown in the following equation, each y _i
The shadow is removed by multiplying the ratio of the illumination light intensity of each pixel with respect to the illumination light intensity of the portion having a large reflectance value in (3) by the illumination light intensity of the ground portion when θ = 0 (d = 0). The bar I (x, y _i ) becomes the corrected value.

[Equation 18]

The image data that has undergone geometric distortion correction and shadow removal is output to an external device via the I / F controller 107.

FIG. 10A shows an example in which a book original is read by a conventional scanner and output by a plotter.
FIG. 10B shows an example in which a book document is read by the scanner of this embodiment and is similarly output by the plotter. As is clear from the figure, according to the present embodiment, it is possible to correct the distortion of the image data due to the inclination of the document and remove the shadow.

In this embodiment, the scan buffer 1
Although the image data is stored in 04, but in the case of the structure having no memory, the image data is fetched by the pre-scan into the reflectance distribution calculation unit 201, and the reflectance distribution of the document surface preceding the other processing units is obtained. You can

Further, the image correction unit 1 only in the book mode.
It is preferable to execute the operation No. 06, and to send the image data directly from the scan buffer 104 to the I / F controller 107 in the case of a normal sheet original.

[0090]

As described above, the image data correction method and the image reading apparatus of the present invention estimate the three-dimensional shape of a document from the image data, and based on the estimated three-dimensional shape, the inclination of the document surface is used. In order to correct the geometric distortion that occurs and / or to remove the shadow, it is possible to perform a correction process for the influence of the three-dimensional information included in the image data.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing a configuration of a scanner according to an embodiment.

FIG. 2 is an explanatory diagram showing a configuration of an image correction unit.

FIG. 3 is an explanatory diagram showing a configuration of an optical system of the scanner of this embodiment.

FIG. 4 is an explanatory diagram showing definitions of a light source direction and illumination light intensity of a scanner.

FIG. 5 is an explanatory diagram showing a cross-sectional shape of a book document.

FIG. 6 is an explanatory diagram showing an illumination light intensity I _S (d) on the surface of a book document which is attenuated by the distance d.

FIG. 7 is an explanatory diagram showing a reflection characteristic of a book document surface.

FIG. 8 is a graph showing reflection characteristics with respect to θ and d.

FIG. 9 is an explanatory diagram showing correction of distortion.

10A is an example in which conventional image data is output by a plotter, and FIG. 10B is an example in which image data corrected in the present embodiment is output by a plotter.

FIG. 11 is an explanatory diagram showing a state in which the document surface of the binding portion of the book document is lifted from the contact glass.

[Explanation of symbols]

101 CCD 106 Image Correction Unit 108 CPU 111 RO
M 111a Illumination light intensity table 112 Fluorescent lamp (light source) 201 Reflectance distribution calculation unit 202 Three-dimensional shape estimation unit 203 Distortion correction unit 204 Shadow removal unit

Claims (12)

[Claims] 1. A method for correcting image data obtained by irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light, to estimate a three-dimensional shape of the document from the image data, A method for correcting image data, which comprises correcting geometric distortion caused by an inclination of a document surface based on the estimated three-dimensional shape. 2. A method for correcting image data obtained by irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light, to estimate a three-dimensional shape of the document from the image data, A method for correcting image data, characterized in that a shadow caused by an inclination of a document surface is removed based on an estimated three-dimensional shape. 3. A method for correcting image data obtained by irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light, to estimate a three-dimensional shape of the document from the image data, A method for correcting image data, which comprises correcting geometric distortion caused by an inclination of a document surface and removing shadows based on the estimated three-dimensional shape. 4. An image reading apparatus for irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light to read image data, based on a maximum luminance value of each line of the image data. Then, the reflectance distribution calculating means for obtaining the reflectance distribution of the document surface, the storage means for storing the illumination light intensity previously obtained as a function of the distance between the scanning surface and the document surface, and the reflectance distribution calculating means. 3D shape estimation means for estimating the 3D shape of the original using the reflectance distribution obtained in step 3 and the illumination light intensity of the storage means, and the original surface based on the 3D shape estimated by the 3D shape estimation means An image reading apparatus comprising: a distortion correction unit that corrects a geometric distortion of image data caused by the inclination of the image. 5. An image reading apparatus for irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light to read image data, based on a maximum brightness value of each line of the image data. Then, the reflectance distribution calculating means for obtaining the reflectance distribution of the document surface, the storage means for storing the illumination light intensity previously obtained as a function of the distance between the scanning surface and the document surface, and the reflectance distribution calculating means. 3D shape estimation means for estimating the 3D shape of the original using the reflectance distribution obtained in step 3 and the illumination light intensity of the storage means, and the original surface based on the 3D shape estimated by the 3D shape estimation means An image reading apparatus comprising: a shadow removing unit that removes a shadow of image data caused by the inclination of the image. 6. An image reading device for irradiating a document placed on a scanning surface with light and photoelectrically converting the reflected light to read image data, based on a maximum luminance value of each line of the image data. Then, the reflectance distribution calculating means for obtaining the reflectance distribution of the document surface, the storage means for storing the illumination light intensity previously obtained as a function of the distance between the scanning surface and the document surface, and the reflectance distribution calculating means. 3D shape estimation means for estimating the 3D shape of the original using the reflectance distribution obtained in step 3 and the illumination light intensity of the storage means, and the original surface based on the 3D shape estimated by the 3D shape estimation means Distortion correction means for correcting the geometric distortion of the image data caused by the inclination of the image data, and the three-dimensional shape estimated by the three-dimensional shape estimation means, the shadow of the image data caused by the inclination of the document surface is removed. An image reading apparatus comprising: a shading removing unit. 7. The three-dimensional shape estimating means calculates the distance between the document surface and the scanning surface based on the reflectance distribution obtained by the reflectance distribution calculating means, and uses the calculated distance as the scanning surface. The intensity of the illumination light, which is a function of the distance from the document surface, is specified, and the angle between the normal to the document surface and the light source direction is calculated based on the following equation. 7. The image reading apparatus according to claim 4, wherein the normal vector (three-dimensional shape) of the document surface is obtained from the angle between the document surface normal and the light source direction. 8. The distortion correcting means is β = α × 1 / cos θ (where β is the corrected data), where θ is the inclination of the document surface and α is the length of one pixel of the read image data. The image reading apparatus according to claim 4, 6 or 7, wherein the distortion is corrected based on the equation. 9. The shadow removing means multiplies the ratio of the brightness value of each pixel of the line to the maximum brightness value of each line by the brightness value of the background portion when the original surface has an inclination of 0 degrees, 6. The method for removing shadows according to claim 5,
The image reading device described in 6 or 7. 10. The shading removing means sets the image data of pixels in which the inclination of the document surface is within a predetermined value or more to a background level.
The image reading device described. 11. The distortion correcting unit and / or the shadow removing unit corrects the image data only when the original is a book original.
The image reading device according to 6, 7, or 10. 12. The image reading apparatus according to claim 4, wherein the reflectance distribution calculating means fetches image data during pre-scanning and obtains a reflectance distribution on a document surface.