KR100895622B1 - Image forming apparatus and image forming method - Google Patents

Abstract

The present invention relates to an image forming apparatus and an image forming method. An image forming method according to the present invention comprises the steps of: extracting a difference in roughness from a captured image to estimate the shape of the document; extracting a text region using the captured image and calculating a rectangular corner point of the text region; Adjusting the vertical spacing of the characters of the text area by using the attribute information of the characters and the estimated document shape, multiplying the coordinates of the rectangular corner points by the inverse matrix of converting the position of each point into a rectangular shape, Adaptively positioning each pixel of the lens. Thereby, distortion of the picked-up image which arises when scanning a book can be corrected so that a user may see it easily.

Description

Image forming apparatus and image forming method {IMAGE FORMING APPARATUS AND IMAGE FORMING METHOD}

1 is a diagram illustrating a state of scanning a book in an image forming apparatus;

2 is a diagram illustrating a configuration of a scanner unit of an image forming apparatus according to an embodiment of the present invention.

3 is a diagram illustrating an upper portion of an image forming apparatus according to an embodiment of the present invention.

4 is a diagram illustrating a configuration of an image forming apparatus according to an embodiment of the present invention.

5 is a view showing a captured image for explaining the present invention;

6 is a diagram illustrating an image processing unit of an image forming apparatus according to an embodiment of the present invention.

7 is a flowchart illustrating an image forming method according to an embodiment of the present invention.

FIG. 8 is a diagram illustrating a case where an embodiment of the present invention is applied and corrected in the image shown in FIG. 5.

The present invention relates to an image forming apparatus and an image forming method, and more particularly, to an image forming apparatus and an image forming method for correcting distortion of a captured image generated when a book is scanned.

The image forming apparatus generally refers to a scanner, a printer, a copier, a facsimile and the like. In particular, in recent years, in order to improve office efficiency and to realize office automation, a multi-function peripheral, which has a complex implementation of various functions such as a copier / printer or a copier / facsimile / printer, has been developed. It is widely used as one.

In general, an image forming apparatus having a scanner module performs scanning when a user places one sheet document on the glass 102 of the scanner module shown in FIG. 1 and presses a scanning button. However, not only sheet documents but also book documents such as books are handled as scanning documents of the image forming apparatus. In such a case, the book documents are placed on the glass 102 of the scanner module to scan the documents.

However, when scanning a book manuscript, as shown in FIG. 1, an area where the pages of the book 100 are stitched (hereinafter referred to as a “fold”) 101 is held on the glass 102 of the scanner module by the focal plane. Since it is separated from, the image quality deterioration such as image distortion occurs due to the scanned image of the excited region. Therefore, when the deteriorated image of the folded portion 101 is printed as it is, it is not only difficult for the user to read, but also the recognition rate of the character recognizer (OCR) is remarkably decreased.

Various techniques for correcting the distorted scanning image by the folded portion 101 of the book 100 are disclosed in the patent literature and the like. For example, Japanese Laid-Open Patent Publication No. Hei 5-161002 discloses a method of measuring the shape of a book by a triangulation method and correcting distortion, and Japanese Laid-Open Patent Publication Hei 11-41455 describes the shape of the page appearance of a scanning image. A method of estimating a three-dimensional shape of a book surface using is disclosed.

However, according to the method described in Japanese Patent Laid-Open No. 5-161002, a special shape measuring device for measuring the shape of a book by a triangulation method is required, and according to the method described in Japanese Patent Laid-Open No. 11-41455, The distortion can be corrected by the amount of calculation, but there is a case where an effective correction cannot be made depending on the conditions.

Accordingly, an object of the present invention is to provide an image forming apparatus and an image forming method for effectively correcting distortion of a scanning image generated when a book is scanned.

It is also an object of the present invention to provide an image forming apparatus and an image forming method for correcting distortion of a captured image to provide a book image that is easy for a user to read.

In order to achieve the above object, the present invention provides an image forming method for correcting a captured image for a document, comprising: estimating the shape of the document by extracting a difference in roughness from the captured image, and using the captured image. Extracting a text area and calculating a rectangular corner point of the text area, and adjusting vertical intervals of characters in the text area by using attribute information of characters in the text area and the estimated shape of the document. And multiplying the coordinates of the rectangular corner points by an inverse matrix that converts the position of each point into a rectangular shape, and adaptively positioning each pixel of the text area.

Extraction of the text area is preferably performed by performing wavelet transform on the captured image.

The extraction of the text area is preferably performed by performing wavelet transform on the image of which the illuminance change of the captured image is corrected.

It is preferable to calculate the square corner point of the text area by binarizing the corrected image and using the binarized image.

The method may further include determining a degree of warp in the horizontal line direction using text line information of the binarized image of the text area.

It is preferable that the shape estimation of the document is estimated by a shape from shading algorithm that restores the three-dimensional shape from the two-dimensional image.

Attribute information of the characters of the text area includes spacing of characters and thickness of characters.

Preferably, the adaptive position of each pixel of the text area is multiplied by the inverse of all coordinates of the picked-up image or the binarized image.

An object of the present invention as described above is an image forming apparatus for correcting a picked-up image for a document, comprising: a shape estimating unit for extracting a difference in roughness from the picked-up image and estimating the shape of the document; An image conversion unit for calculating and correcting the captured image, and for binarizing the corrected image, an area extraction unit for extracting a text area and a rectangular corner point of the text area using the corrected image and the binarized image; And a vertical spacing adjusting unit for adjusting the vertical spacing of the characters of the text area by using the attribute information of the characters of the text area and the estimated shape of the document, and the position of each point in the rectangular corner point coordinates in the form of a rectangle. Multiplying inverse matrix to transform, and adaptively positioning each pixel of the text area It is achieved by the correction section.

Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

2 is a view showing the configuration of a scanner module of the image forming apparatus according to an embodiment of the present invention, Figure 3 is a view showing an upper portion of the image forming apparatus according to an embodiment of the present invention, Figure 4 FIG. 7 is a diagram illustrating a configuration of an image forming apparatus according to an exemplary embodiment.

As shown in FIG. 2, the scanner module 1 includes a glass 2 on which an original is placed, a first traveling body 5 including an exposure lamp 3 and a first reflective mirror 4 for exposing an original. ), A second traveling body 8 composed of the second reflection mirror 6 and the third reflection mirror 7, a CCD (Charge Coupled Device) 9 as an image pickup device for reading an image of an original, and A lens unit 10 for forming an image on the CCD 9, an original reference table 11 serving as a reference for placing an original, and a white reference plate 12 for shading correction provided below the original reference table 11. And a frame 14, the CCD 9 being located on the sensor port 13.

As shown in FIG. 3, the scanner module 1 is provided with the press plate 17 which can open and close with respect to the glass 2, and the detection sensor 18 which detects the opening and closing of this press plate 17. As shown in FIG. As the printer module included in the copying machine 16, in addition to the electrophotographic method, respective printing methods such as an inkjet method, a thermal transfer method and a direct thermal recording method can be applied.

As shown in FIG. 4, the control system of the scanner module 1 performs various image processing on the main control unit 19 for controlling the entirety of the scanner module 1, and the captured image read by the CCD 9. An operation panel that receives various operations of the image processing unit 20, the servo control unit 21 for controlling the first traveling body 5 and the second traveling body 8, and the copier 16 and displays various messages. 21 and a memory 23 for storing a captured image read out by the CCD 9, predetermined data, or the like are connected. In addition, the operation panel 22 is provided with a copy start key and the like for performing copying. The servo control unit 21 further includes a stepping motor 24 for driving the exposure lamp 3, the first traveling body 5, and the second traveling body 8, the first traveling body 5, and the second traveling body. The home position sensor (HP sensor) 25 and the opening / closing sensor 18 which detect whether the sieve 8 is in the home position are connected.

In the image forming apparatus having the above-described configuration, a user places a book as shown in FIG. 1 for copying a book on the glass 2 of the scanner module 1, and a copy located on the operation panel 22. When the start key is pressed, the first traveling body 5 and the second traveling body 8 are moved by the stepping motor 24, and the exposure lamp 3 exposes the original for scanning. The reflected light is reflected by the first reflecting mirror 4, the second reflecting mirror 6, and the third reflecting mirror 7 to form an image on the CCD 9 through the lens unit 10. An image which is read by the CCD 9 and output to the image processing unit 20 while scanning a book original is called a scanning image. On the other hand, the picked-up image described in the claims of the present invention includes a case in which the image is distorted in the case of photographing a document in an image pickup device including a general digital camera as well as such a scanning image.

5 is a diagram showing a scanning image output from the above-described CCD 9 for explaining the present invention. When the distorted scanning image illustrated in FIG. 5 is printed through a printer module or displayed on a monitor to read, it is difficult to read due to the distortion of the image, and the recognition rate is also used when the document image is converted to text using an OCR. Deteriorates a lot.

6 is a view showing an image processing unit of the image forming apparatus according to an embodiment of the present invention, Figure 8 is a view showing an image output by the present invention. The image processor 20 of the image forming apparatus 16 includes an illuminance detector 61, a shape estimator 62, an image converter 63, an area extractor 64, a horizontal line aligner 65, and a vertical line aligner. 66, a vertical space | interval adjustment part 67, and the image restoration part 68 are included.

The illuminance detection unit 61 detects the illuminance change of the image input from the CCD 9. An image acquired in gray or color has different illuminance for each position depending on its shape and surrounding illumination position. That is, the difference in illuminance is caused by the difference in distance from the camera lens. The illuminance near the lens is high and the far end is low. In particular, the difference in illuminance is great when the flash is fired. In addition, other factors affecting the illuminance include vignetting caused by the lens itself in which illuminance of edges and periphery is lower than that of the center of the image in the process of acquiring an image. The illuminance detector 61 detects the change in the illuminance of the document image in consideration of various factors affecting the illuminance of the image.

The shape estimating unit 62 estimates the shape of the book document by using the illuminance change information output from the illuminance detecting unit 61. The Shape From Shading (SFS) algorithm is used to restore the shape of the three-dimensional object from the two-dimensional image in order to extract the illuminance difference due to the shape of the document. The change in illuminance determines the approximate three-dimensional shape of the document and allows setting of document vertices and / or the distance of the document from the lens. That is, the vertical component and the tangential plane are restored at each point of the object, and the depth is calculated through this. The document shape estimated using this SFS is supplied to the image conversion unit 63 and the vertical gap adjusting unit 67.

The image converting unit 63 corrects the scanned image using the illuminance change information output from the illuminance detecting unit 61, and binarizes the corrected scanning image. That is, as mentioned in the illuminance detection section 61, there are various elements causing the difference in illuminance for each region in the scanned image. Taking these into account, a mathematical model of the change in illuminance can be created. In other words, it is usually modeled as a different illuminance by multiplying a flashlight or an external light source with an object having a constant brightness. Therefore, illuminance is corrected by dividing certain values by pixel values using this model.

In this case, elements that can be corrected by inverse transform are first smoothed by inverse transform. Here, the inverse transformation means to change the acquired image due to the illumination difference generated during the scanning of the book to be close to the illumination of the original image, and the equalization means that when there are light and dark parts due to the illumination difference in the whole image, It's a bit darker, and the darker part is a little brighter, which means you can see areas that are hard to see in the darker part. In other words, even in the scanned image, there is less reflection and a dark area is generated. If the dark area is forcibly pulled up by digital processing, the contrast between the text and the background becomes clearer.

In addition, the remaining elements that are not smoothed by the inverse transform may be locally smoothed by forming blocks by area. That is, the overall brightness may not appear evenly through the above smoothing process. In this case, different algorithms should be applied to the light and dark areas, or even if the same algorithm is applied, different parameters must be applied to block-by-block images. In other words, by setting a block of a somewhat small size and moving it, the pixels in the area are processed by area. Even though there is a difference in overall brightness, the average brightness of the pixels in the block is the same, so that the same algorithm or the same parameter can be used to binarize the document.

The area extractor 64 determines whether a given area is a text area, a graphic area, or a picture area by analyzing a document using wavelet transform. That is, the area extracting unit 64 performs wavelet transform on the image corrected by the image converting unit 63, and also finds the bounding box of the letters by using the binarized image. The binarized image can be used to easily distinguish whether each pixel of the binarized image is a character or a background. That is, each pixel has a value of one of 0 (black) to 255 (white), which is a process of recognizing it as a character if it is smaller than the threshold based on a certain threshold and a background if it is larger than the threshold. In this case, the threshold may be adaptively changed block by block as described above, rather than a fixed value.

In addition, the four corners of the obtained text area are found to extract corner points of the rectangular mesh to be covered in the text area. In other words, once the text area is determined, the corner points of the square mesh can be easily extracted. As a specific method, it is necessary to find the point where the non-zero pixel first appears while proceeding inward from the edge of the binarized image.

The horizontal line aligning unit 65 aligns the horizontal direction lines of the rectangular meshes from the horizontal line using the shape of the empty space between the edges flowing horizontally in the text area and the text line. In other words, text lines are curved like a book. Therefore, the degree of deflection is determined through this, and the mesh is used as information for aligning the straight lines.

Determine the degree of curvature along the bottom of the text, or determine the degree of warp along the empty space between the text lines. In this case, Hough Transform can be used as the edge direction detection algorithm.

The vertical line aligning unit 66 first divides the rectangular mesh made by the area extracting unit 64 at regular intervals along the vertical direction. The thickness of the gap is adjusted to accommodate a sufficient amount of text information in it. That is, the interval is adjusted relatively narrowly in the case of low resolution and relatively narrowly in the case of high resolution.

The vertical space adjusting unit 67 adjusts the vertical space of the text area by using the attribute information of the characters of the text area by the area extraction unit 64 and the shape of the document estimated by the shape estimating unit 62. Attribute information of the characters of the text area includes the spacing of the characters and the thickness of the characters. In addition, since the characters in the curved part of the area far from the document vertex are obliquely photographed, the thickness of the character and the blank becomes smaller in proportion to the depth from the vertex, and the focus is also blurred. Use the rate of change of illuminance at the vertical edge of the difference between good focus and blur. In the well-focused areas, the edges will be sharp, so the rate of change in illuminance is high, and the rate of change in illuminance is small. Using the text, the thickness of the blanks, and the degree of defocusing, you can infer how far the part is from the highest point in the document.

That is, by using these elements, the spacing of the vertical lines is adjusted so that one vertical spacing in the image is the same when the document is restored. In addition to the above-described character information of the text area, it is possible to perform more accurate vertical spacing adjustment using the document shape estimated by the shape estimating unit 3.

The image restoring unit 68 induces a transformation equation of each point so that the square mesh obtained by the region extraction unit 64 becomes a smooth mesh at regular intervals, and adaptively changes the position of each pixel in the text area according to this transformation equation. Thereby, the restored image can be obtained.

That is, the scanning image or the binarized scanning image output from the image conversion unit 63 and the aligned scanning image output to the vertical line alignment unit 66 are obtained by multiplying the matrix of the pixel of the book image before distortion. . Therefore, by multiplying all the pixels of these scanning images by the inverse matrix of the square mesh, an image before distortion can be obtained. In other words, the geometrically distorted original book-shaped original as shown in FIG. 5 may be said that each point of the original book-shaped rectangle is moved to another point by a certain matrix product. Therefore, by combining a plurality of parameters obtained above to find out what this matrix is, and then obtaining the inverse of the matrix and multiplying it, a corrected scanning image as shown in FIG. 8, that is, an image close to the original image can be obtained.

7 is a flowchart illustrating an image forming method according to an embodiment of the present invention.

When the book is placed on the glass 2 and the scanning button is pressed, a scanning image is generated by the CCD 9. The scanning image is output to the illuminance detection unit 61 of the image processing unit 20. The illuminance detection unit 61 scans In consideration of various factors affecting the illuminance of the image, the illuminance difference of the scanned image is detected (S71).

The shape estimating unit 62 estimates the shape of the document using the illuminance difference due to the shape of the document among the illumination intensity change information of the detected scanning image (S72). It is preferable that the shape estimation of this document is estimated by a shape from shading algorithm that restores the three-dimensional shape from the two-dimensional image.

The illuminance change information of the scanned image detected by the illuminance detector 61 is output to the image convertor 63 and input to the image convertor 63, that is, an acquired image due to the illuminance difference generated while scanning a book. Is used to correct close to the illuminance of the original image. In addition, the image converting unit binarizes the image whose illuminance is corrected and outputs the binarized image (S73).

The area extracting unit 64 determines whether the image area is a text area or a graphic area through document analysis using wavelet transform on the image whose illuminance has been corrected (S74). In this case, the text area may be extracted by performing wavelet transform on the scanned image. Also, the area extractor 64 extracts a corner point of the rectangular mesh of the text area using the binarized image output from the image changer 63 (S75).

The horizontal line aligning unit 65 determines the degree of warp in the horizontal line direction using the line information of the text obtained by the binarized image in the text area using the information output from the area extracting unit 64, and from this the square mesh Align the horizontal lines of (S76).

The vertical line aligning unit 66 divides the square mesh output from the area extracting unit 64 at regular intervals along the vertical direction (S77). In this case, the thickness of the gap is adjusted so that a sufficient amount of text information is contained therein.

The vertical spacing adjusting unit 67 adjusts the vertical spacing of the characters of the text area by using the attribute information of the characters of the text area and the shape of the document estimated by the shape estimating unit 62 (S78). The attribute information of the characters of the text area refers to the spacing and thickness of the characters, and includes the degree of focus defocusing in addition to these information.

The image correction unit 68 multiplies the coordinates of the rectangular corner points by an inverse matrix for converting the position of each point into a rectangular shape, and adaptively positions each pixel of the text area (S79). In this case, the adaptive position of each pixel of the text area may be adaptively positioned by multiplying an inverse matrix of all coordinates of the scanning image or the binarized image.

Although the image forming apparatus having the scanning module has been described above, the present invention can be applied even when the scanning module is not provided. That is, the printer received to print the image scanned by the scanner by the scanner can print after performing the image correction according to the present invention on the input captured image. Therefore, the application of the present invention is not limited to the case with the scanning module.

In addition, although the present invention has been described taking the case of scanning a book, the present invention can also be applied to distortion generated when photographing an original with a camera or the like.

According to the present invention, distortion of the scanning image generated when scanning a book can be effectively corrected.

In addition, it is possible to correct the distortion of the picked-up image to show the user an easy-to-read book image, and to increase the recognition rate of the document recognizer.

Claims (9)

An image forming method for correcting a captured image for a document, Estimating the shape of the document by extracting an illuminance difference from the captured image; Extracting a text area using the captured image and calculating a rectangular corner point of the text area; Adjusting vertical spacing of characters in the text area by using attribute information of the characters in the text area and the shape of the estimated document; And multiplying the coordinates of the rectangular corner points by an inverse matrix for converting the position of each point into a rectangular shape, and adaptively positioning each pixel of the text area. The method of claim 1, And extracting the text area by performing wavelet transformation on the picked-up image. The method of claim 1, The extracting of the text area is performed by performing wavelet transform on the image of which the illuminance change of the picked-up image is corrected. The method of claim 3, The calculation of the rectangular corner point of the text area is characterized by binarizing the corrected image, and calculating using the binarized image. The method according to claim 3 or 4, And determining a degree of warp in a horizontal line direction using text line information of the binarized image of the text area. The method of claim 1, And the shape estimation of the document is estimated by a shape from shading algorithm which restores a three-dimensional shape from a two-dimensional image. The method of claim 1, The attribute information of the characters of the text area includes an interval of characters and a thickness of characters. The method of claim 4, wherein And the adaptive position of each pixel of the text area is multiplied by the inverse of all coordinates of the captured image or the binarized image. An image forming apparatus for correcting a captured image for a document, A shape estimating unit for extracting a light intensity difference from the picked-up image to estimate the shape of the document; An image converter which calculates a change in illuminance of the captured image to correct the captured image and binarizes the corrected image; An area extracting unit extracting a text area and a rectangular corner point of the text area using the corrected image and the binarized image; A vertical spacing adjusting unit which adjusts a vertical spacing of the characters of the text area by using attribute information of the characters of the text area and the shape of the estimated document; And an image correction unit for multiplying the coordinates of the rectangular corner points by an inverse matrix for converting the position of each point into a rectangular shape, and adaptively positioning each pixel of the text area.

Images