JP3304404B2 - Tilt detection method and image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus such as a document filing system using an optical disk, and to a tilt detection method suitable for detecting a tilt of an input image.

[0002]

2. Description of the Related Art A document filing system using an optical disk has been commercialized, and a large amount of documents can be stored and retrieved with image data. Also, a technique for searching a large amount of text data at high speed has been developed. As described above, an environment in which advanced document information can be used by an electronic method instead of paper is being prepared. On the other hand, with the increase in the capacity of the document system, a large amount of manpower has been used for document registration. For this reason, a technology for recognizing characters in a document and automatically registering the characters in a system has become indispensable.

Various black lines such as ruled lines, enclosing frames, split rules, and underlines are used in a document. In order to recognize a character in a document including such a black line, it is necessary to detect the black line in the document and cut out a character line using information such as the position and inclination of the black line. However, a black line in a document image is often inclined. When scanner input is performed using an automatic paper feeder, it is said that the inclination of a document image due to a shift of a document is within ± 10 degrees. In order to cope with such a case, it is necessary to extract a black line from the inclined image and calculate the inclination of the extracted black line.

Conventional techniques for calculating the slope of the detected black line include (a) an end point fitting method and (b) an iterative end point fitting method (R. Nevatia "MACHINE PERCEPTION"). Understanding ”Keigaku Shuppan) pp. 120
-164 (Showa 61)), (c) Hough transform method (Matsuyama, Kosui "Hough transform and pattern matching" Information Processing Vol. 30 No. 9 (Heisei 1)), and the like.

[0005] Among them, (a) is a method in which a black line is approximated by a straight line connecting end points of the black line, and a slope is obtained. This method requires a small amount of processing and does not require a large storage capacity. However, noise or jaggies (jaggies at the edges of the image)
In some cases, high accuracy cannot be obtained.

In (b), two end points of a given curve are connected by a line segment, and when the distance to a point farthest from the straight line is equal to or more than a certain value, the line segment is divided into two. Further, the same processing is repeated for the divided line segments. The process ends when there are no objects farther than a certain value from the line segments. Thereby, the contour of the black line can be approximated by a segmental line segment, and the inclination can be obtained. However, this method
It is necessary to repeat the calculation of the distance from the straight line to the point for each pixel, and it is difficult to increase the speed. In addition, a large storage capacity is required to represent a polygonal line.

(C) finds a mapping for converting a coordinate of an arbitrary point in the image into a straight line, and finds an accumulation point of the straight lines of the mapping of the black pixels, thereby detecting the inclination of the straight line from the image. . This method requires a large amount of storage capacity to represent the (a, b) plane, and requires a large amount of processing time to find an integration point.

[0008]

Among the conventional methods for detecting the inclination of a black line or the like in a document image, the end point fitting method (a) is not high in accuracy. Also, the iterative end point fitting method of (b),
(c) The method using the Hough transform has a problem that the processing speed is slow.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image inclination detecting method and an image processing apparatus which can approximate a figure in a binary image at high speed with high accuracy and detect the inclination.

[0010]

SUMMARY OF THE INVENTION According to the present invention, there is provided a system comprising:
Encodes the value image, and calculates 2 from the encoded binary image data.
A predetermined figure in a value image is extracted, and a slope of the extracted predetermined figure is obtained by linear approximation using a least squares method.

In particular, it is realized by calculating the minimum second moment of a figure in a binary image at a high speed and performing linear approximation by using an image coding method such as run data coding and contour data coding. .

[0012]

By applying the linear approximation by the least squares method to image processing, a predetermined figure in a binary image can be linearly approximated with high speed and high accuracy, and the inclination can be detected.

In particular, the second moment with respect to a straight line of a run in a certain figure can be represented by a coordinate of the midpoint of the run and a polynomial of the run length. By obtaining a coefficient that minimizes the value of the polynomial, the slope of a straight line that approximates the figure is obtained. Similarly, the minimum second moment can be calculated using the contour length and the contour direction of the contour data. The method of calculating the minimum second moment using data encoded as run data, contour data, etc. reduces the number of times of access to the memory and the number of calculations compared to the method using pixel data, so that figures in a binary image can be displayed at high speed. Can be obtained, which is effective for an object having a large number of pixels.

Although the straight line approximation itself by the least square method is a known method in the field of statistical mathematics and the like, there is no example in which it is applied to inclination detection in the field of image processing. High accuracy, (b) iterative endpoint fitting,
(c) It does not require a large storage capacity as compared with the Hough transform method. Further, even when processing a large number of pixels, there is no problem in terms of processing speed if run data and contour data are utilized.

[0015]

FIG. 1 is a diagram for explaining the principle of extracting a character line from a document image by detecting the inclination of a black line. As shown in FIG. 1A, there is a case where the document image 101 is input by being tilted from the scanner, or the ruled line 102 and the character string 103 in the document are printed while being tilted. As in these cases, in order to recognize a tilted character in a document, it is necessary to first detect the position of a character line in the document. Generally, a character line is detected by searching for a large blank in the vertical and horizontal directions of the document.

However, when the document image itself is inclined, large blanks in the vertical and horizontal directions of the document cannot be found well, and it is difficult to extract a character line from the document image. Further, as shown in FIG. 1A, there is a problem that if a table or a ruled line is mixed with a character line, it becomes difficult to detect a blank. Therefore, first, as shown in FIG. 1B, a black line is extracted from the document (only the horizontal black lines 104 and 105 are detected in the example of FIG. 1B), and then FIG.
The end point coordinates 106 and 1 in the extracted document as shown in FIG.
07 and the inclination 108 of the black line, the inclination is corrected, and the black line is removed. Then, as shown in FIG.
09 can be detected.

FIG. 2 shows a procedure of character recognition using black line inclination detection. First, in step 201, a document image is input from a scanner or the like. Next, in step 202, a black line is extracted, and in step 203, the position and inclination of the black line are detected. A character line in the document is extracted in step 204 using the detected position and inclination of the black line, a character in the character line is recognized in step 205, and a character code is output in step 206.

In the present invention, a straight line approximation by the least squares method will be described as an example of a method for detecting the inclination of a figure in a binary image such as a black line in a document image.

FIG. 3 is a diagram for explaining the basic principle. The XY coordinate system in the image is defined as 301. A set of points for a figure 302 substantially parallel to the X axis, such as a horizontal ruled line in a document
A

[0020]

(Equation 1)

When defined as

[0022]

(Equation 2)

A, b which minimizes the integral value I expressed by the following equation:
Ask for. This is a straight line 303

[0024]

(Equation 3)

This corresponds to minimizing the surrounding second moment. The inclination of the figure 302 can be approximated by the straight line 303 thus obtained.

FIG. 4 illustrates a method of calculating a second moment based on pixel data according to the first embodiment. An XY coordinate system is defined as 401 in the discretized image. A set of pixels included in the figure 402 in this image
S '

[0027]

(Equation 4)

In other words, I in Equation 1 is

[0029]

(Equation 5)

Can be approximated by S represented by Equation 4 is

[0031]

(Equation 6)

It can be modified as follows. Where each coefficient of Equation 6 is

[0033]

(Equation 7)

Then, the number 6 is further increased

[0035]

(Equation 8)

It can be modified as follows. Since S is a quadratic expression of a and b, and always p> 0 and r> 0, there is a minimum value. The values of a and b that minimize S can be obtained by substituting equation 7 into equation 9 below.

[0037]

(Equation 9)

FIG. 5 is a diagram for explaining the principle of a calculation method using run data of the minimum second moment according to the second embodiment. A run is a group of black or white pixels that are continuous in a certain direction in a binary image. In this specification, unless otherwise noted, a simple run means a run of black pixels in the horizontal direction (X lexical direction). The portion 502 painted black in FIG. 5 indicates one run in the image.

When the pixel data is used as in the first embodiment of FIG. 4, the amount of calculation becomes enormous when the number of pixels is large, but when the calculation is performed using the run data of FIG. There is an advantage that even if the second moment is calculated for the image processing, a sufficiently practical calculation amount is sufficient.

Here, Rj: the j-th run belonging to a certain figure (xj, yj): the coordinates of the middle point of Rj lj: the run length of Rj N: the total number of runs belonging to a certain figure

[0041]

(Equation 10)

Can be replaced by

In the following, I ′ of Equation 10 is calculated.

[0044]

[Equation 11]

Where

[0046]

(Equation 12)

In other words, I ′ is the same as

[0048]

(Equation 13)

At this time, it becomes the minimum.

FIG. 6 is a diagram showing the principle of inclination detection using contour data according to the third embodiment. As in the second embodiment, when the number of pixels is large, the second moment of the image is calculated with a sufficiently practical calculation amount without using image data.

With respect to the figure 601 composed of black pixels in the binary image, pixels which are included in the figure and which are in contact with the white pixels in the vicinity of 4 are obtained, and the centers of adjacent ones of these pixels are connected. The outline 602 is set. Also, as shown in FIG. 6, N ′ line segments Ci (1 ≦ i ≦ N
Divide into c). 603 in FIG. 6 is an example of a bend point ri, and 604 is a bend point next to 603 when the contour 602 is traced counterclockwise. The direction Di of the line segment Ci is defined as shown in FIG. E0, e1, etc. of FIG.
Vector corresponding to each direction. Also, the length of the line segment Ci expressed by the coordinates ri and ri + 1 of the pixel in the neighborhood distance of 8 is defined as (Li). In the example of FIG. 6, 605 is a line segment connecting 603 and 604, the length thereof is Li = 3, and the direction D
i = 2. The contour data is several pairs of Li and Di representing the contour C of the figure. Note that the contour has an inner contour and an outer contour, and one connected component is one outer contour and 0
Although the number of inner contours is represented by more than one, when the figure is represented by a single closed curve as shown in FIG. 6, the number of inner contours is zero. Hereinafter, unless otherwise noted, a figure is represented by a single closed curve.

In order to perform a straight line approximation by the least squares method, the integral of Equation 2 may be performed for all pixels in the figure. When this integral is performed using contour data, the result is a and b. It becomes a cubic expression, handling becomes difficult, and the processing amount increases. Therefore, as a simple method, the integral of Equation 2 is performed on the outer contour C of the figure (602 in the figure), and linear approximation is performed.

In the following, for each value of Di, the integral value of Equation 2 on the line segment Ci
Formula of I "i and polynomial of I" i with a and b

[0054]

[Equation 14]

Coefficients pi, qi, ri, si,
Indicates ti and ui.

(Di = 0)

[0057]

(Equation 15)

[0058]

(Equation 16)

(Di = 1)

[0060]

[Equation 17]

[0061]

(Equation 18)

(Di = 2)

[0063]

[Equation 19]

[0064]

(Equation 20)

(Di = 3)

[0066]

(Equation 21)

[0067]

(Equation 22)

(Di = 4)

[0069]

(Equation 23)

[0070]

(Equation 24)

(Di = 5)

[0072]

(Equation 25)

[0073]

(Equation 26)

(Di = 6)

[0075]

[Equation 27]

[0076]

[Equation 28]

(Di = 7)

[0078]

(Equation 29)

[0079]

[Equation 30]

Here,

[0081]

(Equation 31)

And this is given by the following equation:

[0083]

(Equation 32)

Then, when I ″ is substituted into the contour (6 in FIG. 6)
02) is the second moment around the straight line l of the pixel. The values of a and b that minimize I ″ in Equation 32 are the same as in Equation 13.

[0085]

[Equation 33]

Is as follows.

FIG. 8 shows a specific processing procedure of the first embodiment for detecting the inclination of the black line. FIG. 8 shows the detection of the inclination of the black line using the pixel data. FIG. 8 shows one of FIG. 1 (b).
This is a processing procedure of a process of obtaining end point coordinates and inclination by performing linear approximation as shown in FIG. The linear approximation is performed for each connected component in the image in the loop of step 801. First, in step 802, p, q, r, s,
Initialize parameters such as t. Next, in the loop of step 803, coordinates of all black pixels included in the connected component
xi and yi are obtained, and in step 804, each parameter is added according to Equation 7 for each pixel. Where Equation 7
U, which is not used in the subsequent calculations, is not calculated. After adding all the pixels included in the connected component,
In step 805, the gradient an and the Y tangent bn are calculated according to Equation 9.
(An and bn are the slope a and Y intercept, respectively, of the nth connected component
b).

FIG. 9 shows a specific processing procedure of the second embodiment for detecting the inclination of the black line. FIG. 9 shows a black line inclination detection using a run. FIG. 9 shows, like FIG. 8, FIG.
This is a processing procedure of a process of obtaining a slope by performing a linear approximation as shown in FIG. 1C with respect to a black line extracted from an input image as shown in FIG. The linear approximation is performed for each connected component in the image in the loop of step 901. First, in step 902, parameters such as p, q, r, s, and t are initialized. Next, in the loop of step 903, the midpoint coordinates xi, yi and the black run length of all the black runs included in the connected component
li is obtained, and in step 904, each parameter is added according to Equation 12 for every run included in the connected component. However, u which is not used in later calculations is not calculated. After adding all the runs included in the connected component, in step 905, the gradients an and Y according to Expression 13 are calculated.
Find the tangent bn.

FIGS. 10 and 11 show a specific processing procedure of the third embodiment for detecting the inclination of the black line. FIG. 10 shows the detection of the inclination of the black line using the contour. FIGS. 10 and 11 show a linear approximation as shown in FIG. 1C with respect to a black line extracted from the input image as shown in FIG. Is a processing procedure of a process of obtaining an inclination by performing the following. The linear approximation is performed for each connected component in the image in the loop of step 1001. First, in step 1002, contour data of each connected component is generated. Next, step 1003
Initializes parameters such as p, q, r, s, and t.
Next, in the loop of step 1004, the direction (Di) and length Li of the contour forming the contour data are obtained, and step 1 is executed.
At 005, the subroutine 1006, 1
007, 1008, 1009, 1010, 1011, 1
The process branches to 012 and 1013. 1101, 11 in FIG.
02, 1103, 1104, 1105, 1106, 11
07, subroutines 1006, 1007,
1008, 1009, 1010, 1011, 1012,
In 1013, Equation 16, Equation 18, Equation 20, and Equation 2 respectively
Each parameter is added in accordance with 2, 24, 26, 28, and 30. However, u which is not used in later calculations is not calculated. After the addition for all the contour data included in the connected component is completed, in step 1014
According to 3, the inclination an and the Y tangent bn are obtained. In the above example, the contour data is expressed in eight directions. However, the inclination an and the Y tangent bn can be obtained by simplifying the contour data in a smaller number of directions. For example, when the contour is defined in a counterclockwise direction, if the lower side of the contour is a black pixel, the direction Di is often one of 1, 2, and 3, and the other cases may be ignored. it can. Therefore, in such a case, if the value of Di is any one of 1, 2, and 3, the process branches to step 1005. If not, the process ignores step 1004 for the next line segment that forms the contour data. Processing can also be performed. The errors in the values of a and b obtained in this method are negligible, but the processing time can be greatly reduced.

In the first to third embodiments, the black line in the document image has been described as an example. However, the inclination of a rectangle in a document image or a slender black figure obtained by interpolating the space between characters in a character line is described. Can also be requested.

FIG. 12 shows the configuration of a system for implementing the present invention. Reference numeral 1201 denotes an image processing apparatus incorporating the inclination calculation method of the present invention. The image processing device 1201 is connected to an image storage device 1202 such as an optical disk device, an image input device 1203 such as an image scanner, and an image transmission device 1204 such as a facsimile. From these image input devices, image data is sent to the graphic detection unit 12 via an image input unit 1205 provided in the image processing device 1201.
06, the selector 1207, the image correction unit 1209, and the selector 1210. The figure detecting unit 1206 detects a figure to be subjected to inclination detection processing such as a black line or a character line from the input image. The inclination detection unit 1208 performs linear approximation on the input image or the graphic output from the graphic detection unit 1206 by the above-described method to detect the inclination. The selector 1207 switches the input to the inclination calculation unit 1208 to the output of the image input unit 1205 or the output of the graphic detection unit 1206 according to an instruction from the outside. The image correction unit 1209 outputs an image obtained by rotating and correcting the input image according to the result of the inclination calculation unit 1208. An image output unit 1211 converts an input image or an image whose rotation has been corrected by the image correction unit 1209 into an image storage device 1212 such as an optical disk, an image recognition device 1213 such as an OCR, an image display device 1214 such as a CRT, and an image transmission device 1215 such as a facsimile. Etc. to an external device. The selector 1210 switches the input to the image output unit 1211 to the output of the image input unit 1205 or the output of the image correction unit 1209 according to an external instruction or the like.

The quantitative effect of each embodiment of the present invention will be described. Assuming that M is the number of black pixels in the black line, the first embodiment using pixel data requires 3M multiplications and 5M additions to determine the slope of the black line by the least squares method.

On the other hand, in the second embodiment using runs, when N is the number of runs in the black line, 6N multiplications, 1N divisions, and 6N additions are required. In a general computer, it can be assumed that the processing time spent for multiplication and division is the same, and that the time spent for addition is negligibly smaller than the time spent for multiplication. If such an assumption holds, the processing time of the second embodiment using the run is shorter than that of the first embodiment using the image data when M> 7/3 · N. For black lines in a document image, it is usually between M = 10N and 2000N, and the method using a run is extremely advantageous in terms of processing time.

In the third embodiment using the contour, N '
Is the number of contour data in the black line, 9N 'multiplications, 2N'
You need to divide times. Therefore, the processing time of the third embodiment using the contour when M> 11 / 3.N 'is shorter than that of the first embodiment using the pixels. In general, M = 5N 'to 4000N', and the amount of calculation required is extremely small.

[0095]

According to the present invention, linear approximation of a binary image
High-precision and high-speed execution can be achieved, and it is easy to use the image processing apparatus such as a character recognition apparatus for detecting the inclination of a preprocessed (document) image.

[Brief description of the drawings]

FIG. 1 is a diagram showing the principle of the present invention.

FIG. 2 is a diagram showing a processing procedure of the present invention.

FIG. 3 is a diagram illustrating the principle of linear approximation of a figure in a binary image according to the present invention.

FIG. 4 is a diagram illustrating the principle of a calculation method using pixel data of a linear approximation of a figure in a binary image according to the first embodiment of the present invention.

FIG. 5 is a diagram illustrating the principle of a calculation method based on run data of linear approximation of a figure in a binary image according to a second embodiment of the present invention.

FIG. 6 is a diagram illustrating the principle of a calculation method based on contour data for linear approximation of a figure in a binary image according to a third embodiment of the present invention.

FIG. 7 is a diagram showing a definition of a direction of contour data of a figure in a binary image according to a third embodiment of the present invention.

FIG. 8 is a diagram showing a procedure of a calculation method using pixel data of a linear approximation of a figure in a binary image according to the first embodiment of the present invention.

FIG. 9 is a diagram showing a procedure of a calculation method based on run data of linear approximation of a figure in a binary image according to the second embodiment of the present invention.

FIG. 10 is a diagram illustrating a procedure of a calculation method using outline data of a straight line approximation of a figure in a binary image according to a third embodiment of the present invention.

FIG. 11 is a diagram illustrating a procedure of a subroutine of a calculation method using outline data of a linear approximation of a figure in a binary image according to a third embodiment of the present invention.

FIG. 12 is a diagram showing a configuration of a system for implementing the present invention.

[Explanation of symbols]

101: Document image, 102: Black line in document image, 103
... Character strings in the document image, 104, 105 ... Horizontal black lines detected in the document image, 302, 402 ... Figures in the binary image, 301, 401 ... Coordinate system in the image, 303, 40
3, 501, 601: straight line approximating the figure in the binary image, 502: black run, 602: contour.

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Kazuki Nakajima 1-280 Higashi Koigakubo, Kokubunji-shi, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd. (56) References JP-A-59-133677 (JP, A) JP-A-4-96878 (JP, A) JP-A-3-1444863 (JP, A) JP-A-58-95465 (JP, A) JP-A-2-302612 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T ⁷ /00-7/60 G06K 9/32 G06T 1/00 H04N 1/40

Claims (7)

(57) [Claims] 1. An input binary image is run-encoded, a linear figure in the binary image is extracted from the encoded binary image data , and coordinates of a middle point of each run of the extracted linear figure are obtained. value
And a second moment of the linear figure around a predetermined axis from the run length
The straight line calculated using the second moment
The linear equation of the axis that minimizes the second moment of the figure
A tilt detection method characterized by approximating the straight line figure to the straight line obtained by a multiplication method. 2. A means for inputting a binary image, a means for extracting a linear figure in the binary image from the run-coded binary image data, and a coordinate value of a midpoint of each run of the linear figure. Passing
The second moment of the straight line figure around a predetermined axis from the run length
Is calculated, and the linear diagram is expressed using the second moment.
The inclination of the linear equation of the axis where the second moment of the
An image processing apparatus comprising: a slope detecting unit obtained by a small square method; and a unit for performing rotation correction of the binary image according to the obtained tilt. 3. A means for inputting a binary image of the document by photoelectric conversion, means for extracting from the linear shape in the binary image run coded binary image data, each of the straight line graphic
The straight line around the predetermined axis from the coordinate value of the midpoint of the run and the run length
Calculate the second moment of the figure and use the second moment
The second moment of the linear figure represented by
A means for detecting the inclination of the linear expression of the axis by the least squares method; a means for rotating and correcting the binary image in accordance with the obtained inclination; and a means for recognizing characters in the document. An image processing apparatus characterized by the above-mentioned. 4. A means for inputting a binary image of the document by photoelectric conversion, means for extracting from the linear shape in the binary image run coded binary image data, each of the straight line graphic
The straight line around the predetermined axis from the coordinate value of the midpoint of the run and the run length
Calculate the second moment of the figure and use the second moment
The second moment of the linear figure represented by
Means for detecting the inclination of the linear equation of the axis by the least square method, and means for correcting the rotation of the binary image according to the obtained inclination, and storing the binary image of the document as a document file An image processing apparatus comprising: 5. A means for inputting a binary image of the document by photoelectric conversion, means for extracting from the linear shape in the binary image run coded binary image data, each of the straight line graphic
The straight line around the predetermined axis from the coordinate value of the midpoint of the run and the run length
Calculate the second moment of the figure and use the second moment
The second moment of the linear figure represented by
Image transmitting means for transmitting a binary image of the document, comprising: means for detecting the inclination of the linear expression of the axis by the least square method; and means for rotating and correcting the binary image in accordance with the obtained inclination. An image processing apparatus comprising: 6. An outline of the input binary image is encoded, a linear figure in the binary image is extracted from the encoded binary image data, and the direction and length of the extracted linear figure on the outline are defined.
Calculates the second moment about the specified axis of the straight line figure from
And the second moment of the linear figure represented using the second moment
The linear equation of the axis where the next moment is minimized is calculated by the least squares method.
And calculating a slope of the straight line by approximating the straight line figure . 7. A means for inputting a binary image, a means for extracting a linear figure in the binary image from the contour-coded binary image data, a direction and a length of the linear figure on the contour line
Calculates the second moment about the specified axis of the straight line figure from
And 2 of the straight line figure represented using the second moment
Least squares the slope of the linear equation of the axis where the next moment is minimized
An image processing apparatus comprising: means for detecting a tilt obtained by a method; and means for rotating and correcting the binary image according to the obtained tilt.