JPH0244486A - Document picture processing method - Google Patents

Abstract

PURPOSE:To detect the inclination of the whole of a document with high accuracy even in the document having a few amount of videotex or the videotex of multi-size by detecting the inclination of a videotex string by checking position relation among videotex areas. CONSTITUTION:A videotex area detecting mechanism 2 extracts the videotex area by detecting continuous black image elements and compulsorily divided black image elements or features on which those image elements are projected, and a videotex area selecting mechanism 3 extracts a videotex area group having similar nature. Next, a videotex string detecting mechanism 4 performs a videotex string detection processing or a videotex area detection processing from a selected videotex area or the position relation of an original videotex area. And a videotex string processing mechanism 5 connects feature points for detecting the inclination of the videotex area having similar features (vertical width, vertical and horizontal width, arrangement and change point of black image element, or identification result of rectangular pattern, etc.) for a detected videotex string with a straight line, and detects the inclination by assuming the inclination of the straight line as that of the document. In such a way, it is possible to find the inclination of a document image with a few amount of character strings with high accuracy.

Description

DETAILED DESCRIPTION OF THE INVENTION (1) Technical field to which the invention pertains The present invention relates to detection of character/figure strings in a document image in document reading devices, document filing devices, etc. ) detection.

The present invention relates to a document image processing method that performs processes such as detecting the inclination of a two-character/graphic string section at a distance and with high precision.

(2) Conventional technology How to detect character strings from document images.

As a prior art, “Paper component extraction method that does not depend on format information” IEICE Theory (D) Vol. J66-D, No. 1 (
1983) [Reference 1]. This method calculates the projection of black pixels in a document image, and detects the inclination of a character string or document from the periodicity of the projection function. This method gives good results when there are many character strings in one document, but it is not sufficient when there are only a few character strings or when the position of character strings in a document changes depending on the number of columns. It was difficult to obtain accurate accuracy.

As a method for detecting the skew of a document image, there is a conventional technique known as ``A method for correcting the skew of a document image''.
D) Vol, J69-D, No. 11 (1986)
(Reference 2), “Study on skew detection and word segmentation of English documents”, 1985 IEICE National Conference, No.

1510 (1987) (Reference 3).

The method in Document 2 involves extracting the contours of black pixels on a document image to obtain its circumscribed rectangle, and then rotating the document image at a fixed angle to determine the characteristics of the circumscribed rectangle (for example, the length of the base). is projected in a certain direction, and the angle at which the projection shows a steep peak is detected as the inclination of the document.It speeds up processing by using the features of the circumscribed rectangle rather than using the input document image itself. This is the method that was planned. This method can accurately detect the inclination of a document if the angular resolution is small, but it has the disadvantage that a small angular resolution increases the amount of processing and reduces the processing speed.

On the other hand, the method in Document 1 calculates a circumscribed rectangle from a block of black pixels on a document image, then detects rectangles of equal size, and connects the rectangle of interest with adjacent rectangles that exist above, below, to the left, and to the right. The process of determining the local direction of characters is performed over the entire text area of two document images, and the average value of the detected individual inclinations is regarded as the inclination of the document. Although this method is effective in some cases, it has the disadvantage that inclination detection accuracy decreases if the number of rectangles is small.

As shown above, these methods assume that the document being processed has a large number of character figures that can be extracted as circumscribed rectangles or projections. This method has the disadvantage that it is difficult to obtain sufficient tilt detection accuracy for documents consisting of characters of multiple sizes.

(3) Purpose of the Invention The purpose of the present invention is to solve the problems of conventional methods devised on the premise that there are many characters in two documents in two-document image processing, and to Naku 2
Documents with few characters and figures.

It is an object of the present invention to provide a method for accurately detecting the interval of a two-character graphic string, its direction, or the inclination of the entire document even for a document containing multi-sized character graphics.

(4) Structure of the Invention (4-1) Differences between Features of the Invention and the Prior Art The present invention can be applied even if one document is tilted by focusing on a character string or graphic string in a certain limited section on two documents.

We focused on the fact that characters and figures of equal size are lined up in a specific direction (e.g., vertically or horizontally), and that their mutual relationships are preserved. Alternatively, predict and extract a character/figure string or a character/figure string area from the peripheral distribution, and detect the text/figure string interval and slope of the document image from the mutual relationship of the positions of the character/figures within the 5 character/figure string or within the character/figure string area. The most important feature is that Therefore, the method of Reference 1 which calculates the inclination of a character string or document from the projection of black pixels in the document, which was taken up as prior art, the method of Reference 2 which calculates the inclination of a document using only the statistical properties of the circumscribed rectangle, and The method of Reference 3, which calculates the inclination of the entire document by statistically processing the local inclination found from the positions of vertically adjacent rectangles of the characters in the document, is that it processes by selectively using figures in one document. different.

(4-2) Example [Example 1] FIG. 1 is a block diagram illustrating the processing method of the present invention,
1 is an input terminal for a document image; 2 is a character/figure area detection mechanism that detects continuous black pixels, forcibly divided black pixels, or features projected therefrom, and extracts a 2-character/figure area; 3;
Reference numeral 4 designates a character/graphic area selection mechanism, 4 a character/graphic string detection mechanism for detecting a character/graphic string and a character/graphic string area, 5 a character/graphic string processing mechanism, 6 an output terminal, and 7 a control section.

Document image data consisting of binary values expressed as "0" and "l", such as white and black, is input from the input terminal 1, and the 3-character graphic area detection mechanism 2 detects the following in the input document image data: (b) Enclose discontinuous parts based on the results of black pixel continuity detection processing [in the case of Fig. 2 (a)], or (b) undesired black pixels in the black pixel continuity detection process. If there is no discontinuity, forced division is performed to create enclosures (separated at regular intervals [in the case of Figure 2 (b) (C)]) or partitioned taking into account changes in the peripheral distribution [Figure 2 (d)]
) from the character/figure area M? that surrounds the block of black pixels. Detects its existence position information, that is, the upper side.

Coordinates of base, left side, right side Y il+ Y iZ+
xil+xiZ is determined and sent to the primary processing mechanism.

Furthermore, the circumscribed rectangle M? Instead of short interval marginal distribution B? (S
Horizontal address Vr1 as JI area or position information
．． Yrz.

Alternatively, the subsequent processing can be approximately realized by determining the vertical address (Xil+xiZ).

Next, the character/graphic area selection mechanism 3 extracts a group of character/graphic areas M: having similar properties.

This character/graphic area group M1 is 1, for example, 1 character/graphic area M?
From the position information, determine the vertical width MT and horizontal width MW of each character/figure area.
, calculate the histograms H(y) and H(x) of
From there, select a character/graphic area with a vertical width MTi (horizontal width MW) that is approximately equal to the representative value MT (MW) of the vertical width (or horizontal width) of the character/graphic area found there, and group-stitch those character/graphic areas together. (Equations (1), (1)'). In the case of a figure such as a line segment or a case where individual characters are connected, the characteristics obtained for the character figure area that is forcibly divided at regular intervals are used.

For narrowing width: MT*(1-α)-β<MT elbow*(1+α)+β
For width: M*(1-α)-β<gate-,<MW*(1+α)+β
-(1) 'α and β are determined by the image resolution of the data, but 8 pixels/
In the case of mm, α is 0.05 to 0.10. β is 1-2
All you have to do is choose. In addition, in selecting the two-character graphic area, instead of the histogram H(Z, H(x), the rectangle occupancy functions G(y), G(x
), a value approximately proportional to the area occupied in the document image for each size of one character/graphic area can be determined, and a character/graphic area suitable for detecting the inclination of two character strings can be determined. FIG. 3(a) shows the document image data tilted by about 5 degrees, and FIG. 3(0)) shows the document image data shown in FIG. 3(a). -, -r 工, ``nu, r-'' r Ａ J Histogram H(y) taken for the reduced width of a character/figure area M9 such as ``i'' where each block is surrounded by a rectangle, and its rectangle occupancy function G( y). From Fig. 3(b), the vertical width MT, MT, ,M
It can be seen that there are many character/graphic areas with T.

Next, the character/graphic string detection mechanism 4 selects the selected character/graphic area Mi or the original character/graphic area M? Character/graphic string detection or character/graphic string area detection processing is performed based on the positional relationship of We will discuss the processing of , but when using information in the vertical direction, it is expressed by distinguishing the subscripts of the symbols.

FIG. 4 shows how character and graphic strings are detected, and shows a case where the character and graphic region Mi (or M?) of interest exists in the character and graphic string of the second row.

It shows the process of extracting a character/figure string. As this figure shows, the character/figure area of interest M1 (or M?)
is projected in the horizontal direction on the document image, the rectangle closest to the character and graphic area of interest is detected among the character and graphic areas where the projections overlap, and if it exists on the right side, the character and graphic area is converted to Ml-+C or M? -+).

Next, the character/figure area to be focused on is Ml. 2 (or M?
Z), take the projection to the right in the same way as above,
The process of detecting the closest character/graphic area is repeated (hereinafter, this process will be referred to as propagation process).

The same propagation process as for the right side is repeated for the left side of the character/graphic area M+ (or M't). In this way, the character/graphic string for the second row can be detected.

Next, the one character/figure string processing mechanism 5 performs (a) (circle detection processing) on the detected character/figure string with equal characteristics (vertical width, vertical width and width, arrangement of black pixels, changing points, (rectangular pattern identification results, etc.) for tilt detection of text/figure areas (■Address of the center point of the text/figure area, ■Address of the center point of the bottom side, ■Address of the center point of the top side, etc.) with a straight line. Connect, the slope of the straight line θ
? is detected by regarding it as the skew of the document. The method for determining the slope is the slope θQ, which is obtained by connecting feature points for slope detection with a straight line between character and graphic regions that are farthest within a limited section in a character and graphic string and have the same characteristics. As shown in , between character and graphic areas that are the same character string and have the same characteristics.

Tilt θ between feature points for tilt detection? It is conceivable to find a value with a small variance of the average value θ1, or a peak peak in a histogram of slope values, and detect the slope by regarding it as the slope θ of the document image.

In addition, the reference line that minimizes the variation (squared error, etc.) of feature points for tilt detection that occurs when the connection between two character and figure strings is expressed as a straight line is determined by least squares approximation, and the direction of the reference line is determined by the character and figure. It is also possible to set the direction of arrangement of the columns, that is, the inclination θ, (θX) of the character/figure column.

(b) The process of detecting the arrangement of character/figure strings is performed in multiple directions (
For example, the dispersion of the inclination θa of the character/figure string determined by the vertical and horizontal directions and the steepness of the peak of the histogram are compared, and the one with the smaller 1 variance or the sharper peak is determined as the alignment direction of the character/figure string. Alternatively, in a character string formed by the character/graphic area of interest, the number of occurrences of the character/graphic area within a certain interval may be determined, and the direction in which the character/graphic string is arranged may be set to the larger number of appearances.

(+1) The inclination of the character string selected in the process of the ([+) term, or the average of the (approximate times) of a plurality of character strings is regarded as the inclination θ of the document, and is output to the output terminal 6.

FIG. 6 shows the result of detecting the tilt of the document image shown in FIG. 3(a) in this way and correcting the tilt using the detected tilt value.

The control unit 7 plays the role of transmitting which process has been selected in each processing mechanism.

By using the above-described processing, the present invention can accurately and quickly determine the slope not only for document images with many one character strings but also for document images with few two character strings. So.

It is effective for high-speed, high-precision format recognition of tilted document images and for accumulating high-quality images when filing a single document.

[Embodiment 2] FIG. 7 shows a modification of Embodiment 1 in which the character/graphic area selection mechanism 3 is removed and the processing of the 9 character/graphic string detection mechanism 4 is limited to or focused on the character/graphic string area as shown in FIG. An example is shown in which prediction of a character/graphic area is performed in the following manner. That is, the character/graphic string detection mechanism 4 detects the character/graphic area M? (or Mt) in the horizontal direction on the document as shown in FIG. 8 to obtain the projection function F(y),
The maximum value (large value) of F(y) is set at the center IP of the character/figure string.
, and the interval P, ±r*MT+ is defined as the "character/figure string area" as shown in the figure. Feature points for inclination detection are connected with straight lines in a plurality of corresponding character/figure area groups existing within the interval (δ*MTL), and a plurality of inclinations θ? Find 9 slope θ? The histogram of the document image is calculated, the largest value is detected, and the tilt candidate
Let it be h.

The inclination of the document image is calculated according to the type of feature point (■Address of the center point of the character/figure area, ■Address of the center point of the base,
(2) The one with the smallest variation is selected from among the tilts (such as the address of the center point of the upper side) and is set as the tilt θ of the document image.

Since this method omits or simplifies a part of the processing, it has effects such as speeding up the processing and making it possible to reduce the scale of the processing. Furthermore, the inclination can be detected even in a string of characters and figures in which the projections of blocks of black pixels do not overlap.

[Third Embodiment] FIG. 9 is a diagram illustrating a method in which the character/graphic string detection is developed into the same character/graphic string detection in the processing of the character/graphic side processing mechanism 5 of the first embodiment. Note that the * mark shown in the figure is “
Either "0" or "1". That is, the same processing is performed up to the process of determining a character/graphic string from the character/graphic region M (or M?) of interest, and when determining the inclination θ?, the inclination in each direction is determined and processed.

For example, the slope θ? When calculating, calculate the angles of the inclinations in the right (front) direction and the left (rear) direction separately and tabulate them as shown in Figure 10. (1) From the character figure area of interest M: (or M?) Inclination Rθ in the right direction? and the leftward tilt Lθ? There is a difference of more than θTH, and (ii) predict the leftward inclination of the character/figure area of interest. The predicted inclination in the right direction of the character/graphic area Mt-1 (or M?-1) adjacent to the right side of the character/graphic area of interest is Rθ? , I or θTH, if it is less than Mt (or M?) and Ml,
It can be considered that the character string is separated between 1 (or M?, l).

This method detects the spacing between characters and figures to find the inclination of the document image, so if the character or figure is shifted due to columns or a mixture of characters, figures, tables, or photographs, or if the center of the character or figure is mixed due to a mixture of uppercase and lowercase letters, It is possible to accurately detect the skew of a document even when the character/figure column width changes. Furthermore, if this method is applied to continuous line segments, it is possible to distinguish between straight lines and curved lines.

By adding this method to conventional document structure recognition in this way, more accurate document structure recognition becomes possible.

[Embodiment 4] FIGS. 11 and 12 are diagrams illustrating a method in which the two-periphery distribution is applied to character/graphic string detection and tilt detection. First, one document is divided into tanzaku-shaped regions of height h (Fig. 11(a)), and the peripheral distribution of each is determined by vertically projecting black pixels (Fig. 11(b)). The cluster represents a projection). If the input image is multivalued, you can simply accumulate the pixel density. Next, the section where the value of the two marginal distributions is greater than a certain value ε and the width is greater than or equal to ζ (see Figure 11)
b) Detect x1 to x2) shown in the figure and use it as its projection component.

Here, ε and its value may be set based on the magnitude of noise to be removed. This projected component can be considered to reflect the position of the character string as it is. In addition, if two documents include charts, etc., the size of one chart is larger than the size of characters in the document, so if the width of the projected component is less than a certain value η, the character string is All you have to do is select the projection component according to . The value of η needs to be set based on the string width. This value may be set in advance as a parameter, or if the number of characters and figures in the document is large, a histogram of the widths of the projected components may be obtained and the value may be set based on its representative value.

FIG. 12 shows the process of clustering each projection component obtained by the method shown in FIG. 11 and assigning a different label to each character/graphic string. First, in FIG. 12, the same label ■ is given to each projection component that overlaps with each other when viewed in the vertical direction (
FIG. 12(a) Step 1). Next, among those labeled with the same number, those with a small degree of overlap when viewed in the vertical direction are assigned a different label (2) (FIG. 12A), step 2). Here, each of the projected components given the same label is taken as a candidate for a character/figure sequence, and its direction is determined. The direction of the character/figure string may be determined by least squares approximation using a straight line passing near the midpoint of the projected components, or the direction between adjacent projected components may be determined for all projected components and the overall average determined. It may be determined by using the method described above. Alternatively, the method described in the character/graphic string processing mechanism of the first embodiment may be used. This process is performed on the openings of characters and graphics to which the same label is given, and the orientation of the entire two-character and graphics sequence is averaged to determine the inclination of the document. Next, based on the slope obtained in step 2, the positions of continuous projection components are predicted, and those that deviate significantly from the predicted positions are given a new label ■ (
FIG. 12(C), step 3). The direction of the character/figure string is determined by the same processing as described in step 2, and a more accurate inclination of the document than that obtained in step 2 is determined.

This processing makes it possible to detect the skew of a document even if there are character strings in different positions in each column. Here, the projection components given the same label reflect individual character strings as they are. Therefore, character strings in two documents can be extracted at the same time as detecting the tilt of one document.

As mentioned above, one figure or table can be included by dividing the area of the input document into tanzag shapes and using the marginal distribution found in each area.

Even in documents where the position of character strings differs from column to column, it is possible to detect the inclination and extract nine character strings. In addition.

It is not necessarily necessary to perform all of the processing from step 1 to step 3, and it is possible to omit a part of it depending on one document. Further, although the case where the character string is in the vertical direction has been described as an example, it goes without saying that the same processing is possible even in the case where the character string is in the horizontal direction.

FIG. 13 shows a flowchart of the main parts in the configuration diagram shown in FIG. 1.

In process (2), a circumscribed rectangle is detected, in process (2) a histogram is calculated, in process (2) a maximum value is calculated, and in process (2), as shown in FIG. 3(b) (local maximum value M
T1. M T t------ is detected, the tilt is detected in process (2), the tilt is determined in process (2), and (the tilt) of the document image is detected in process (2).

FIG. 14 shows a main part flowchart corresponding to the embodiment shown in FIGS. 11 and 12.

In processes (1) and (2), the area is divided into tanzag-shaped areas as shown in FIG. 11(a). Processing 0. 1st in up to ■
Figure 1 (b1 Obtain the projection as shown in Figure 1. In the process @, the first
The process shown in FIG. 2(a) is performed, and in process (2), labeling is performed as shown in FIG. 12(b). Obtain the tentative tilt of the document by averaging in the process.

In process (2), labeling is performed as shown in FIG. 12(C). Obtain the tilt of the document in process (2). The result is output in process (■).

(5) As described in detail of the invention, according to the present invention, character/graphic strings of a document image, five character/graphic string alignment directions (vertical writing direction or horizontal direction),
The spacing between characters and figures, as well as the inclination of the characters and figures, can be detected quickly and accurately, so it is possible to correct the inclination of one document image or
Recognition of character/figure alignment in document recognition It is effective in character type identification using character/figure alignment in single character recognition.

[Brief explanation of the drawing]

Fig. 1 is a block diagram illustrating the processing method of the present invention. Fig. 2 is a conceptual diagram of text/graphic areas showing different extraction methods for text/graphic areas. Fig. 3 (a) and (b) are tilted document images (a ) and its histogram and rectangle occupancy function b), Figure 4 is a conceptual diagram of character/figure string detection, Figure 5 is a conceptual diagram of slope detection,
FIG. 6 is a diagram showing the result of tilt correction of the document image shown in FIG. 3(a), FIG. 7 is a simplified configuration diagram of the present invention, FIG. FIG. 9 is a conceptual diagram of character/figure string detection, and FIG. 10 is a diagram of tilt detection results for each direction. Fig. 11 is a diagram showing the method of determining the projection component using marginal distribution, Fig. 12 is a diagram showing the principle of detecting a character/figure sequence and its inclination by labeling the projection component, and Fig. 13 is the diagram shown in Fig. 1. FIG. 14 shows a flowchart of the main parts corresponding to the embodiment shown in FIGS. 11 and 12. In the figure. The input terminal of 1 is a character/figure area detection mechanism. is a text/figure area selection mechanism. is a character/figure string detection mechanism. is a character/figure string processing mechanism. is an output terminal. is the control section.

Claims (4)

[Claims] (1) In a document image processing device that detects and corrects the tilt of a document and imports the document image data input from the image input unit, the document image processing device detects and corrects the tilt of the document, and then detects and corrects the tilt of the document. A character/figure area detection mechanism that determines the area of a character/figure based on the position and size of a circumscribed rectangle of a partial block obtained by dividing the block, or the peripheral distribution obtained by projecting black pixels, and a character/figure area detection mechanism. size and frequency of appearance,
Alternatively, a character/graphic area selection mechanism that selects a character/graphic area to be used for inclination detection based on the characteristics or pattern recognition results within the character/graphic area, and the selected character/graphic area or all character/graphic areas that exist in the vicinity. A character/figure string detection mechanism detects the arrangement of character/figures based on the degree of overlap with the character/figure areas and their positional relationship, and a character/figure string detection mechanism detects the inclination of the character/figure string from the positional relationship of the character/figure areas that make up each character/figure string. 1. A document image processing method, comprising: a character/graphic string processing mechanism, and detects the inclination of the character/graphic string by checking the positional relationship between the character/graphic areas. (2) Claim (1) characterized in that the character/graphic area selection mechanism is removed, and the character/graphic string area to be processed is limited in the character/graphic string detection mechanism, thereby simplifying the tilt detection process. Document image processing method described. (3) In the character/graphic string processing mechanism, the inclinations between character/graphic areas are determined in multiple directions using a specific character/graphic area as a starting point, and the inclinations between the character/graphic area of interest and other character/graphic areas are determined by direction. Claim: 1. An identical character/figure string section is detected by detecting a change in the inclination of the character/figure string and a change in the deviation of the character/figure string from a reference line.
1) Document image processing method described. (4) In the text/figure area detection mechanism, the input document image is divided into small regions and the marginal distribution for each area is determined, and in the text/figure area selection mechanism, the marginal distribution obtained by the text/figure area detection mechanism is The part corresponding to the character string is selected, and the character/graphic string detection mechanism extracts the character/graphic string based on the overlapping position of the peripheral distribution selected by the character/graphic area selection mechanism. 2. The document image processing method according to claim 1, wherein the inclination of the entire document or the direction of the document is determined based on the inclination of the character/graphic string obtained by the character/graphic string detecting mechanism.