JP2768249B2 - Document image layout analyzer - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document image layout analyzing apparatus.

[0002]

2. Description of the Related Art In a document image recognition apparatus, a layout analysis process is performed on a digital image obtained by inputting a document by image scanning or the like, and a character recognition process is performed on each extracted character image. The process flow of converting to a character code is taken. Here, the layout analysis refers to a process from extracting a character line from an input image to extracting characters.

As a document relating to a layout analysis method that has been proposed so far, a document recognition system using high-precision character segmentation processing based on character structure information (Sun, et al., Transactions of Information Processing Society of Japan, Vol. 33, No. .99,
1992). This is a method for Japanese documents, and its outline will be described below. In this method, an input document image is subjected to binarization and inclination correction processing, and then subjected to layout analysis. Region division is performed, only the character region is extracted, and a black pixel connected component is extracted by performing labeling processing on the character region. The obtained block of black pixels is called a candidate figure. Since there are many separated characters for "north" and "c" (characters composed of multiple connected components) and many contact characters that make up one connected component by touching them, Processing for generating a figure is required. The most frequent correction graphic size is taken as the average character size. The candidate figures are projected on the horizontal axis, and character lines are extracted from the distribution. Merge candidate shapes in a column,
Deal with separators. Candidate figures with a size close to the average size are preferentially cut out, and the remaining figures are forcibly integrated for separated characters, separated and extracted for punctuation, and final punctuation results Get.

In another layout analysis method,
As an example for English documents, TSUJIMOTO
"Major Components of
aComplete Text Reading S
ystem "(TUJIMOTO et al., Procedi
ngs of the IEEE, Vol. 80, N
o. 7, 1992) will be described. A document image is represented by run length, (1) connected components of black pixels are extracted, and adjacent connected components at a relatively short distance of about 1 mm are integrated and segmented. (2) Classify each segment into text lines, figures, pictures, etc. (3)
Combine adjacent text segments into segments. This extracts a block of text. Here, a word can be extracted by the processing of (1) and (2), and a text line can be extracted by the processing of (3).

According to these methods, when character lines having different numbers of points exist in the same document in close proximity, they cannot be correctly extracted. In the layout analysis process, the document composition rule that the space between columns is larger than the space between lines, the space between lines is larger than the space between characters, and the space between characters is larger than the space within characters is established in the input document image. The layout analysis of the document image is performed on the premise that the document image is processed. At this time, for example, as shown in FIG. 10, if there is a close proximity to a character interval having a different number of points, the line interval of a text line having a small number of points becomes smaller than a character interval of a character having a large number of points, and the number of points having a large number of adjacent points becomes large. Characters are integrated into text lines with a small number of points, making it impossible to extract correct lines and characters.

As described above, when characters having different numbers of points are mixed in the same document, layout analysis is difficult.

[0007]

In the conventional layout analysis processing, the space between columns is larger than the space between lines, the space between lines is larger than the space between characters, and the space between characters is larger than the space within a character. Only if the typesetting rules of
The layout analysis of the document image worked normally. Such a rule is not always satisfied in a document in which characters having different numbers of points are mixed.

An object of the present invention is to realize a high-precision layout analysis performance for a document in which characters having different numbers of points are mixed in the same document, which is difficult with the conventional method.

[0009]

The present invention provides a document image input means, a means for extracting a connected component from the document image, and a means for classifying the connected component according to size information. Means for respectively generating partial images containing only connected components belonging to the class, and for each of the partial images,
It is characterized by comprising layout analysis means for performing layout analysis and extracting layout information, and means for synthesizing the respective layout information.

[0010]

The connected components are classified according to the size information, and partial images containing only the connected components belonging to the obtained class are generated, whereby characters having different numbers of points are allocated to the respective partial images. A layout analysis is performed on each partial image by a conventional method. Layout of each partial image
Synthesis of layout information obtained from the results of
Then, a final layout analysis result is obtained. When inconsistency occurs between layout information when synthesizing layout information, priority is given to the layout analysis result of a class that has a large number of connected components in each partial image and includes connected components with a character-like size. To synthesize.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the embodiment.
The document image input through the image input processing is subjected to a binarization process and is converted into a scanning signal of the image. FIG. 3 is an example of an image obtained by performing a binarization process after scanning an input document image. This input document contains, in addition to the text corresponding to the main text that occupies most of the title text, four components of different sizes: large text corresponding to the title, small ruby text, and granular noise. Things.

Each connected component of black pixels is extracted by the connected component extracting means 11 to obtain circumscribed rectangle information 101 of the connected component. Here, a black pixel corresponds to an element part constituting a document such as a character or a figure, and a white pixel corresponds to a background part. Next, when the two circumscribed rectangles have an inclusion relationship, or when the degree of overlap of the circumscribed rectangles is larger than a certain degree, the two rectangles are integrated. Here, the degree of overlap is represented by, for example, the ratio of the area of the overlapping portion to the sum of the areas of two overlapping rectangles. FIG. 4 shows the result of performing integration of rectangles having an inclusion relationship and integration using the degree of overlap with the circumscribed rectangle of the connected component.

The connected component classifying means 12 classifies each connected component into classes according to the size of a circumscribed rectangle of the connected component extracted and integrated by the connected component extracting means 11, and classifies the connected components into class information. Output. Here, the class is a set of connected components having similar sizes. For example, a class C ₃ corresponding to a heading character and a class C corresponding to a main character
_2, around here ruby and characters, making the class C ₁ corresponding to punctuation, four kinds of class C ₀ corresponding to the small noise is considered. As a criterion for classification, the height, width, area, contour length, and the like of the circumscribed rectangle can be considered. Here, as an example, classification into classes is performed using the area of the rectangle. Specifically, the area of each rectangle is determined, and for example, K-m
Each of the rectangles is classified into classes C ₀ , C ₁ , C ₂ , C ₃ ,... by clustering using the eans method (for example, K = 4).

Based on this, the partial image generating means 13 allocates connected components belonging to each class to each of the partial images, and generates partial images 103 to 104 for each class. For example, a partial image including only connected components belonging to the class C _i is defined as P _i (i = 1, 2, 3). FIG.
(D) is the result of assigning connected components to each partial image according to the classification information into the class of the circumscribed rectangle. In this example, the partial image P ₀ is the noise, the partial image P ₁ is the letter characters including separation in ruby and characters remained separate without being integrated process during rectangular integrated portion, the partial image P ₂ is present minute partial character, the partial image P ₃ is equivalent to the character of the title portion.

The converted partial images are subjected to layout analysis processing by the corresponding layout analysis means 14 to 15, respectively.
Here, the partial image layout analysis means uses the method of the grandchildren described in the prior art, Tujimot, since a rectangle having a substantially equal rectangular area has already been assigned to each partial image.
It is possible to carry out using the method of o et al. Here, as an example, the method proposed by Tsuji entitled “Structural Analysis of Page Image Based on Split Detection Method” in 1985, in the Report of the Technical Research Report on Electronics and Communication Science in Pattern Recognition and Learning PRL85-17 is applied to partial images of each class. It shall be applied in common. The layout analysis method using this method obtains a projection pattern in the horizontal and vertical directions in each partial image, cuts out lines in consideration of the periodicity of line arrangement, and assumes that characters are substantially square. Estimate character pitch from line width. Using the estimated character pitch, characters are cut out from the character line at substantially equal intervals in consideration of some fluctuations. By using such a conventional technique, characters scattered in a positional relationship
Characters and lines can be extracted in consideration of the periodicity of character arrangement. FIGS. 6A to 6D show the results of extracting character lines and characters from the partial images of each class when inputting the same input image example.

Layout information 105 to 10 for each partial image
6 is synthesized by the layout information synthesizing means 16 and outputs a final layout analysis result for all the documents.
An embodiment relating to the layout information synthesizing means 16 will be described with reference to FIG.

First, connected component count histogram generating means 2
Based on 0, a connected component count histogram 200 which is a histogram of the number of rectangles belonging to each class is obtained, and the number of connected components included in each class is stored. FIG. 7 is a connected component count histogram when the input image example of FIG. 3 is input.

When the layout information selecting means 22 synthesizes the layout information, if one connected component causes inconsistency in a plurality of layout information as a result of simple superposition of the layout information of each partial image, A priority request signal 202 is issued to the degree determining means 21.

The layout information selecting means 22 synthesizes each layout information. At this time, when an inconsistent portion of the layout information is found, a priority request signal 202 is generated and sent to the priority determination means 21. Next, according to the priority signal 201 obtained from the priority determining means 21, layout information of a class having a higher priority is selected.
This is performed for inconsistencies regarding all layout information, and a layout information synthesis result 107 is output.

The priority determining means 21 outputs the priority request signal 20
2 is received, the connected component count histogram 200 is referred to, the frequencies in a plurality of classes corresponding to the inconsistent parts in the layout information are compared, and the class having the maximum frequency is subjected to layout information synthesis. Priority is given. Also, at this time, if the size of the circumscribed rectangle of the connected component included in the class is small and most of the connected components included in the partial image are regarded as noise, or the size of the connected component included in the partial image is large No priority is given when many are considered to be figures, tables, etc.
The result priority signal 201 is returned to the layout information selecting means 22.

FIG. 8 shows the result of simply superimposing the layout information of the partial images of each class. Horizontal strokes of characters "row" in the same figure, the right side of "ga", each connected component of the "ability", in, although inconsistent with respect to the layout information has occurred,
Referring to the frequency of the number of characters belonging to the partial images P ₁ and P ₂ involved in the contradiction, the frequency of the number of characters belonging to the partial image P ₂ is high. In this part, the priority is added to the layout analysis result of the partial image P ₂ , And is prioritized in layout information synthesis. The connecting components belonging to the partial image P ₀ is the priority is not given because it is considered rectangle size contains a lot of small noise components. Therefore, at the time of layout information synthesis, connected components other than those overlapping with the layout information of other partial images are deleted. FIG. 9 shows a result of synthesizing layout information, and a correct layout analysis result is obtained regardless of a document in which characters having different numbers of points are mixed.

With the above-described method, a layout analysis result for the entire document can be obtained without being swayed by the character size.

[0024]

For example, in a document in which lines composed of characters having different numbers of points are close to each other as shown in FIG. 10, a partial image is generated for each size of a connected component, and a layout analysis process is performed for each partial image of each class. , A correct analysis result can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a layout analysis method according to an embodiment of the present invention.

FIG. 2 is a block diagram relating to one embodiment of a layout information synthesizing unit in FIG. 1;

FIG. 3 is an example of an input document image.

FIG. 4 shows a result of extracting a circumscribed rectangle of a connected component extracted by a connected component extracting unit when the document image of FIG. 2 is input.

FIG. 5 shows a result of generating a partial image by allocating the connected components classified into each class by the partial image generating means to each partial image with respect to the same input image. (A) Partial image P
₀ , (b) partial image P ₁ , (c) partial image P ₂ , (d)
Partial image P ₃ .

Against 6 the input image, the layout analysis result of the layout analysis result (a) partial images P ₀ for each partial image, (b) partial layout analysis result of the image P _1, (c)
Layout analysis result of the partial image P _2, (d) partial image P
Layout analysis result of ₃ .

FIG. 7 is a connected component histogram obtained by counting the number of connected components classified into classes by the connected component classifying unit when the document image of FIG. 2 is input.

FIG. 8 shows a result obtained by superimposing layout analysis result information of partial images of each class on the input image by a layout information combining unit.

9 shows a result of integrating the layout information of the input image with the layout information superimposition result of FIG. 7 by giving priority to layout information of a class having a higher priority.

FIG. 10 is a conceptual diagram illustrating an effect when the present technique is applied to a document image example including characters having different point numbers in the same line. (A) an original image, (b) a partial image of a class in which a character with a large number of points is extracted, and (c) a partial image of a class in which a character with a small number of points is extracted.

[Explanation of symbols]

 REFERENCE SIGNS LIST 11 connected component extraction means 12 connected component classification means 13 partial image generation means 14 to 15 layout analysis means 16 layout information synthesis means 20 connected component count histogram generation means 21 priority determination means 22 layout information selection means 101 circumscribed rectangle information 102 Classification information of connected components into classes 103 to 104 Image planes 105 to 106 Image plane layout information 107 Layout information synthesis result

Claims (2)

(57) [Claims] 1. A document image is input as a binary image signal,
Connected component extracting means for extracting connected components, connected component classifying means for classifying the connected components according to size information, and only connected components belonging to a class obtained by performing classification according to size information. A partial image generating means for generating and storing each of the included partial images, a layout analyzing means for performing layout analysis on the partial images and extracting layout information, and a layout information synthesizing means for synthesizing the respective layout information Means for analyzing a document image layout. 2. The document image layout analysis device according to claim 1, wherein the layout information synthesizing unit measures a frequency distribution of a connected component for each class from layout information output by each of the layout analyzing units, and When inconsistency arises between the connected component count histogram generation means that is stored as a number histogram and the layout information output by each of the layout analysis means, the priority when synthesizing the layout information with reference to the connected component count histogram is determined. Priority determining means for determining, prioritizing layout information according to the priority, synthesizing layout information,
A document image layout analysis device, comprising: layout information selection means for outputting final layout information.