KR20010104164A - A method for automatically analyzing document layout - Google Patents

Abstract

A method is disclosed for automatically analyzing and storing a document regardless of typeface size, line spacing, or document structure without the need for additional processing such as additional parameter input. An object of the present invention is to provide an automatic document structure analysis method capable of dividing a document image having various typefaces, lines, and document structures irrespective of parameters. The present invention divides an input document image into maximum homogeneous regions, and divides each region. In order to accurately classify text, pictures, tables, and lines, it is hierarchized into pyramid structures and analyzed in multiple stages.Methodity is measured to determine which areas are divided, and texture-based analysis is performed on areas where periodicity is difficult to find. Documents with various typographical sizes, lines, and document structures can be analyzed automatically and accurately regardless of parameters.

Description

A METHOD FOR AUTOMATICALLY ANALYZING DOCUMENT LAYOUT}

The present invention relates to an automatic document structure analysis method, and more particularly, to a method for accurately analyzing and storing a document regardless of typeface size, line spacing, and document structure through multi-step analysis, periodicity measurement, and texture analysis. .

Due to the development of information technology and industrial informatization, the usage of documents containing information is steadily increasing, and the documents containing such information are continuously published in newspapers, magazines, reports, books, etc., because the paper documents are easy to read. .

However, if the document to be published is large, the cost of publishing and storage space are increased. Therefore, if a large paper document can be automatically converted into an electronic document, content retrieval is not only easy, but also the storage space is drastically reduced. Can be.

However, the process of automatically converting a paper document into an electronic document is not an easy task, and in order to perform such automatic document conversion, document structure analysis must be preceded.

Representative methods of document structure analysis and document segmentation are the bottom-up approach and the overall information of the document, which is a method of composing words starting with local information such as black pixels or connecting elements and continuously combining words into lines and lines into paragraphs. A top-down approach is to split the document into columns, split the columns into blocks, the blocks into rows, and the words into words.

In the bottom-up approach, a connection element extension method of gradually extracting a connection element from an input document image and then gradually combining elements of the same type is widely used.

The connection element expansion method generally has a large time complexity for classifying, analyzing, and extending the connection elements, and especially when there are many connection elements such as newspapers, the time complexity increases exponentially.

In order to reduce this time complexity, a method of layering a document structure with a least cost tree and adding a heuristic teaching method to the connection element extension method has recently been proposed. However, this method can also introduce segmentation errors due to early erroneous expansion.

On the other hand, the top-down approach is less time-consuming than the bottom-up approach and is similar to what a person sees from a lower resolution to a higher resolution, but it becomes more difficult to split a document with a complex document structure or various font sizes that cannot be divided into rectangles. It has a disadvantage.

Also, among many methods related to geometric document structure analysis or document segmentation, the texture-based method that performs document segmentation by finding the same textured region in the document image by considering the regions such as text, pictures, and graphics in the document image as a textured region The disadvantage is that the time complexity is very large. In other words, to find the characteristics of the texture area, a number of masks are required to extract local information because various filters are needed for the desired local spatial frequency and direction. Textured areas cannot be detected, and conversely, increasing the size of the mask increases the time complexity.

Therefore, in order to make up for the drawbacks of this texture-based method, multi-step analysis is applied to extract local features at each resolution using only two filters. In this case, it takes a lot of time to extract the multi-level texture information. Different types of regions but with similar textures often cause confusion or mixing problems.

In addition to the methods mentioned above, Antonacopoulos used white space to extract region outlines, and Jain and Yu used a typical bottom-up method for document segmentation and region classification, and a document model with top-down information for logical structure analysis. . However, these methods have a problem in that the setting of several thresholds and parameters is inevitable because performance depends on the thresholds and parameters.

Accordingly, the present invention was created to compensate for the above disadvantages, and an object of the present invention is to provide a method for measuring periodicity of a document's periodic characteristics and a variety of typefaces through texture-based analysis of regions in which periodic characteristics cannot be extracted. The purpose of the present invention is to provide an automatic document structure analysis method capable of dividing a document image with line and document structure irrespective of parameters.

In the automatic document structure analysis method of the present invention for achieving the above object, a) determine whether the document to be analyzed is input, and when it is determined that the document is input, converts the input document from low resolution to high resolution to pyramid Generating a structure and extracting a multi-level image of the document; b) In the step a), the top-level image of the low-resolution pyramid structure is analyzed by connecting element analysis to extract the boundary rectangle of the document, and the periodicity of a predetermined region is measured in the horizontal and vertical directions of the remaining images except for the top-level image. Extracting periodic characteristics; c) determining whether the predetermined region is a region having a single period in step b), if it is determined that the region does not have a single periodicity, detecting a position to divide a document, dividing the document into a plurality of regions, and then proceeding to step b) ; d) if it is determined that the period measured in step c) has a single period, classifying the maximum homogeneous region into text, picture, table, and line; And e) supplying the text classified in step d) to a character recognizer, compressing a picture, regenerating a table, and vectorizing a graphic such as a line and storing it in a database.

According to the present invention, a periodicity measurement method for layering a document to be analyzed into a paramid structure, starting at a low resolution and performing a high resolution, and extracting periodic characteristics of a specific region is proposed. By dividing a document into regions having a predetermined area, detecting the periodicity of the divided predetermined region, and classifying and storing the maximum homogeneous region into text, pictures, tables, and lines, the structure of the document is precisely irrespective of typeface size, line size, and document structure. Can be analyzed automatically.

1 is a diagram showing the configuration of a document input device to which the present invention is applied.

2 is a flowchart illustrating an automatic document analysis and storage process according to the present invention.

3 is a diagram illustrating a process for measuring periodicity according to the present invention.

4 is a diagram illustrating one example of dividing a document to which the present invention is applied in a horizontal direction.

<Explanation of symbols for main parts of drawing>

10: document input unit 20: processor

30: database 40: display unit

Hereinafter, the present invention will be described in more detail with reference to the following examples.

1 is a block diagram showing a configuration of a document input device to which the present invention is applied, and analyzes the document input unit 10 for inputting a document and the input document and classifies it into a text area, a graphic area, a picture area, and a table area. A processor 20 operable to input each classified area into a text recognizer using ASCII code, compress an image area, regenerate a table area, and vectorize a graphic area such as a line; A database 30 storing the output information of the processor 20 and a display unit 40 for displaying a document stored in the database 30 according to a user's request.

2 is a flowchart illustrating a document analysis and storage process of a geometric structure according to the present invention.

The operation of the flowchart shown in FIG. 2 will be described in detail with reference to FIG. 1.

The document input through the document input unit 10 is supplied to the processor 20, and the processor 20 receiving the document to be analyzed checks whether the document is input (step S11), and forms the document in a pyramid structure. In step S12, the multi-step image is extracted.

In the step S12, first, the image is repeatedly reduced by 1/2, vertical 1/2, or 1/4 at a time until the resolution of the input image is smaller than 50 X 50 pixels. Extract the image.

Here, each S ⁽ⁿ⁾ pixel at the n (integer) level corresponds to four pixels S _i ^(n-1) (i = 1,2,3,4) at a more detailed n-1 level, S _i ^(n-1) pixels become child pixels of S ⁽ⁿ⁾ , and if one or more of the four pixels S _i ^(n-1) (i = 1,2,3,4) are black pixels, then S ^{(n) which} is the parent pixel of ^is also assigned to the black pixel. The equation for this is expressed as the following equation when the black pixel is 1 and the white pixel is 0.

Therefore, the number of pixels in S ⁽ⁿ⁾ is 1/4 of the number of pixels in _Si ^(n-1) . In this way, a pyramid structure of the input document is generated to output a multi-level image.

After the output of the multi-stage image is completed in step S12, the processor 20 calculates the inclination angle of the document to extract the periodicity for the text area, the picture area, the table area, and the graphic area from the multi-stage image. Proceeding to step S13 of generating a post-projection histogram.

That is, the inclination angle [theta] of the document is obtained, and the projection histogram in the horizontal (or vertical) direction is obtained using the following equation according to the direction of the inclination angle [theta].

Here, I (x, y) is a brightness value in an image whose width is width x height, and P _H ⁽ⁿ⁾ is a horizontal projection histogram at an nth level. FIG. 3B shows an example of a projection histogram of the inclined text area of (a).

After the generation of the projection histogram is completed, the processor 20 proceeds to leveling the projection histogram obtained from step S13 to remove noise components of the projection histogram.

That is, in the projection histogram of FIG. 3B, the noise component of the projection histogram is removed using the following equation.

Where s is the kernel size and m _y is an integer. FIG. 6 is a diagram illustrating a result of smoothing the projection histogram of FIG. 5.

After smoothing the projection histogram through step S14, the processor 20 proceeds to step S15 for calculating the slope between neighboring elements, and uses the following equation to calculate the slope between the neighboring elements.

FIG. 3D is a diagram showing the result of differentiating the smoothed projection histogram of FIG. 3C. After the calculation of the slope between the neighboring elements is completed through the above equation, the processor 20 proceeds to step S16 for calculating a sign intersection point at which the sign of the slope intersects, and in step S16, the sign of the slope crosses. The intersection is calculated using the following equation.

The sign intersection point calculated from the above equation corresponds to a local maximum value or a local minimum value of the projection histogram, where (e) of FIG. 3 represents a sign intersection point according to the derivative result of the smoothed projection histogram of FIG. It is shown.

When the calculation of the neighboring code intersections is completed by the step S16, the processor 20 calculates a variance of the distance between them from the calculated neighboring code intersections to calculate a frequency distribution for a predetermined region of the document. Proceeding to step S17, the variance V of step S17 is calculated using the following equation.

Where to be.

The low variance of the distance to the neighboring sign intersection calculated from the above equation indicates that the frequency distribution of the rows in the region is almost identical, and thus the region is composed of homogeneous regions.

On the contrary, a high variance value of the distance to the neighboring code intersection point calculated from the above equation means that the frequency distribution of the rows in the area is irregular, and thus there is a part to be divided at a predetermined position of the area.

When the calculation of the variance of the distance to the neighboring code intersection point is completed from the step S17, the processor 20 normalizes the range of the variance value V to calculate the normal value P, and calculates the calculated normal Proceeding to step S18, from the value P, a determination value D (P) for determining whether the predetermined region has a single periodicity is calculated. Step S18 calculates the normal value P using the following equation.

Here, when the variance value V of the predetermined area goes to zero, the normal value P becomes 1, and if the variance value of the predetermined area goes to 1, the normal value P becomes zero. Therefore, the periodicity of the predetermined area is represented by the normal value P having a range of 0 to 1.

As described above, after the normal value P is calculated by the step S18, the determined value D (P) for determining whether the predetermined region has a single periodicity according to the calculated normal value P. Calculate

That is, the determination value D (P) for determining whether a predetermined area has a single period is determined by the following equation, where the threshold value TH is set to 0.5, and the threshold value TH is a variance value ( The value is when V) is 1.099.

If the determined value D (P) calculated from the above equation is 0, it means that the variance of the difference between the sign intersection points is high and not a homogeneous region, so that at least one division is required for the predetermined region. However, if the determination value D (P) is 1, it means that the variance of the difference between sign intersections is low and homogeneous, so that no division for a predetermined region is necessary.

When the calculation of the determination value D (P) is completed in step S18, the processor 20 proceeds to step S19 of checking whether a predetermined region has a single period according to the determination value D (P). .

If it is determined in step S19 that the document should be divided according to the determination value D (P), the process proceeds to step S20 of dividing the document in a horizontal or vertical direction, and step S20 is shown in FIG. As shown in FIG. 4 (a), when the document is horizontally projected, the white space is larger than other white spaces, and the black space is divided when the document is horizontally projected. As shown in FIG. Since the space is larger than other black spaces, it is classified as the case where the white space next to it is to be divided. For these two cases, the division position is found by the following method and divided into two areas.

If the white space is larger than the other white spaces when the document is horizontally projected, and there is a need to divide them, the set of white spaces arranged in the order of smallest to largest W, W _{i and} middle elements W _med When the maximum element is W _max , and W _i &gt; W _med and W _i = W _max , the element is divided into any element W _i .

In the case of a black area when the horizontal projection of a document to split the white space of the cursor next to it than the other black area has an arbitrary element of the set of black space aligned in order of size B B _i, the intermediate elements, B _med When the maximum element is referred to as B _max , and B _i > B _med and B _i = B _max , it is divided into any of the above elements W _i-1 .

After the document is divided through the above process, the process proceeds to step S13 of calculating periodicity for the divided document area.

However, like other conventional methods, the document splitting method through the above process also includes an area in which the input document touches or overlaps two areas, a paragraph starting with a large letter, a few words and a single line. In the case of the periodicity measurement, it is difficult to divide or classify.

Therefore, in the present invention, after dividing a document input through the above processes into maximum homogeneous regions, the processor 20 adds a step (S21) of confirming the division result of the document. In step S21 for confirming the result of document division, the determination value D (P) calculated in step S18 is 1, and it is determined whether the number of sign intersections of the horizontal projection histogram is less than or equal to three. If the determined value D (P) is 1 and the number of sign intersections of the horizontal projection histogram is less than or equal to 3, the processor 20 determines that the periodicity measurement result is unreliable and is textured. The analysis proceeds to step S22. In the case of the texture based analysis, a process of using a conventional 2D Harr wavelet and performing texture analysis is as follows.

First, the wavelet transform is calculated by a pyramid algorithm similar to the one-dimensional, and the two-dimensional wavelet transform is a one-dimensional wavelet transform applied in the x- and y-axis directions, and the original image A _{n + 1} at every level. Decompose f into A _n f, D _n ¹ f, D _n ² f, and D _n ³ f.

That is, first every second row of A _{n + 1} f is convolved with a one-dimensional filter, and every second column of results is convolved with another one-dimensional filter. As illustrated in FIG. 13, the document input through the above process may be converted into four sub-images called LL, LH, HL, and HH. Here, the LL lower image is the same as the result of A _n f in which the horizontal axis and the vertical axis are the low frequency parts, and the LH sub image is the low frequency part and the vertical axis is the high frequency part, and the same as the D _n ¹ f result.

In addition, the HL lower image indicates a portion where the horizontal axis is a high frequency and the vertical axis is a low frequency, as shown in the result of D _n ² f, and the HH lower image shows a portion where both the horizontal axis and the vertical axis are in high frequency, and is equal to the D _n ³ f result. .

In the case of the 2D Harr wavelet of the binary image, LL ⁽ⁿ⁾ , LH ⁽ⁿ⁾ , HL ⁽ⁿ⁾ , and HH ⁽ⁿ corresponding to A _n f, D _n ¹ f, D _n ² f, and D _n ³ f ⁾ . ⁾ Sub images are computed from the following equation.

According to the conversion coefficient calculated from the above equation, it is classified into picture, text, horizontal line, and vertical line. For example, if the coefficient of the LH sub-image is high and the coefficient of the HL sub-image is low, it means that there is only a horizontal component and thus is assigned to the horizontal line (H). On the contrary, if the coefficient of the LH sub-image is low and the coefficient of the HL sub-image is high, it means that there is only a vertical component and thus is assigned to the vertical line (V).

If both LH and HL subpictures are high, this means that there are horizontal and vertical components, so assign them as text (T) .If the coefficients of both LH and HL subpictures are low, the picture or background has no horizontal and vertical components. If the coefficient of the LL lower image is high, it is allocated to the picture (I), and if the LL lower image coefficient is low, it is assigned to the background (X). The above contents are shown in the following table.

LH LOW (0) HIGH (1) HL LOW (0) I / X H HIGH (1) V T

Second, the convolution of all classes is performed as a mask to determine the allocation result of neighbor coefficients, and the mask of each class representing the relationship between neighbors is shown in the following table.

The result of convolving corresponding coefficients at the (x, y) point of the nth level is calculated from the following equation.

Here, m _i ^c is the i th element of the mask of the corresponding class, b _i is the class assigned to b _i is class c, Equals 1, otherwise 0.

Third, the convoluted result of each level is accumulated using the following equation, and the cumulative process as described above is applied from the highest level (N level) to the lowest level (1) and the final value of the coefficient c corresponding to each class. The cumulative value A ¹ _{x, y} (c) is calculated.

Finally, at the lowest level, level 1, the class of the corresponding coefficient is determined as a picture, text, horizontal line, or vertical line. The determination method is the picture (I), the text (T), the vertical line (V), and the horizontal line (H). The class of the coefficient c corresponding to the maximum value of the final cumulative value of is determined.

In step S13, the non-text area (picture, horizontal line, vertical line) and the small area of the text area are removed from the result of document analysis through the texture-based analysis.

On the other hand, if it is determined in step S21 that the determined value D (P) is 1 and the number of sign intersections of the horizontal projection histogram is not less than or equal to 3, the processor 20 determines that the document has been correctly analyzed. In step S23, the document is classified into a text area, a line area, a table area, and a picture area, respectively.

For example, through the above process, one or more peaks appear in the projection histogram having a periodic component in the horizontal direction and smoothed in the horizontal direction, and the determined value D (P) in the step S19 is 1, If the number of sign intersections of the horizontally smoothed projection histogram in step S16 is three or more, the area is classified as a text area.

If it has a periodic characteristic in the longitudinal direction and at least one peak in the projection histogram smoothed in the longitudinal direction, the determined value D (P) in step S19 is 1, and the horizontal smoothing in step S16. If the number of sign intersections in a projection histogram is less than or equal to 3, the area is classified as a text area consisting of one line, such as a title, subtitle, legend, or caption.

Characters of the classified paragraph text area are supplied to a character recognizer and stored in the database 30.

On the other hand, after connecting the broken line, the area is composed of one connecting element, and if the ratio of horizontal and vertical is 5 times or more, it is classified as a line, and this classified line is vectorized into a graphic area to the database 30. Stored.

Then, horizontal lines are extracted for the unclassified area, and upper and lower lines having a size similar to the area of the predetermined area are detected, and when one or more horizontal lines except for the upper and lower lines are detected, they are classified into a table. The sorted table is regenerated and stored in the database 30.

All remaining areas not classified above are classified as picture areas, and the classified picture areas are compressed and stored in the database 30.

As described above, according to the present invention, the input document image is divided into maximum homogeneous regions, and a pyramid structure is formed to accurately classify each region into text, pictures, tables, and lines, and periodicity measurement and texture-based analysis are performed. By applying it, it is possible to automatically divide a document having various typeface sizes, lines, and document structures automatically without the need for an additional process such as additional parameter input.

In addition, although the present invention has been described in detail with reference to the embodiments described above, the present invention is not limited thereto, and modifications and improvements are possible within the scope of ordinary knowledge of those skilled in the art.

Claims (8)

a) determining whether a document to be analyzed is input, and when it is determined that the document is input, processing the input document at a high resolution at a low resolution to generate a pyramid structure, and extracting a multi-stage image of the document; b) In step a), a connection element analysis is performed on the top-level image of the low resolution pyramid structure to extract the boundary rectangle of the document, and the periodicity of a predetermined region is measured in the horizontal and vertical directions of the remaining images except for the top-level image. Extracting periodic characteristics; c) determining whether a predetermined area is a single period in step b), and if it is determined that the predetermined area does not have a single periodicity, detecting a position to divide a document, dividing the document into a plurality of areas, and then proceeding to step b). ; d) if it is determined that the period measured in step c) has a single period, classifying the maximum homogeneous region into text, picture, table, and line; And e) supplying the text classified in step d) to a character recognizer, compressing a picture, regenerating a table, and vectorizing a graphic such as a line and storing the text in a database. The method of claim 1, wherein b) b-1) obtaining an inclined angle of the document and generating a projection histogram according to the direction of the angle; b-2) smoothing the projection histogram generated in step b-1) to remove noise components; b-3) calculating a slope between neighboring elements by differentiating the smoothed projection histogram in step b-2); b-4) calculating the sign intersection of the projection histogram from the slope between the neighboring elements; b-5) calculating a frequency distribution in a predetermined region by calculating a variance of the distance between sign intersections of the projection histogram calculated in step b-4); And b-6) calculating a normal value by normalizing the variance in step b-5), and calculating a determination value for determining the periodicity of the predetermined region from the calculated normal value. Analytical Method. The method of claim 2, wherein b-6) b-iii) calculating a normal value P and outputting a normal value P by calculating Equation 1 below; And b-ii) calculating the determination value D (P) for determining a single periodicity of a predetermined area according to the normal value by calculating Equation 2 below. .. Equation 1 .. Equation 2 The method of claim 1, wherein the step c) when said arbitrary element of the set W of the diluent space aligned in order of size W _i, the intermediate element W _med, and the maximum element of W _max, W _i> W _med And W _i = W _max , the document is divided by the arbitrary element W _i . The method of claim 1, wherein the step c) is B _i > B _med when any element of the set B of the black spaces arranged in size is B _i , the middle element B _med , and the maximum element B _max . And B _i = B _max , the document is divided by the arbitrary element W _i-1 . a) determining whether a document to be analyzed is input, and when it is determined that the document is input, processing the input document at a high resolution at a low resolution to generate a pyramid structure, and extracting a multi-stage image of the document; b) In step a), a connection element analysis is performed on the top-level image of the low resolution pyramid structure to extract the boundary rectangle of the document, and the periodicity of a predetermined region is measured in the horizontal and vertical directions of the remaining images except for the top-level image. Extracting periodic characteristics; c) determining whether a predetermined region is a region having a single period in step b), detecting a position to divide a document when the determination is not made with a single periodicity, dividing the document into a plurality of regions, and then proceeding to step b). ; d) if it is determined in step c) that the predetermined area has a single period, verifying the result of document division; e) analyzing the document according to the texture mood analysis only for the corresponding area when it is determined that the document division result is incorrect in step d); f) classifying the largest homogeneous region into text, picture, table, and line when the document division result is determined to be accurate in step e) or the document division result is determined to be accurate in step d); And g) supplying the text classified in step d) to a character recognizer, compressing a picture, regenerating a table, and vectorizing a graphic such as a line and storing it in a database. 7. The automatic document structure analysis method according to claim 6, wherein the step c) determines that a predetermined area of the document does not have a single period when the determination value is zero. 7. The automatic document structure analysis method according to claim 6, wherein the step d) determines that the document analysis result is incorrect when the determination value is 1 and the code intersection point is 3 or less.