KR100331035B1 - Automatic Analysis of Format Document Image - Google Patents

Abstract

The present invention relates to a method for automatically interpreting a form document image, and more particularly, by receiving a form document of an existing large office made of paper through an input medium such as a scanner or a camera, and converting the form document into an electronic form document. The purpose of the present invention is to provide an effective method for automatically analyzing the format document image, which enables the automatic processing of the format document having a complicated structure.

According to the present invention, in the automatic interpretation method of a format document image for converting an arbitrary format document image filled with contents into an electronic form document, an expression model for expressing an input format document image is analyzed by analyzing the structure of the format document image. First step; A second step of comparing the representation model for the input format with a standard model already registered; A third step of adding an item to be processed for the new form to the item definition database after registering as a new model if there is no matching model as a result of the comparison of the second step; And a fourth step of extracting an item area to be processed from the classified input form document image and converting it into an electronic form document after classifying the matching model if there is a matching model as a result of the comparison in the second step. An automatic interpretation method of the form document image is provided.

Description

Automatic Analysis of Format Document Image

The present invention relates to a method for automatically interpreting a form document image, and more particularly, by receiving a form document of an existing large office made of paper through an input medium such as a scanner or a camera, and converting the form document into an electronic form document. The present invention relates to an effective method for automatically interpreting a format document image, which can automatically process a format document of a complex structure.

Format documents are formatted in the same format, even if there is a variation in the physical format in which the position and size of each item is strictly determined, and the position and size of the item maintain its topology and content. Can be divided into logical formats.

When processing a form document in a physical format, only a portion of the form may be removed from the scanned document image, and then the corresponding portion may be processed using OCR. However, this method requires that prior knowledge of all forms to be interpreted is registered as a form model. However, since most documents used in an office are modified by each user to be re-edited or resized according to a predetermined format, in order to use the conventional method, it was necessary to register several form models for one form. For this reason, the conventional method of interpreting the format document in physical format is not effective for automatically processing the scanned form document image.

The present invention has been made to solve the problems of the prior art as described above, after analyzing the structure of the form to generate an expression model, and compares it with the existing standard model to perform the classification of the format by logical form, Character separation and character recognition processing from the item area to be extracted from the generated form, and the result can be automatically converted into an electronic document that can be reused and edited to automatically process the format contained in the document image. The purpose is to provide an automatic interpretation method of the format document image.

1 is a flowchart of an automatic interpretation method of a form document image according to an embodiment of the present invention.

FIG. 2 is a flowchart of a process of generating an expression model of a format document image of FIG. 1;

3 is a flowchart of a classification and registration process of the form document image of FIG. 1.

4 is an example of a form document used for generating a representation model of a form according to the present invention;

5A is an example of a form document that may be used to classify and register a form according to the present invention in accordance with the present invention,

FIG. 5B is an execution example of registering a form document as shown in FIG. 5A;

FIG. 6 illustrates an example of extracting an item from a form document as shown in FIG. 5A.

FIG. 7 is an example of separating characters in an arbitrary item in a format document as shown in FIG. 5A;

FIG. 8 illustrates an embodiment in which a result of recognizing a format document as illustrated in FIG. 5A is converted into an HTML file.

9A-9D are diagrams showing the results of analysis of any form applied to the present invention and the method using the method according to the present invention.

According to the present invention for achieving the object as described above, in the automatic interpretation method of the form document image for converting any form document image filled with the electronic form document, the structure of the form document image by analyzing the input form Creating a representation model representing the document image; A second step of comparing the representation model for the input format with a standard model already registered; A third step of adding an item to be processed for the new form to the item definition database after registering as a new model if there is no matching model as a result of the comparison of the second step; And a fourth step of extracting an item area to be processed from the classified input form document image and converting it into an electronic form document after classifying the matching model if there is a matching model as a result of the comparison in the second step. An automatic interpretation method of the form document image is provided.

More preferably, the first step may include: a fifth step of extracting line segments of the form with respect to the scanned form document image; A sixth step of extracting a rectangle surrounding the item of the form from the extracted vertical and horizontal segments; And generating a representation model of the input format image by information on the relationship between the extracted length, height, position, and segment of the vertical and horizontal segments and information on the upper, lower, left, and right edges forming the extracted visual form. There is provided a method for automatically analyzing a formatted document image including the seventh step.

Also preferably, the fifth step may include: an eighth step of extracting connected components forming horizontal and vertical segments by mathematical morphology calculations; A ninth step of filtering the connection elements that are not suitable to be regarded as lines of the form by calculating the width and the slope of the line segments for the extracted connection elements; And align all horizontal connectors from top to bottom, and then align all vertical connectors from left to right to group adjacent horizontal or vertical connectors into one line to extract the segment of the form. An automatic analysis method of a format document image including the tenth step is provided.

More preferably, the sixth step may include: an eighth step of excluding the candidate rectangle if the sum of the rectangles covers the entire candidate rectangle; Extracting candidate rectangles as formatting items when edges of other rectangles cover the entire edge of one candidate rectangle; And a tenth step of extracting the candidate rectangle as a form item when the edges of other rectangles cover a portion of one candidate rectangle and the length of the uncovered part is long enough. A method is provided.

More preferably, in the tenth step, the grouping of the horizontal or vertical connecting elements adjacent to each other in a single line may include a distance d _ij if the two connecting elements i and j are on the same side as the standard line segment. Is | d _i -d _j | and if two segments i and j are on the other side of the standard line, the distance d _ij is calculated as | d _i + d _j | There is provided a method for automatically analyzing a format document image, wherein the connecting elements in the direction are grouped into one line.

More preferably, the second step may be performed by calculating the distance between the logical structure information of the form, the physical position and the size information from the currently registered form model, and determining whether the calculated distance is equal to or less than a predetermined value. There is provided a method for automatically analyzing a format document image, which is characterized by comparing the features.

In addition, according to the present invention, in the automatic interpretation method of a form document image for converting an arbitrary form document image filled with a content into an electronic form document on a computer, the input form document image is represented by analyzing the structure of the form document image. A first step of creating an expression model; A second step of comparing the representation model for the input format with a standard model already registered; A third step of adding an item to be processed for the new form to the item definition database after registering as a new model if there is no matching model as a result of the comparison of the second step; And as a result of the comparison of the second step, if there is a matched model, classify it into a matching format, and then perform a fourth step of extracting an item region to be processed from the classified input format document image and converting the area into an electronic format document. A recording medium for recording the program is provided.

This method does not process only the matching of images with the conventional method, but analyzes the structure of the format document and uses its physical coordinates and logical structure together. When the logical structure is used to match the format document, it is possible to appropriately cope with a change in the position and size of the format document, thereby solving the problem of having to repeatedly register the same form, such as when classifying the form into a physical form.

In addition, according to the present invention, it is possible to process the case where the lines of the form, which were the main cause of the item extraction error in the conventional method, are partially broken, and also when the handwritten characters and the lines of the form are in contact with each other. Complex forms can be processed as in the case of enclosing, so that the error rate of item extraction can be significantly lower than that of the conventional method.

Hereinafter, with reference to the accompanying drawings, a preferred embodiment of the automatic interpretation method of the format document image according to the present invention will be described in detail.

1 is a flowchart of a method for automatically analyzing a format document image according to an embodiment of the present invention, FIG. 2 is a flowchart of a process of generating a representation model of the form document image of FIG. 1, and FIG. 3 is a form of FIG. 1. A flowchart of the classification and registration process of document images.

As shown in the figure, in the method for automatically analyzing a form document image according to an embodiment of the present invention, when a form document image is input (101), a representation model of a form is generated (102).

In the process of generating the representation model of the form, first, as shown in FIG. 2, the line segment of the form is extracted (201), and surrounding the item of the form from the extracted vertical and horizontal segments. Extract the rectangle (202).

Subsequently, information on the length, height, position, and line segment of the extracted vertical and horizontal segments and information on upper, lower, left, and right edges forming the extracted visual form may be used. The representation model is generated (203) and ends.

Here, the line segment extraction 201 of the form first extracts the connected components forming horizontal and vertical segments by mathematical morphology calculation, and the width and the slope of the line segments with respect to the connected components. Calculate and filter out the connectors that are not suitable to be considered lines of the form, then align all horizontal connectors from top to bottom, and align all vertical connectors from left to right, Group the connecting elements in the vertical direction into one line.

And, in determining whether the current connecting element is close to the line segment of the current form, if two connecting elements i and j are on the same side as the standard line, the distance d _ij is | d _i -d _j |, and the two segments i and j If is on the other side of the standard line, the distance d _ij is | d _i + d _j |

Here, the standard line segment has the same slope as the form, and is a connecting element passing through the longest line segment of the form, and d _i and d _j represent a distance from the center of the standard line segment and each connecting element.

Next, a process 202 of extracting a rectangle of a form from vertical or horizontal line segments of the form extracted through the line segment extraction process 201 of the form will be described.

In order to determine whether the horizontal or vertical lines of the extracted form are valid edges of the rectangle surrounding the items of the form, processing is performed according to the following conditions.

First, if the sum of the rectangles covers an entire candidate rectangle, the candidate rectangle is excluded from consideration.

If the edges of the other rectangles cover the entire edge of one of the candidate rectangles, the candidate edges are valid, and thus the candidate rectangles are extracted.

In addition, since the edges of the other rectangles cover a portion of one edge of one candidate rectangle, and the edge of the candidate is valid when the length of the uncovered portion is long enough, the candidate rectangle is similarly extracted.

As described above, an expression model expressing a format is generated from information extracted through a structure analysis process of the format document image as follows. This representation model is (NH, NV, H, V, D _h , D _v ) for the line segments of the form and (NR, (X _min , Y _min ), (X _max , Y) for the rectangles of the form. _max ), (E _top , E _bottom , E _left , E _right )).

Where NH and NV are the number of extracted horizontal and vertical segments, H and V are matrices representing the relationship between horizontal and vertical lines, D _h and D _v are the lengths of the horizontal and vertical segments, and NR is The number of rectangles extracted, (X _min , Y _min ) and (X _max , Y _max ) are the position coordinates of the top left and bottom right corners of the outermost rectangle that surrounds the item in the form, and (E _top , E _bottom , E _left , E _right ) are the four edges of the top, bottom, left, and _right of the rectangle, indicating the number of horizontal or vertical lines.

On the other hand, after performing a process 102 of generating a representation model of the form, the process of classifying and registering the form is performed.

Next, a form classification and registration process 103 for matching and classifying an expression model for an input form document image generated in the process of generating a representation model of the form, with a standard model already registered will be described.

The classification and registration process 103 of the form document image is performed by matching the logical structure information, physical position and size information of the form obtained from the structure analysis process of the form document image with a model of the currently registered form. If it exists, the process proceeds to the proper form. If there is no matching form, it is a new form document. Therefore, the user receives the form name and information of the item to be processed and registers it as a new form model.

The result of the above process of the present invention has the advantage of utilizing the physical coordinates and the logical structure of the format document by analyzing the structure of the format document rather than the simple matching of the images as in the conventional method.

The use of the logical structure in matching can suitably cope with variations in the position and size of the form document, thereby solving the problem of repeatedly registering the same form, which is a disadvantage in classifying the form into a physical form.

Next, the process of classifying and registering the form shown in FIG. 3 will be described in detail.

As shown in FIG. 3, in order to classify the input format document image, the distance is calculated for the input format document image F for all the format model F _i , and the format model F _s closest to the distance is obtained (301).

Subsequently, it is determined whether the distance between the obtained form model F _s and the input form document image F is smaller than a threshold determined through experimentation (302), and if it is small, after classifying the input form document image into the form model F _s (304), If it is large, the input format document image F is registered as a new format model F (303) and then terminated.

On the other hand, after performing the classification and registration process 103 of the form, it is determined whether registration has been performed (104).

As a result of the determination, when a new form model registration process is performed, after inserting an item to be processed for the new form model in the item definition database (107), character separation and character recognition processes within the item are performed (106). If the registration process has not been performed, the information of each item to be processed in the form Fs is taken from the item definition database (105), and processed through a separate character separation and recognition process (106) suitable for the item.

The results processed through the above steps are converted into a file 108 for reuse and re-editing.

In the embodiment of the present invention, the file conversion format is HTML, and the final processing result of the input format image is expressed as an HTML file.

Then, the theoretical description of the automatic interpretation method of the format document image according to the present invention described above will now be described in detail through an exemplary document.

First, the information about the line segment of the form (NH, NV, H, V, D _h , D _V ) and the information about the rectangle of the form (NR, (X _min , Y _min ), (X _max , Y _max ), (E _top , E _bottom , E _left , E _right )) describes the procedure for creating an expression model that expresses styles.

Referring to Fig. 4, an example of a simple format document applied to the present invention is shown. Since the expression model F of the format as shown in FIG. 4 has 5 NHs and 3 NVs, the commands H and V representing the relationship between the horizontal line and the vertical line are as follows.

The values of D _h and D _v are the lengths of the horizontal component and the vertical component, for example, the value of D _{h'0 '} is the number of pixels between the horizontal line “0” and the horizontal line “1”.

In addition, the number of NRs is 7 (the number of squares), and (X _min , Y _min ) and (X _max , Y _max ) are used as the coordinates of the outermost rectangle of each rectangle in the item extraction step. On the other hand, (E _top, E _bottom, E _left, E _right) calculation of the value, for the case rectangular R4 for example, the top (top) edge of the square E _top is horizontal. 2, the floor (bottom) edge E _bottom is horizontal 4, Left edge E _left becomes vertical line 0, and right edge E _right becomes vertical line 1, resulting in (2,4,0,1).

Next, the procedure for classifying and registering a form will be described with reference to FIGS. 5A and 5B.

First, if there is a matching form document that matches the currently registered form document, the process proceeds to the proper processing for that form. If there is no matching form, it is a new form document. .

After obtaining the expression model of the arbitrarily drawn form (FIG. 5A) having the handwritten number as the content of the item, it is compared with the reference model to determine whether it is a model already registered or a new model. Register to the reference model through the registration window as shown.

Referring to FIG. 6, an execution example of extracting an item of a form based on information of each rectangle from an expression model F obtained through the step of generating an expression model of a form is illustrated with a brief description.

The processing screen of FIG. 6 finds a horizontal rectangle and a vertical line of the form to find a rectangle of the form, and the information of the rectangle (ie, the number of rectangles, the coordinate values of each rectangle, and the horizontal / horizontal line corresponding to the four edges of the rectangle). Minute number) to find the part corresponding to the item of the form shown by the small pop-up window and the square of FIG.

Next, referring to FIG. 7, an example of separating characters in items is illustrated. The order of each item extracted in the “item extraction” step is shown in FIG. 7 by a predetermined rule when generating an expression model. Number them in the same order. As shown in FIG. 7, the coordinate values of the outermost quadrangle of the extracted characters of each item are transferred to the character recognizer and recognized one by one, and description thereof will be omitted.

This recognized result is automatically converted into an HTML file which is a Web document by tag generation as shown in FIG.

Finally, referring to FIG. 9, an example of actually using the automatic interpretation method of the form document area according to the present invention in a general form is illustrated. FIGS. 9A to 9D illustrate examples of form images, structure analysis result screens of form images, The result screen of the classification of the format video and the item processing result screen of the format video are shown.

As described in detail above, the automatic interpretation method of the form document image of the present invention receives a form document of an existing large office made of paper through an input medium such as a scanner or a camera, and converts the form document into an electronic form document. Structured form It enables the automatic processing of document images, and in the form that maintains the topology to analyze the structure of any form document filled with contents, it is a logical form that is regarded as the same format even if the size and position variation are changed. Since the structure analysis method is included, it is effective to classify the format document image with minimal registration.

In addition, if the lines of the form which were difficult to process by the conventional methods are broken, and in the case of the dotted line form, when the handwritten characters and the line of the form are in contact with each other, the form is inclined, and the result is good. There is an effect that can be obtained.

Although the technical idea of the automatic interpretation method of the format document image of the present invention has been described together with the accompanying drawings, this is illustrative of the best embodiment of the present invention and is not intended to limit the present invention. In addition, it is obvious that any person skilled in the art can make various modifications and imitations without departing from the scope of the technical idea of the present invention.

Claims (7)

In the automatic interpretation method of a form document image for converting an arbitrary form document image filled with content into an electronic form document, A first step of analyzing a structure of the format document image to create an expression model representing the input format document image; A second step of comparing the representation model for the input format with a standard model already registered; A third step of adding an item to be processed for the new form to the item definition database after registering as a new model if there is no matching model as a result of the comparison of the second step; And And a fourth step of extracting an item area to be processed from the classified input form document image and converting it into an electronic form document after classifying the matching form if there is a matching model. Automatic interpretation of document images. The method of claim 1, The first step is, A fifth step of extracting line segments of the form from the scanned form document image; Extracting a rectangle surrounding the item of the form from the extracted vertical and horizontal segments; And A method of generating a representation model of an input format image based on the relationship between the length, height, position, and segment of the extracted vertical and horizontal segments and the information about the upper, lower, left, and right edges forming the extracted visual form. Automated interpretation of format document image including 7 steps. The method of claim 2, The fifth step, An eighth step of extracting connected components forming horizontal and vertical segments by mathematical morphology calculation; A ninth step of filtering the connection elements that are not suitable to be regarded as lines of the form by calculating the width and the slope of the line segments for the extracted connection elements; And After aligning all the horizontal connectors from top to bottom, align all vertical connectors from left to right to group adjacent horizontal or vertical connectors into one line to extract the segment of the form. An automatic interpretation method of a form document image including the tenth step. The method of claim 2, The sixth step, An eighth step of excluding the candidate rectangle if the sum of the rectangles covers the entire candidate rectangle; Extracting candidate rectangles as formatting items when edges of other rectangles cover the entire edge of one candidate rectangle; And And a tenth step of extracting the candidate rectangle as a form item when the edges of the other rectangles cover a portion of one edge of one candidate rectangle and the length of the uncovered portion is long enough. . The method of claim 3, wherein In the tenth step, the grouping of the horizontal or vertical connecting elements adjacent to each other by one line is performed. If two connecting elements i and j are on the same side as the standard line, the distance d _ij is | d _i -d _j |, and if two segments i and j are on the other side of the line, the distance d _ij is | d _i + d _j | If the distance is less than a predetermined threshold value is determined to be close to the automatic document document image analysis method characterized by grouping the horizontal or vertical connection elements in a single line. The method of claim 1, The second step, Method for automatic analysis of form document image, characterized in that the logical structure information of the form and the physical position and size information is calculated by comparing the distance with the currently registered form model to determine whether the calculated distance is equal to or less than a predetermined value. . On your computer, In the automatic interpretation method of a form document image for converting an arbitrary form document image filled with content into an electronic form document, A first step of analyzing a structure of the format document image to create an expression model representing the input format document image; A second step of comparing the representation model for the input format with a standard model already registered; A third step of adding an item to be processed for the new form to the item definition database after registering as a new model if there is no matching model as a result of the comparison of the second step; And As a result of the comparison of the second step, if there is a matched model, the program is executed to perform the fourth step of extracting an item area to be processed from the classified input form document image and converting it into an electronic form document. Recording medium.