JP2010061693A - Slip recognition method and slip recognition apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To recognize a table structure and automatically decide on an attribute of the smallest rectangular shape in a slip recognition apparatus that automatically recognizes a layout of a slip. <P>SOLUTION: Items are classified into a logic row from a connection relationship in a row direction of the smallest rectangular shape that is extracted from a slip image; logic rows that have sequentially the same row structure are put into a block; and blocks that have the same column structure are put into a unit of logic column between blocks that connect to the top and the bottom of a table. From this table structure, a block that has the largest number of belonging rows is decided on as a body portion; an upper portion and a lower portion of the block are decided on as a header portion and a footer portion, respectively; the header portion is set as a title portion; character recognition is performed in a logic column unit to decide on an attribute of the title portion; and an attribute of the smallest rectangular shape of the body portion is decided on from the attribute of the title portion. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a form recognition method and form recognition apparatus for recognizing a format of a form in order to recognize characters on the form, and more particularly, to automatically analyze the table structure of a form having a table and configure each table. The present invention relates to a form recognition method and form recognition apparatus for determining item attributes.

  A character recognition method for recognizing characters on a form is used for automatic input of data. In this character recognition, an image at a specified position on a form is taken out and character recognition is performed.

  In order to identify a character on a form, it is necessary to define character attributes such as a data name (field identifier) and a character type of the character.

  FIG. 87 is an explanatory diagram of the prior art.

  FIG. 87 shows a transfer request form as a form. This transfer request form includes items such as “transfer destination”, “bank name”, and “branch name”. In order to automatically recognize such items, it is necessary to define attributes such as field ID (data name), start position, end position, number of digits, recognition category (character type) of each item. Conventionally, this definition information has been registered for each form. For this registration, the operator has input for each form.

International Publication No. 97/005561

  However, in the prior art, definition information has to be registered in advance. For this reason, only a form in which definition information is registered in advance can recognize characters. For example, in a financial institution, when a transfer is automatically input, there are various formats for a transfer request form created by a company. There is a problem that creating such definition information for each form in advance is troublesome.

  Even if the definition information of the form is registered, there is a problem that the definition information needs to be changed when the format of the form is changed.

  An object of the present invention is to provide a form recognition method and a form recognition apparatus that automatically recognize definition information of each item of a form.

  Another object of the present invention is to provide a form recognition method and a form recognition apparatus that analyze the arrangement of each item of a form and determine the attribute of each item.

  Still another object of the present invention is to provide a form recognition method and a form recognition apparatus for determining a minimum rectangular attribute of a form and recognizing characters in the data portion of the form.

  The present invention is a form recognition method for recognizing a table structure of a form. The method includes the steps of extracting a minimum rectangle composed of vertical and horizontal ruled lines of the form from the form image, and logical lines constituting each row of the table from the connection relation of the minimum rectangle in the line direction of the form. A step of grouping the logical rows having the same row structure into one block, and a block having the same column structure between the blocks connected above and below the table Extracting the block having the same column structure in units of logical columns, and the number of rows belonging to the logical row, the block, and the table structure analyzed into the logical columns is the maximum. Determining a block as a body part, determining blocks located above and below the block as a header part and a footer part, and a logical line of the header part as a header part And recognizing the character of the heading part in units of the logical columns to determine the attribute of the heading part in units of the logical columns, and from the attribute of the heading part in units of the logical columns, Determining an attribute of the minimum rectangle in the logical column.

  In another aspect of the present invention, the step of extracting the relationship in the row direction includes the step of classifying the minimum rectangle into the minimum rectangle constituting each row from the connection relationship of the minimum rectangle, the continuous and the same row And the step of extracting the relationship in the column direction includes the step of extracting a relationship of blocks having the same column structure between blocks connected vertically.

  According to still another aspect of the present invention, the step of analyzing the table structure includes a step of classifying the minimum rectangle to be connected into a group based on the connection relationship of the minimum rectangle, and a step of classifying the group into a component unit of the table. And analyzing a relation between a row and a column between the minimum rectangles for each component of the table.

  In still another embodiment of the present invention, the step of analyzing the relationship in the row direction includes a step of analyzing a nested structure of rows formed by the minimum rectangle and setting nested information in the minimum rectangle.

  In still another embodiment of the present invention, the analyzing step includes a step of extracting a table structure of each component and a step of integrating components having the same table structure.

  In still another aspect of the invention, the analyzing step includes a step of extracting a table structure of each component and a step of recovering ruled lines in the component from the table structure of the component.

  According to still another aspect of the invention, the step of analyzing the relation in the row direction includes the step of detecting a strike line given to the row from the structure of each row, and removing the strike line, Determining a relationship.

  In still another embodiment of the present invention, the analyzing step includes a step of detecting that the minimum rectangles having the same size are consecutive, and a step of integrating the continuous minimum rectangles into one minimum rectangle.

  In the present invention, the minimum rectangle of the table structure is extracted, and the physical arrangement of the minimum rectangle is detected. The table structure of the form is analyzed from the physical arrangement of the minimum rectangle, and the attribute of the minimum rectangle is determined. Since the physical arrangement of the minimum rectangle is detected and the attribute of the minimum rectangle is determined, the table structure of the form can be automatically analyzed. For this reason, the trouble of registering each item of the form in advance can be saved. Further, even if the format of the form is changed, it is possible to save time and labor for changing the definition information manually.

It is a block diagram of one embodiment of the present invention. It is 1st form example explanatory drawing of one embodiment of this invention. It is a 2nd example document explanatory drawing of one embodiment of this invention. It is a whole flowchart (the 1) of one embodiment of this invention. It is a whole flowchart (the 2) of one embodiment of this invention. It is explanatory drawing of the ruled line extraction process of FIG. It is explanatory drawing of the group extraction process of FIG. It is explanatory drawing of the connection relation of the group extraction process of FIG. It is explanatory drawing of the classification | category process of the table component of FIG. FIG. 5 is an explanatory diagram of logical row classification processing of FIG. 4. It is operation | movement explanatory drawing of the logical line classification | category process of FIG. It is explanatory drawing of the nested extraction process of FIG. It is explanatory drawing of the integration process of the table | surface structure part of FIG. It is explanatory drawing of the ruled line creation process of FIG. It is explanatory drawing of the block classification | category process of FIG. It is explanatory drawing of the ruled line recovery process of FIG. FIG. 16 is an explanatory diagram of the logical row integration process of FIG. 15. FIG. 5 is an explanatory diagram of the logical column determination process of FIG. 4. It is explanatory drawing of the nesting extraction process of the column direction of FIG. It is explanatory drawing of the nested structure of the 1-digit and 3-digit ruled line of FIG. It is explanatory drawing of the nesting analysis of locations other than the main table of FIG. It is explanatory drawing of the attribute provision of the block of FIG. It is explanatory drawing of the flag setting process of FIG. It is explanatory drawing of the headline recognition process of FIG. It is explanatory drawing of the headline wording dictionary of FIG. FIG. 6 is a relationship diagram of tables for the processes of FIGS. 4 and 5. It is explanatory drawing of the horizontal ruled line table of FIG. It is explanatory drawing of the vertical ruled line table of FIG. It is explanatory drawing of the ruled line item table of FIG. It is a gradient calculation processing flowchart. It is a coordinate correction process flowchart. It is a group processing flowchart. It is explanatory drawing of the table update of FIG. It is a component element classification processing flowchart (the 1). It is a component element classification processing flowchart (the 2). It is table explanatory drawing (the 1) after the classification | category of FIG.34 and FIG.35. FIG. 36 is a table explanatory diagram (No. 2) after the classification of FIGS. 34 and 35; It is a logical line classification processing flowchart. It is a nested structure analysis processing flowchart. It is table explanatory drawing (the 1) after logical line extraction. It is table explanatory drawing (the 2) after logical line extraction. It is an integrated processing flowchart. It is table explanatory drawing (the 1) of form example 2. FIG. It is table explanatory drawing (the 2) of form example 2. FIG. FIG. 10 is a table explanatory diagram (No. 1) after group integration of a form example 2; FIG. 11 is a table explanatory diagram (No. 2) after group integration of a form example 2; It is a ruled line creation processing flowchart. It is explanatory drawing of ruled line creation. It is table explanatory drawing (1) after ruled line creation. It is table explanatory drawing (2) after ruled line creation. It is a block classification processing flowchart. It is table explanatory drawing (the 1) after block classification. It is table explanatory drawing (the 2) after block classification. It is explanatory drawing of a ruled line recovery process. It is a ruled line recovery process flowchart. It is table explanatory drawing (1) before ruled line recovery. It is table explanatory drawing (2) before ruled line recovery. It is table explanatory drawing (1) after ruled line recovery. It is table explanatory drawing (the 2) after ruled line recovery. It is table explanatory drawing (3) after ruled line recovery. It is explanatory drawing of a logical line integration process. It is a logical line integration processing flowchart. It is table explanatory drawing (the 1) before logical row integration. It is table explanatory drawing (the 2) before logical row integration. It is table explanatory drawing (the 1) after logic line integration. It is table explanatory drawing (the 2) after logic line integration. It is table explanatory drawing (the 3) after logical line integration. It is a logical column determination processing flowchart. It is table explanatory drawing (the 1) after logical column determination. It is table explanatory drawing after the logical column determination (the 2). It is table explanatory drawing after the logical column determination (the 3). FIG. 10 is a flowchart of a 1-digit and 3-digit ruled line nesting analysis process. It is table explanatory drawing after a nesting analysis. It is a nesting analysis processing flowchart other than the main table. It is table explanatory drawing (the 1) after a nesting analysis. It is table explanatory drawing (the 2) after a nesting analysis. It is an attribute provision process flowchart. It is table explanatory drawing after an attribute provision. It is a flag setting processing flowchart. It is table explanatory drawing after flag setting. It is a heading recognition process flowchart. It is table explanatory drawing after headline recognition. It is an attribute setting process flowchart. It is table explanatory drawing after an attribute setting. It is a character recognition processing flowchart. It is explanatory drawing of a character recognition result. It is explanatory drawing of a prior art.

  FIG. 1 is a block diagram of a form recognition apparatus according to an embodiment of the present invention, FIG. 2 is an explanatory diagram of a first form example used in the embodiment of the present invention, and FIG. It is explanatory drawing of the 2nd example of a form used for embodiment.

  As shown in FIG. 1, the reader 1000 optically reads a form. A processor (CPU) 1001 analyzes a form and performs character recognition. The memory 1002 stores a headline wording dictionary 1003 and a general dictionary 1004. The headline wording dictionary 1003 stores a predetermined headline wording and its image data. The general dictionary 1004 stores a dictionary for each character recognition category / character font for character recognition of the data portion.

  The CPU 1001 analyzes the ruled line of the form, the ruled line extracting unit 1005 for extracting the minimum rectangle composed of the ruled lines, the analyzing unit 1006 for analyzing the arrangement of the extracted minimum rectangles and analyzing the table structure, and the analyzed table structure. , An attribute determination unit 1007 that determines the attribute of each minimum rectangle, and a character recognition unit 1008 that performs character recognition according to the attribute. As will be described later, each of these units represents processing performed by the CPU 1001 in blocks.

  FIG. 2 shows a first form example to be analyzed. This form is a transfer request form. The first form in FIG. 2 is for entering the requester, request date, scheduled transfer date, transfer destination (bank name, branch name), subject, account number, recipient name, amount, subtotal, and total. In this form, (number) is a horizontal ruled line extending in the horizontal direction, [number] is a vertical ruled line extending in the vertical direction, and a number is a minimum rectangle (hereinafter referred to as a ruled line item) surrounded by the vertical ruled line and the horizontal ruled line. ). Accordingly, the form shown in FIG. 2 includes 20 horizontal ruled lines (1) to (20), 13 vertical ruled lines [1] to [13], and 176 ruled line items 1 to 176.

  FIG. 3 shows a second form example to be analyzed. This form is also a transfer request form. The second form in FIG. 3 is for entering the client, bank name, branch name, subject, account number, payee name, amount, subtotal, total, application, and remarks. Also in this form, (number) is a horizontal ruled line extending in the horizontal direction, [number] is a vertical ruled line extending in the vertical direction, and a number is a minimum rectangle surrounded by the vertical ruled line and the horizontal ruled line (hereinafter referred to as ruled line items). Call). Accordingly, the form shown in FIG. 3 includes 30 horizontal ruled lines (1) to (30), 20 vertical ruled lines [1] to [20], and 171 ruled line items 1 to 171. This form has a table portion divided into two.

  The present invention automatically analyzes the type of each ruled line item of a form having such a table structure.

  4 and 5 are overall flowcharts of an embodiment of the present invention, and FIGS. 6 to 25 are explanatory diagrams of the processing.

  (S1) The CPU 1001 extracts a horizontal ruled line and a vertical ruled line on the form, and a minimum rectangle (ruled line item) surrounded by the horizontal ruled line and the vertical ruled line from the image data of the form. As shown in FIG. 6A, the number of consecutive black dots in the image data is counted, and when a predetermined number or more of dots are consecutive, it is determined as a horizontal ruled line or a vertical ruled line. FIG. 6B shows an example of horizontal ruled lines and vertical ruled lines. The ruled lines are extracted as physical coordinates, and the vertical and horizontal ruled lines are indicated by start coordinates and end coordinates. The ruled line item is obtained by extracting a minimum rectangle surrounded by two vertical ruled lines and horizontal ruled lines as shown in FIG. This ruled line item is also indicated by start coordinates and end coordinates.

  Next, the CPU 1001 calculates the slope of the image data form, and corrects the start and end coordinates of the ruled line item, the horizontal ruled line, and the vertical ruled line in accordance with the skew.

  (S2) As shown in FIG. 7, the CPU 1001 classifies the ruled line items such that those in the positional relationship to be connected are in the same group. With reference to the coordinates of each ruled line item, those who are in a positional relationship connected vertically or horizontally are classified into the same group. As shown in FIG. 7, the first form in FIG. 2 is classified into three groups. FIG. 8 shows the connection relationship. As shown in FIG. 8, in order to determine whether or not ruled line items are connected, the distance between ruled line items can be determined in the horizontal or vertical direction as well as when the ruled line items are completely connected in the horizontal or vertical direction. Those that are within a predetermined threshold range are also determined to be connected.

  (S3) Next, the CPU 1001 classifies the group into the constituent elements of the table. The group into which the ruled line items are classified is further subdivided into structural units of a table structure for each group. As shown in FIG. 9, the structural unit is composed of a main table portion, a portion protruding upward (upward convex portion), and a portion protruding downward (lower convex portion). For this reason, it is determined in which group each ruled line is located (which group is configured) from the coordinates. Then, a search is made for each group unit that can form a rectangle (square, rectangle) with two vertical ruled lines and horizontal ruled lines. Then, the largest rectangle among the rectangles formed by ruled lines is set as the largest rectangle in the group. For each group, a ruled line item existing at a position not included in the maximum rectangle is searched, and an item positioned above the maximum rectangle is classified as an upper convex portion, and a lower one is classified as a lower convex portion. Then, the ruled line items included in the maximum rectangle are classified into the main table portion. Hereinafter, this subdivided unit is called a group.

  When the maximum rectangle is obtained in the form shown in FIG. 9, the maximum rectangle is a range surrounded by the horizontal ruled line a, the horizontal ruled line b, the vertical ruled line c, and the vertical ruled line d. Accordingly, the main surface portion is ruled line items 3 to 26, the upward convex portion is ruled line items 1 to 2, and the downward convex portion is ruled line items 27 to 32.

  (S4) The CPU 1001 classifies the group of constituent elements in the table into logical rows. For each group, the ruled line items to be configured are classified into logical lines in which those having logical connections in the row direction (the horizontal direction of the form) are grouped together. The logical connection referred to here means a connection in a state of connection based on the concept of the same width connection.

  It is determined that ruled line items having the same width connection form the same logical line. The same width connection will be described with reference to FIGS.

  As shown in FIG. 10A, when ruled line items are adjacent with the same line width and share the constituent line segments of the ruled line items, it is determined that the currently referenced ruled line items constitute the same logical line. In FIG. 10A, in the ruled line item A and the ruled line item B, the line segment a4 and the line segment b1 are the same line segment, and the length of the line segment is also the same. Judge that it is in a state.

  As shown in FIG. 10B, when ruled line items are close to each other and have the same line width, it is determined that the ruled line items being referenced constitute the same logical line. As shown in FIG. 10B, when the distance d between the ruled line item A and the ruled line item B is equal to or less than a predetermined threshold and the line segment a4 and the line segment b2 have the same length, the two ruled line items are: It is determined that the connection is established.

  As shown in FIG. 10C, when a plurality of ruled line items are adjacent to each other with the same width and share the constituent line segments of the ruled line item with respect to one ruled line item, the reference ruled line item is the same logical line. Is determined. In FIG. 10C, since the sum of the line segment b2 of the ruled line item B and the line segment c2 of the ruled line item C is the same as the length of the line segment a4 of the ruled line item A, the three ruled line items are connected. Judge that it is in a state.

  As shown in FIG. 10D, when each ruled line item is close and the line width of one ruled line item and the sum of the line widths of a plurality of ruled line items are the same, the ruled line item being referenced has the same logic. Judge that it constitutes a row. In FIG. 10D, the distance d2 between the ruled line item B and the ruled line item C is less than the threshold, and the distance d1 between the ruled line item A and the ruled line items B and C is less than the threshold. Since the sum of the line segment b2 of the ruled line item B and the line segment c2 of the ruled line item C is substantially equal to the length of the line segment a4 of the ruled line item A, it is determined that the three ruled line items are in a connected state.

  In this way, each group is classified into logical rows. As shown in FIG. 11, the group 3 of the form in FIG. 2 is classified into 11 logical rows.

  Next, the nested structure portion in the row direction is analyzed. A rectangle having a logical meaning made up of a plurality of ruled line items is called a nested structure. As shown in FIG. 12A, since the items of “bank name” and “branch name” are related to the item of “transfer destination”, this portion is defined as a nested structure. As shown in FIG. 12 (B), in the case of FIG. 12 (A), the ruled line item 1 (transfer destination) is the parent of the ruled line item 2 (bank name). Accordingly, a parent-child relationship is set for the ruled line items 1 and 2. Ruled line item 2 (bank name) and ruled line item 3 (branch name) have a sibling relationship (same level relationship). Accordingly, a sibling relationship is set for the ruled line items 2 and 3.

  In some cases, as shown in the second form in FIG. 3, the table component is divided into left and right parts for the sake of layout. If this is judged from the logical meaning of the structure of the table, it is necessary to combine them into one table structure in order to determine the table structure. For this reason, as shown in FIG. 13, when there are a plurality of groups located side by side, the logical row structure constituting each group is determined. When the structure of the group by each logical row is the same, the table structure is integrated.

  Furthermore, when a part of the table divided into the left and right disappears due to a curling or the like, a ruled line is created in that part. When the above-mentioned tables separated into the left and right are merged, any of the ruled lines in one of the tables may be lost due to a curl or the like. As shown in FIG. 14, since the logical row structures of the two groups match except for the portion where the ruled lines disappear, the ruled lines are matched so that the other logical row structure and the one logical row structure are matched. Create a new. Thereby, a ruled line item including the generated ruled line is generated. Thus, after creating ruled lines, the above-described table structure integration processing is performed.

  (S5) Next, consecutive logical rows having the same structure are grouped into blocks. Compare the structures of logical rows adjacent to each other in one group. Then, logical rows having the same structure are classified into one block. The logical line comparison elements include the number of ruled line items to be configured, the arrangement of ruled line items in the logical line, and the height and width of the logical line. These are compared to determine whether they have the same structure.

  In the form example in FIG. 15, seven logical rows in one group are classified into three blocks 1 to 3. Block 1 is composed of logical rows 1, block 2 is composed of logical rows 2 to 6, and block 3 is composed of logical rows 7.

  Also, the blocks having different structures are converted into the same block structure. In other words, when a part of the ruled line in the table is lost due to a blur or the like, a ruled line that has disappeared from another logical row structure is newly generated. A ruled line item is generated accordingly.

  If it demonstrates in the example of the table | surface of FIG. 16, a ruled line will lose | disappear and it will classify | categorize into the blocks 1-5. However, the block 2 and the block 4 are blocks having the same structure. Accordingly, a ruled line is created in the block 3. Thereby, the blocks 2 to 4 have the same structure. As a result, the blocks are integrated into three.

  Next, the logical lines divided by the strikethrough are integrated. In salary transfer forms handled by financial institutions, the transaction part may be erased with a strike line to clarify that it is not subject to processing. In this case, since the strike line is extracted as a ruled line, the analysis result becomes inaccurate. Therefore, it is necessary to analyze the table structure excluding the strikethrough. In the state classified into the blocks, the block in the portion drawn with a strike-through has the following characteristics.

  First, there are separate blocks above and below. Second, the upper block and the lower block have the same structure. If there is a block having this feature, a block excluding the ruled line due to the erased line is created in this block. When this block has the same structure as the upper block, it is determined that the block structure has changed due to the crossing out. Then, the erase line is erased, and along with this, ruled line items, logical lines, and blocks are recreated.

  As shown in FIG. 17, if there is a strikethrough in block 3 and block 5, the block is classified into seven. Here, the block 2 and the block 4 have the same structure. When the erased line of the block 3 is erased, the same structure as that of the block 2 is obtained. The same applies to blocks 4-6. Accordingly, the erase line is erased, and the ruled line item and the logical line are recreated. Along with that, the block is recreated. Thereby, it is reclassified into 3 blocks.

  (S6) Next, the logical column structure is determined. What has a table structure consists of a word fixed part and a word variable part. The wording fixed part is a fixed part of a form such as a heading part of a table. The word variable part is a variable part of a form such as a data part of a table. The logical column structure is determined by determining the logical column structure between the word fixing unit and the word variable unit. In other words, it is determined whether the blocks connected in the vertical direction have the same column structure. If the same column structure is used, these blocks are integrated in units of logical columns. For example, in the example of FIG. 18A, the logical columns are classified into six.

  In this case, as shown in FIG. 18B, the block 2 of the wording variable section has a two-column structure of “A bank” and “Kakiku branch”. It has a structure. In this case, since one column (transfer destination) of block 1 is the heading of two columns of block 2, when viewed as a table structure, two columns of block 2 are regarded as one column.

  Similarly, as shown in FIG. 18C, the block 1 of the wording fixed unit has a structure of two columns of “bank name” and “branch name”, but the block 2 of the wording variable unit has one column. It has a structure. In this case, since the two columns of block 1 (bank name and branch name) are the headings of the first column of block 2, when viewed as a table structure, the two columns of block 1 are regarded as one column.

  Next, a logical nesting structure in the column direction is extracted. As a result of determining the logical column structure between the blocks, as shown in FIGS. 18B and 18C, one block has a plurality of columns and the other block has a single column logical column structure. If so, these are expressed as nested structures. The ruled line items constituting the nested structure are collectively given one logical meaning.

  FIG. 19A shows a nested structure in the case of FIG. As shown in FIG. 19A, ruled line item 2 and ruled line item 3 are in a sibling relationship (at the same level). Both ruled line items are combined into a ruled line item that forms a column for the ruled line item 1. FIG. 19B shows a nested structure in the case of FIG. As shown in FIG. 19B, ruled line item 1 and ruled line item 2 are in a sibling relationship (at the same level). Both ruled line items are combined into a ruled line item that forms a column for the ruled line item 3.

  Further, the nested structure of the 1-digit ruled line item part and the 3-digit ruled line item part is analyzed. As shown in FIGS. 20A and 20B, when there are an amount column and a fee column in the table structure of a form, a one-digit ruled line item or a three-digit ruled line item continues. A one-digit ruled line item is one in which a one-digit number is entered or printed in one ruled line item. Further, the 3-digit ruled line item means that a 3-digit number is entered or printed in one ruled line item.

  Together these have one meaning. These are logically combined into one ruled line item. For this reason, the vertical size and horizontal size of a 1-digit ruled line item or a 3-digit ruled line item are defined in advance. When an arbitrary number of ruled line items that match these sizes are arranged, they are nested and logically combined as one ruled line item.

  By using such a result, in the character recognition process, continuous ruled line items can be handled as one character string. For example, as a result of character recognition in the ruled line item of part a in FIG. 20A, when the result of part b is obtained, the ruled line item has a nested structure. Are combined and handled as one character string. The same applies to the 3-digit ruled line item shown in FIG.

  Next, a rectangular nested structure having a special structure is analyzed. Many parts of the form other than the main table have a special structure. For these, nested structure analysis processing is performed for each structure. In FIG. 21A, ruled line item 2 has a parent-child relationship with ruled line item 1. The ruled line item 4 has a parent-child relationship with respect to the ruled line item 2. The ruled line item 5 has a parent-child relationship with respect to the ruled line item 3. The ruled line item 3 has a sibling relationship with the ruled line item 2. The ruled line item 5 has a sibling relationship with the ruled line item 4. In FIG. 21B, the ruled line item 2 has a parent-child relationship with respect to the ruled line item 1. The ruled line item 4 has a parent-child relationship with respect to the ruled line item 2. The ruled line item 5 has a parent-child relationship with respect to the ruled line item 3. The ruled line item 3 has a sibling relationship with the ruled line item 2. The ruled line item 5 has a sibling relationship with the ruled line item 4. The nested structure is analyzed in the same manner as the above-described analysis process of the nested structure in the row direction and the column direction.

  (S7) Next, an attribute is given to the block of the table structure part. The form includes a heading part and a data part corresponding to the heading part. Also, depending on the form, there is a part for collecting data (for example, a sum total column in the data part). Here, attributes are assigned to the blocks constituting the table structure section, and each block has a meaning. That is, as shown in FIG. 22, a part where data in a table is entered or printed is defined as a body part, and a body part attribute is given to a block having the largest number of logical rows among the blocks constituting the table. The part where the headline of the table is written or printed is defined as the header part, and the header part attribute is given to the block located above the body part. A part in which data (for example, a total column or the like) that summarizes the data part of the table is entered or printed is defined as a footer part, and a footer part attribute is given to a block located below the body part.

  (S8) Next, a heading flag is set in the ruled line item constituting the heading. A heading flag is set for the ruled line item corresponding to the heading in accordance with the following rules from the table structure format. The ruled line items for setting the heading flag are as follows, as shown in FIG.

-Ruled line items that make up the header block-Ruled line items that make up the top logical row of the body block-Ruled line items that exclude the rightmost of the ruled line items that make up the footer block-Logical line of the downward convex part Ruled line items excluding those located at the rightmost among the ruled line items that make up the ruled line-For ruled line items that constitute a rectangular logical row other than the table, among the nested ruled line items, the one located at the bottom right Ruled line items excluding the character If this heading part is extracted, the characters to be used can be limited, and character recognition processing specialized for the characters (words) can be performed. Therefore, it is possible to recognize the wording of the heading part with high accuracy.

  (S9) Next, the type of heading is determined. Character recognition is performed for a character string existing in a ruled line item for which a heading flag is set. As shown in FIG. 25, a headline wording dictionary 1003 (see FIG. 1) is used as a recognition dictionary used for character recognition in the headline part. The headline wording dictionary 1003 is registered with words (transfer destinations and the like) existing as headlines and their images. Therefore, character recognition can be performed with higher accuracy than when a general character recognition dictionary is used.

  The heading table shown in FIG. 24 is referred to using the heading name obtained from the recognition result as a key. As shown in FIG. 24, attribute information (character recognition category, character font, etc.) corresponding to the heading name is registered in the heading table. Then, a record (attribute information) matching the heading name is extracted.

  (S10) Next, the recognition attribute of the ruled line area corresponding to the headline is determined. For this reason, the logical column information including the ruled line item in which the heading is found is referenced to find the ruled line item of the data portion corresponding to the heading. That is, of the ruled line items constituting the logical column, the recognition characteristics such as the character recognition category and the character font are copied from the ruled line item of the heading for the ruled line items other than the heading.

  (S11) Next, character recognition in an area other than the headline is performed. Character strings existing in ruled line items other than the headline ruled line items are recognized using the general character recognition dictionary 1004 (see FIG. 1). At this time, the character recognition dictionary 1004 is provided separately for each recognition category such as kanji and numbers. Therefore, according to the recognition attribute of the data part described above, a recognition dictionary is selected from the general dictionary and character recognition is performed. For this reason, the data portion can also be recognized with high accuracy.

  Further, the obtained recognition result is set in the recognition result notification area. Thereby, the data name of the data part, the number of data, and the character recognition result are obtained.

  In this way, the relation between the row and the column of each ruled line item is determined, and classified into a heading part and a data part corresponding thereto. Then, the heading part is character-recognized using a heading recognition dictionary to obtain a heading name. The data name of the data part corresponding to the heading part is found from the relationship of the structure of the logical column. For this reason, it is not necessary to previously define the format of each item of the form.

  Furthermore, the recognition attribute is obtained from the heading name by the heading table. Accordingly, the data portion is recognized as characters. For this reason, the table structure of the form can be analyzed, and the accuracy of character recognition results in the data portion can be improved.

  Next, each process of above-mentioned step S1-S11 is demonstrated in detail. Here, FIG. 26 is a relationship diagram of tables used for each process. As shown in FIG. 26, the horizontal ruled line table 1105 stores the start coordinates and end coordinates of the extracted horizontal ruled lines. The vertical ruled line table 1106 stores the start coordinates and end coordinates of the extracted vertical ruled lines. The ruled line item information table 1104 stores the start coordinate and end coordinate of the extracted ruled line item (minimum rectangle), the group number to which it belongs, the block number, and the like. The group information table 1100 stores the top ruled line item address, the top block number, etc. of each classified group.

  The block information table 1101 stores the top ruled line item address, the top logical line address, and the like of each extracted block. The logical line information table 1102 stores the first ruled line item address, block number, and the like of each extracted logical line. The logical column information table 1103 stores the first ruled line item address and the like of each extracted logical column.

(A) Description of Ruled Line Extraction Processing FIG. 27 is an explanatory diagram of a horizontal ruled line information table, FIG. 28 is an explanatory diagram of a vertical ruled line information table, and FIG. 29 is an explanatory diagram of a ruled line item table.

  As is well known, the horizontal ruled line is extracted by extracting a group of black pixels in which a certain number or more of black pixels continue in the horizontal direction. Then, as shown in FIG. 27, each extracted horizontal ruled line is numbered, and its start coordinate and end coordinate are stored in the horizontal ruled line information table 1005.

  As is well known, vertical ruled lines are extracted by extracting a group of black pixels in which a certain number or more of black pixels continue in the vertical direction. Then, as shown in FIG. 28, each extracted vertical ruled line is numbered, and its start coordinate and end coordinate are stored in the vertical ruled line information table 1006.

  As is well known, the minimum rectangle (ruled line item) is also extracted by extracting the minimum rectangle surrounded by two horizontal ruled lines and two vertical ruled lines. A number is assigned to each extracted ruled line item. As shown in FIG. 29, the ruled line item information table 1104 includes a start coordinate SC, an end coordinate EC, an upper horizontal ruled line number ULN, a lower horizontal ruled line number LLN, a left vertical ruled line number LVN, and a right vertical ruled line number RVN. Is stored. The ruled line item information table 1104 includes group number GN, block number BN, row number LN, column number RO, upper ruled line item URN, lower ruled line item LRN, left ruled line item ERN, right ruled line item RRN, and nested information IN. Has each column. Each of these fields is used by later processing.

  Next, the inclination correction will be described. FIG. 30 is a flowchart of the slope calculation process.

  (S20) The horizontal ruled lines in the horizontal ruled line information table 1105 are referred to. Then, it is determined whether all horizontal ruled lines have been referenced.

  (S21) The inclination of the referenced horizontal ruled line is calculated. The slope θ is calculated by the following equation (1), where Ye is the end Y coordinate, Ys is the start Y coordinate, Xe is the end X coordinate, and Xs is the start X coordinate.

tan θ = (Ye−Ys) / (Xe−Xs) (1)
Then, the process returns to step S20.

  (S22) When all the horizontal ruled lines are referred to in step S20, an average value of the inclination is calculated. The average value of the inclination is calculated by (total of inclination values) / (number of horizontal ruled lines). And it ends.

  Next, the coordinate correction process will be described. FIG. 31 is a flowchart of coordinate correction processing.

  (S25) Each ruled line item in the ruled line item table 1104 is referred to. It is determined whether all ruled line items in the ruled line item table 1104 have been referenced. If all ruled line items have been referenced, the process proceeds to step S27.

  (S26) The upper left coordinates (start coordinates) Xs, Ys of the ruled line item are corrected according to the inclination tan θ. The corrected coordinates X and Y are expressed by the following equations (2) and (3).

X = Xs + Ys × tan θ (2)
Y = Ys−Xs × tan θ (3)
Then, the lower right coordinates (end coordinates) Xe and Ye of the ruled line item are corrected according to the inclination tan θ. The corrected coordinates X and Y are represented by the following equations (4) and (5).

X = Xe + Ye × tan θ (3)
Y = Ye-Xe × tan θ (4)
Then, the process returns to step S25.

  (S27) Next, the horizontal ruled line table 1105 is referred to in order to correct the horizontal ruled line coordinates. It is determined whether all horizontal ruled lines in the horizontal ruled line table 1105 have been referenced. If all horizontal ruled lines have been referenced, the process proceeds to step S29.

  (S28) The left coordinates (start coordinates) Xs and Ys of the horizontal ruled lines are corrected according to the inclination tan θ. The corrected coordinates X and Y are expressed by the aforementioned equations (2) and (3).

  Then, the right coordinates (end coordinates) Xe and Ye of the horizontal ruled line are corrected according to the inclination tan θ. The corrected coordinates X and Y are expressed by the aforementioned equations (4) and (5).

  Then, the process returns to step S27.

  (S29) Next, the vertical ruled line table 1106 is referred to in order to correct the vertical ruled line coordinates. It is determined whether all vertical ruled lines in the vertical ruled line table 1106 have been referred to. If all vertical ruled lines have been referenced, the process ends.

  (S30) The left side coordinates (starting coordinates) Xs and Ys of the vertical ruled lines are corrected according to the inclination tan θ. The corrected coordinates X and Y are expressed by the aforementioned equations (2) and (3).

  Then, the right coordinates (end coordinates) Xe and Ye of the vertical ruled line are corrected according to the inclination tan θ. The corrected coordinates X and Y are expressed by the aforementioned equations (4) and (5).

  Then, the process returns to step S29.

  In this way, horizontal ruled lines, vertical ruled lines, and ruled line items (minimum rectangle) are extracted. Then, in order to correct the coordinate error due to the skew of the form, the inclination is calculated from the horizontal ruled line coordinates. Since the slope is an average value, the slope can be obtained accurately. Further, the coordinates of the ruled line item, the horizontal ruled line, and the vertical ruled line are corrected based on the inclination.

(B) Description of Group Classification Processing FIG. 32 is a group processing flowchart, and FIG. 33 is an explanatory diagram of table update after group processing.

  (S31) A ruled line item is referenced from the ruled line item table 1104. It is determined whether all ruled line items have been referenced. If all the ruled line items have been referenced, the process ends.

  (S32) Other ruled line items are extracted from the ruled line item table 1104. It is determined whether all other ruled line items have been referenced. If all ruled line items have been referenced, the process proceeds to step S35.

  (S33) If all the ruled line items have not been referred to, it is determined whether the two extracted ruled line items are connected. As described with reference to FIG. 8, it is determined whether the positional relationship between the two ruled line items is a connection relationship. If there is no connection relationship, the process returns to step S32.

  (S34) When two ruled line items are connected, other ruled line items are reflected in the group information table 1100 to which the one ruled line item belongs. As shown in FIG. 33, the group information table 1100 includes a group number GN, a start coordinate SC, an end coordinate EC, a ruled line item number RIN, a block number BLN, a logical line number LR, a leading ruled line item address HLA, and a leading logical line information address. Consists of HRA and head block address HBA. The group rectangle coordinates SC and EC and the ruled line item number RIN in the group information table 1100 are updated. Further, a group number is set in the ruled line item table 1104. Then, the process returns to step S32.

  (S35) If it is determined that all ruled line items have been referenced, it is determined whether there is a ruled line item to be connected to one connected ruled line. When there is a ruled line item to be connected, the process returns to step S31. If there is no ruled line item to be connected, group information is newly created in the group information table 1100. Then, the process returns to step S31.

  In the form example in FIG. 2, as shown in the group information table 1100 in FIG. 33, the form is classified into two groups.

  In this way, each ruled line item is classified into a group in order to separate the ruled line item from the front portion and the others.

(C) Explanation of component element classification processing FIGS. 34 and 35 are flow charts of component element classification processing, and FIGS. 36 and 37 are explanatory diagrams of tables after classification processing.

  (S40) Group information is extracted from the group information table 1100. Determine whether all group information has been referenced. If it is determined that all group information has been referenced, the process ends.

  (S41) The horizontal ruled lines of the group are extracted from the horizontal ruled line table 1105. It is determined whether all the horizontal ruled lines of the group have been referred to. If it is determined that all the horizontal ruled lines of the group have been referenced, the process proceeds to step S43.

  (S42) The horizontal ruled lines are compared, and the horizontal ruled lines having the minimum Y coordinate and the maximum Y coordinate are extracted from the horizontal ruled lines reaching the left and right of the group. The process returns to step S41.

  (S43) The vertical ruled line information is extracted from the vertical ruled line table 1106. It is determined whether all vertical ruled lines have been referenced. If it is determined that all the vertical ruled lines have been referenced, the process proceeds to step S45 (FIG. 35).

  (S44) Two vertical ruled lines having intersections with the left and right ends of the horizontal ruled line having the minimum Y coordinate and the maximum Y coordinate are extracted. The process returns to step S43.

  (S45) The ruled line items constituting the group are referred to from the ruled line item table 1104. It is determined whether all ruled line items constituting the group have been referenced. When all the ruled line items of the group are referred to, the process returns to step S40 in FIG.

  (S46) In step S42 and step S44, two horizontal ruled lines and two vertical ruled lines constituting the maximum rectangle of the group are extracted. Therefore, it is determined whether the ruled line item is within this maximum rectangle. That is, it is determined whether the ruled line item is located within a range (circumscribed rectangle) surrounded by the extracted horizontal ruled line and vertical ruled line.

  (S47) If the ruled line item is within the range, the ruled line item is made to belong to the group. Further, when the start coordinates of the group information and the ruled line item match, the first ruled line item address HLA of the group information is set to the ruled line item. Then, the process returns to step S45.

  (S48) If the ruled line item is not within the range, the start Y coordinate of the ruled line item is compared with the circumscribed rectangle start Y coordinate. When the start Y coordinate of the ruled line item is larger than the start Y coordinate of the circumscribed rectangle, the ruled line item is classified into an upward convex group. If an upward convex group has been created, it is determined whether it is connected to the ruled line item of the upward convex group. When the ruled line item is connected to the ruled line item of the upward convex group, the ruled line item is added to the group.

  The same applies to the downward convex group. When the start Y coordinate of the ruled line item is smaller than the start Y coordinate of the circumscribed rectangle, the ruled line item is classified into the downward convex group. If a downward convex group has been created, it is determined whether it is connected to a ruled line item of the downward convex group. When the ruled line item is connected to the ruled line item of the downward convex group, the ruled line item is added to the group. Then, the process returns to step S45.

  In this manner, each group is classified into the constituent elements of the main surface portion, the upper convex portion, and the lower convex portion. In the form example of FIG. 2, the forms classified into two groups are classified into four groups. That is, as shown in the group information table 1100 in FIG. 37, there are four groups. Then, as shown in the ruled line item table 1104 in FIG. 36, the ruled line items are also classified into four groups. That is, there are four group numbers.

(D) Description of Logical Row Classification Processing FIG. 38 is a logical row classification processing flowchart, FIG. 39 is a flowchart of nested analysis processing of FIG. 38, and FIGS. 39 and 40 are explanatory diagrams of tables after logical row extraction. .

  (S50) Group information is extracted from the group information table 1100. It is determined whether all group information has been referenced. When all group information is referenced, the process ends.

  (S51) Two horizontal ruled lines having intersections at the left and right ends of the group are extracted from the horizontal ruled line table 1105. It is determined whether all horizontal ruled lines have been extracted. When all the horizontal ruled lines of the group are extracted, the process returns to step S50.

  (S52) When all horizontal ruled lines are not referred to, the group is formed from the ruled line item table 1104, and ruled line items located on the two horizontal ruled lines are extracted. It is determined whether all ruled line items have been extracted. When all the ruled line items of the group are referred to, the process returns to step S51.

  (S53) It is determined whether the horizontal ruled line coordinate located below of the two horizontal ruled lines matches the lower Y coordinate of the ruled line item.

  (S54) If they do not match, there is a possibility of a nested structure as shown in FIGS. 12 (A) and 12 (B). For this reason, a nested structure analysis process shown in FIG.

  (S55) A logical row table 1102 is created for classification into logical rows. If the logical line table has been created, the ruled line item is added. If a logical row table has not been created, a new logical row table is created.

  In this way, ruled line items are classified into logical lines in each group.

  Next, the nested structure analysis process will be described with reference to FIG.

  (S56) Two ruled line items are extracted from the ruled line item table. For example, the first time is the first and second, and the second time is the second and third.

  (S57) It is determined whether the start Y coordinate of the ruled line item matches the Y coordinate of the upper horizontal ruled line of the logical line, and the end Y coordinate of the ruled line item matches the Y coordinate of the lower horizontal ruled line of the logical line. If they match, the process ends.

  (S58) If they do not match, the start X coordinates of the two ruled line items are compared. If the start X coordinates of the two ruled line items match, the process proceeds to step S59. If the start X coordinates of the two ruled line items match, a parent-child relationship is set for the two ruled line items as shown in FIG. Then, the process returns to step S56.

  (S59) If the X coordinates do not match, the start Y coordinates of the two ruled line items are compared. If the start Y coordinates of the two ruled line items do not match, the process returns to step S56. If the start Y-coordinates of the two ruled line items match, a sibling relationship is set for the two ruled line items as shown in FIG. Then, the process returns to step S56.

  As shown in FIG. 41, a logical row table 1102 is created. The logical row table includes the number LN, the start coordinate SC, the end coordinate EC, the group number GN, the block number BN, the ruled line item number RIN, the logical column number LL, the upper horizontal ruled line number ULN, and the lower horizontal ruled line number LLN. , Left vertical ruled line number ERN, right vertical ruled line number RRN, and first ruled line item address HLA. In the example of the form in FIG. 2, a logical row table 1102 is created as shown in FIG.

  Further, the group information table 1100 is added with the number of logical rows LR and the head logical row address HLA. As shown in FIG. 40, the nesting information IN of the ruled line item table 1104 includes a nesting flag INF, a hierarchy number KN, a child ruled line item number CRN, and a ruled line item link number RLN. The nesting flag INF is “ON” when the ruled line items constitute a nesting structure. In the form example in FIG. 2, the ruled line items 10, 11, and 12 constitute a nested structure, and thus are “ON”.

  The hierarchy number KN indicates the depth of the nested structure hierarchy (number of parent-child relationships). In the form example of FIG. 2, “0” is set for the ruled line item 10 as the parent, and “1” is set for the ruled line items 11 and 12 as the children. The child ruled line item number CRN sets a child ruled line item number to the parent ruled line item when the ruled line item has a parent-child relationship. In the form example of FIG. 2, the child ruled line item number 11 is set as the parent ruled line item 10.

  The rule line item link number RLN indicates the serial number of a ruled line item in a sibling relationship when the ruled line item has a sibling relationship. In the form example in FIG. 2, the ruled line items 11 and 12 are in a sibling relationship, so “0” is set in the ruled line item 11 and “1” is set in the ruled line item 12.

  In this way, each ruled line item is classified into logical lines. This logical line helps find lines that have the same structure.

  Next, a table integration process when the tables are separated as in the form example 2 in FIG. 3 will be described.

  FIG. 42 is a table integration processing flowchart, FIGS. 43 and 44 are table explanatory diagrams before group integration of the form example 2, and FIGS. 45 and 46 are table explanatory diagrams after group integration of the form example 2. .

  (S60) Two groups having the same start Y coordinate and end Y coordinate are extracted from the group information table 1100.

  (S61) The logical row information constituting the two groups is extracted one by one. Determine if all logical rows have been referenced. If all the logical lines have been referenced, the process proceeds to step S63.

  (S62) From the extracted logical line information, the number of logical lines having the same start Y coordinate and end Y coordinate is counted. Then, the process returns to step S61.

  (S63) The ratio of the number of matched logical lines to the total number of logical lines is calculated. Then, the ratio is compared with a predetermined threshold value. If the ratio is less than or equal to the threshold, the two groups are judged as separate tables and the process ends.

  (S64) If the ratio is equal to or greater than the threshold, it is determined that the two groups are one table. Then, it is determined whether there are logical lines having different structures in the two groups. If there are logical lines having different structures, a ruled line creation process described in FIG. 47 is performed.

  (S65) The two groups are integrated into one group. And it ends.

  This will be described with reference to the form example in FIG. In the form example of FIG. 3, the ruled line item table 1104 is set as shown in FIG. The logical row table 1102 and the group information table 1100 are set in FIG. That is, the group is classified into four. By the integration process described above, the groups 2 and 4 are integrated. That is, as shown in FIG. 45, the logical row of group 4 is deleted in the logical row table 1102. Also in the group table 1100, the group 4 is deleted. Similarly, in the logical row table 1102 and the group information table 1100, the number of ruled line items in group 4 is added to the number of ruled line items in group 2. In the ruled line item table 104, the group number of the ruled line item of group 4 is changed to group 2. Further, in the horizontal ruled line table 1105, the end X coordinate of the horizontal ruled line constituting the logical row of group 2 is extended to the end X coordinate of group 4.

  In this way, it is determined whether the logical row structures of the groups are the same or not, and the tables are integrated.

  Next, the ruled line creation process of FIG. 42 will be described. 47 is a ruled line creation processing flowchart, FIG. 48 is a ruled line creation explanatory diagram, and FIGS. 49 and 50 are table explanatory diagrams after the ruled line created.

  (S66) The logical row information is extracted from the two groups. Determine whether all logical lines have been referenced. If all logical lines have been referenced, the process ends.

  (S67) It is determined whether the heights of the two logical lines match. If they match, the ruled line creation process is unnecessary, and the process returns to step S66. If they do not match, the logical line information of the next line is extracted. Then, it is determined whether the sum of the height of the next row B and the row A of one group matches the height of the row C of another group. As shown in FIG. 48, it is assumed that logical rows A and B of group 2 correspond to logical row C of group 4. In this case, in the logical row C of the group 4, the ruled line disappears due to the curl. In this case, the sum of the height of the next row B of group 2 and the height of row A matches the height of row C of group 4. If not, the process returns to step S66.

  (S68) If they match, a horizontal ruled line having the same Y coordinate as the boundary line between row A and row B is generated in row C. Then, one line is divided into two lines. Then, the process returns to step S66.

  In this way, ruled lines that disappear due to curling or the like are detected from the table structure. Then, ruled lines are created. As shown in FIG. 49, in the ruled line item information table 1104, the lower horizontal ruled line number of the ruled line item 158 and the ruled line item 159 is changed to the newly created horizontal ruled line number. The end Y coordinate is changed to the coordinate of the horizontal ruled line. A ruled line item 160 and a ruled line item 161 are newly created. In the logical row table 1102 of FIG. 50, row C is divided into two vertically. Add the lower logical line to the logical line table. In the group table 1100, the number of logical rows in group 4 is incremented by one.

(E) Description of Block Classification Processing FIG. 51 is a block classification processing flowchart, and FIGS. 52 and 53 are explanatory diagrams of tables after block classification.

  (S70) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process ends.

  (S71) Two lines of logical line information constituting the group are extracted. For example, the first time extracts the first and second lines, and the second time extracts the second and third lines. Determine whether all logical rows have been extracted. If all the logical rows of the group have been extracted, the process returns to step S70.

  (S72) The structures of the two extracted logical rows are compared. This comparison compares the following items: Are the number of ruled line items configured the same? Does the arrangement of ruled line items match in the logical line? Are the logical lines the same height and width?

  (S73) If the logical row structures do not match, it is determined that both logical rows constitute another block. Therefore, block information that does not link both logical rows is created. Then, the process returns to step S70.

  (S74) If the logical row structures match, it is determined that both logical rows constitute the same block. Therefore, block information that links both logical rows is created. Then, the process returns to step S70.

  In this manner, logical rows having the same row structure are classified into one block for each group. As shown in FIG. 53, the block information table 1101 includes a block number BN, a start coordinate SC, an end coordinate EC, a group number GN, a ruled line item number RIN, a logical row number LN, a logical column number RO, a top logical row address HRA, It consists of a head logical column address HRA, an upper horizontal ruled line number ULN, a lower horizontal ruled line number LLN, a left vertical ruled line number LVN, and a right vertical ruled line number RVN.

  The block information table 1101 includes a start coordinate SC, an end coordinate EC, a group number GN, a ruled line item number RIN, a logical line number LN, a top logical line address HRA, an upper horizontal ruled line number ULN, a lower horizontal ruled line number LLN, A left vertical ruled line number LVN and a right vertical ruled line number RVN are set.

  Then, as shown in FIG. 52, the block number BN of each ruled line item is set in the ruled line item table 1104. A block number BN for each logical row is set in the logical row table 1102. As shown in FIG. 53, the block number BLN and the head block number address of each group are set in the group table 1100.

  Next, processing for integrating blocks classified into different blocks in block classification will be described. 54 is an explanatory diagram of the ruled line recovery processing, FIG. 55 is a flowchart of the ruled line recovery processing, FIGS. 56 and 57 are table explanatory diagrams before the ruled line recovery, and FIGS. 58 to 60 are table explanatory diagrams after the ruled line recovery. It is.

  As shown in FIG. 54, it is assumed that the ruled line disappears due to the ruled line item 63 and the ruled line item 64 in the form example 1 of FIG. This portion is classified as block B because the structure is different from the upper and lower logical rows. In addition, since the block B has a structure with one logical line, the order of extracting ruled line items is different from the other lines. The processing will be described below with reference to FIG.

  (S75) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process is terminated.

  (S76) The block information of the group is extracted from the block information table 1101. Check whether all block information has been extracted. If all blocks have been extracted, the process proceeds to step S79.

  (S77) The next block information is extracted from the block information table 1101. Check whether all block information has been extracted. If all blocks have been extracted, the process returns to step S76.

  (S78) The structures of the logical lines constituting the two block information are compared. That is, it is determined whether the number of ruled line items constituting the logical row constituting the block is the same. It is determined whether the arrangement of ruled line items in the logical line matches. It is determined whether the logical line has the same height and width. If they match, the number of blocks is counted. Then, the matching block information is marked. Then, the process returns to step S77.

  (S79) From the block information table 1101, a block with the largest number of matches is set as a reference block, and a block with the smallest number of matches is extracted. Referring to FIG. 54, the block A is extracted as the reference block, and the block B is extracted as the block having the smallest number of matches. Check whether all block information has been extracted. If all blocks have been extracted, the process returns to step S75.

  (S80) The arrangement of ruled line items in the logical rows of the reference block A and the block B is compared. That is, the coordinates of the ruled line item are converted into relative coordinates from the logical line start X coordinate and Y coordinate, and compared. Then, when the arrangement of the ruled line items is different, the height h1 of the ruled line item in the block A is compared with the height h2 of the ruled line item in the block B. In this comparison, if h1 <h2, it is determined that the ruled line item disappears. Therefore, the ruled line item of the block B is provisionally divided so as to coincide with the height of the ruled line item of the block A, and logical row information is provisionally created for the block B. When the logical row of block A matches the logical row of provisionally created block B, the group information, block information of block B, logical row information, ruled line item information, and horizontal ruled line information are edited. Then, the process returns to step S79.

  This will be described with reference to the example of FIG. The table before the ruled line recovery is as shown in FIGS. That is, as shown in the ruled line item table 1104 in FIG. 56, the ruled line item 59 to the ruled line item 66 are classified into the block 3. The ruled line items 47 and 48 are classified into the block 2.

  After the ruled line is restored, the end coordinates of the ruled line items 63 and 64 are changed to the coordinate values of the horizontal ruled line 12 in the ruled line item table 1104 as shown in FIG. Ruled line items 177 and 178 are newly created. The block number of the ruled line item having the horizontal ruled line 11 on the upper side and the horizontal ruled line item 12 on the lower side and the ruled line item having the horizontal ruled line 11 on the upper side and the horizontal ruled line 13 on the lower side is changed to the block 2. The logical line number of the ruled line item is changed to the serial number in block 2. The lower horizontal ruled line number of the ruled line item whose lower side is the horizontal ruled line 13 is changed to “12”. The table 1104 is sorted so that the ruled line numbers in the same group are in the logical row order. The table is sorted so that ruled line items having the same logical line number are in the order of the X coordinate. The block number of the ruled line item after group 3 and block 4 is changed to block 2, and the logical row number is changed to the serial number in block 2.

  In the logical row table 1102 of FIG. 59, the logical row information of group 3 and block 3 is changed to a range surrounded by horizontal ruled lines 11 and 12. The block number of the logical row information of group 3 and block 3 is changed to block 2. The logical line number is changed to the serial number (4) in block 2. The number of ruled line items in group 3 and block 2 is changed to “14”, and the lower horizontal ruled line number is changed to “12”. Create new logical row numbers for group 3 and block 2. The block number of the logical row information after group 3 and block 4 is changed to block 2. Change the logical line number to the serial number in block 2.

  In the block information table 1101 of FIG. 59, the number of ruled line items in group 3 and block 2 is changed to “140”, the number of logical lines is changed to “10”, and the lower horizontal ruled line number is changed to “19”. Block 3 and block 4 of group 3 are deleted from the table.

  In the group information table 1100 of FIG. 60, the number of blocks in group 2 is changed to “2”, and the number of ruled line items is changed to “147”. In the horizontal ruled line table 1105, the end Y coordinate of the horizontal ruled line 12 is changed to the same coordinate value as that of the horizontal ruled line 11. The horizontal ruled line 13 is deleted.

  In this way, the logical row structure is compared between adjacent different blocks. As a result, a block similar in structure to the reference block is searched, and ruled lines are added so that the similar block becomes the logical row structure of the reference block. Thereby, the drowned ruled line is recovered.

  Next, a process of integrating the ones in which the erased lines are entered in the ruled line items of the form and are classified into different blocks into the same block will be described. 61 is an explanatory diagram of logical row integration processing, FIG. 62 is a logical row integration processing flow diagram, FIGS. 63 and 64 are table explanatory diagrams before filter row integration, and FIGS. 65 to 67 are after logical row integration. FIG.

  FIG. 61 shows an example in which a strike line is entered in the ruled line item of the form, and a horizontal ruled line is entered as a erased line between the ruled line item 59 and the ruled line item 73. In the original form, the ruled line item 59 and the ruled line item 73 are combined into one ruled line item. One ruled line item is divided into two by a strike line. The two divided lines are combined into one line. Further, in order to correct the contents of entry, even when the ruled line item is partially divided by the drawn correction line, the analysis can be performed by the integration process.

  This will be described with reference to the processing flow of FIG.

  (S81) Steps S75 to S79 of the ruled line recovery process of FIG. 55 are executed to extract blocks having a different number of matches from the reference block. It is determined whether the block being referred to is a reference block. If the referenced block is a reference block, the process returns to step S79 (FIG. 55).

  (S82) The logical row information of the reference block is extracted from the logical row table 1102. The height of the logical line below it is added to the height of the first logical line of the reference block one line at a time. The logical line is referenced until the added height matches the height of the reference block. If they do not match, the process returns to step S79 (FIG. 55).

  (S83) If they match, all ruled line items connected vertically are integrated among the ruled line items constituting the referenced logical row. That is, ruled line items having the same Y coordinate and a shared Y coordinate are integrated. Further, a logical line composed of integrated ruled line items is provisionally created. Then, the created logical row is compared with the logical row structure of the reference block. When the logical row structures match, group information, block information, logical row information, and ruled line item information are edited. Then, the process returns to step S79 (FIG. 55).

  In this way, the reference block and another block having a structure different from that of the reference block are extracted, and the presence of a strikethrough is detected from the height of the other block and the height of the reference block. When the presence of the strikethrough is detected, the ruled line items positioned above and below are integrated to temporarily create a logical line. Compare the structure of the temporary logical line with the structure of the logical line of the reference block. For this reason, the erased line and the corrected line can be removed.

  When a strike line is drawn as shown in FIG. 61, the ruled line item table 1104 is created as shown in FIG. Each of the logical row information table 1102, the block information table 1101, and the group information table 1100 is created as shown in FIG.

  When the strikethrough in FIG. 61 is removed and blocks A, B, and C are integrated into block A, the end coordinates from ruled line item 59 to ruled line item 72 are displayed in ruled line item table 1104 as shown in FIG. The lower horizontal ruled line number is changed to “13” to the coordinate value of the horizontal ruled line 13. The block number from the ruled line item 59 to the ruled line item 72 is changed to block 2. The logical line numbers from the ruled line item 59 to the ruled line item 72 are changed to serial numbers in the block 2. The ruled line item 73 to the ruled line item 86 are deleted. The block number after the ruled line item 101 is changed to the block 2, and the logical line number is changed to the serial number in the block 2.

  As shown in FIG. 66, in the logical row information table 1102, the logical row end Y coordinate of group 3 and block 3 is changed to the Y coordinate of horizontal ruled line 13, and the lower horizontal ruled line number is changed to “13”. The block number of group 3 and block 3 is changed to block 2, and the logical row number is changed to serial number (4) in block 2. The block number after group 3 and block 4 is changed to block 2, and the logical row number is changed to the serial number in block 2.

  As shown in FIG. 66, in the block information table 1101, the number of ruled line items of group 3 and block 2 is changed to “140”, the number of logical lines is changed to “10”, and the lower horizontal ruled line number is changed to “19”. Block 3 and block 4 of group 3 are deleted from the table.

  As shown in FIG. 67, in the group information table 1100, the number of ruled line items in group 2 is changed to “147”, the number of blocks is changed to “2”, and the number of logical rows is changed to “11”. In the horizontal ruled line table 1105, the horizontal ruled line 12 is deleted.

(F) Description of Logical Column Determination Processing FIG. 68 is a logical column determination processing flowchart, and FIGS. 69 to 71 are explanatory diagrams of tables after logical column determination. The logical column determination processing will be described with reference to FIG.

  (S90) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process ends.

  (S91) The block information of the two blocks of the extracted group is extracted from the block information table 1101. It is determined whether all the block information of the extracted group has been extracted. If all the block information has been extracted, the process returns to step S90.

  (S92) Ruled line item information constituting the first logical row of each extracted block is sequentially extracted from the ruled line item table 1104. It is determined whether all ruled line item information of the extracted logical line has been referenced. When all the ruled line item information is referred to, it is determined whether the nesting information is set in step S97 described later. If no nesting information is set, the logical sequence between the two extracted blocks is completed, and the process returns to step S91. When nesting information is set, group information, block information, and logical row information are updated. Furthermore, each piece of information is also updated in the logical rows of two or less in the reference block, as in the first row. Then, the process returns to step S91.

  (S93) It is determined whether the ruled line items of the two logical rows have the same width. If they are not the same width, the next ruled line item is extracted on the narrow logical line side. Then, the widths of the current ruled line item and the next ruled line item are added on the narrow logical line side. The ruled line item of the wide logical line is compared with the added value to determine whether the width is the same. The extraction of ruled line items is repeated until the same width is obtained.

  (S94) In this way, the correspondence in the column direction between the ruled line item of the first logical row of the first block and the ruled line item of the first logical row of the second block has been found.

  Next, the number of extracted ruled line items is counted.

  (S95) In order to detect the nested structure, it is determined whether or not the number of extracted ruled line items is “1”.

  (S96) When the number of ruled line items is “1”, it is not a nested structure. Therefore, a serial number from the left side is set as a logical column number in the extracted ruled line item.

  (S97) When the number of ruled line items is not “1”, a nested structure is used. For this reason, the same serial number is set as the logical column number in the nested ruled line item. Then, the nesting information is set in the ruled line items constituting the nesting structure.

  (S98) Logical column information is created, and a link is set between ruled line items in the same logical column. At this time, a link is set between different blocks in the same group and the same column number set between logical rows. Then, the process returns to step S92.

  In this way, it is determined whether the blocks connected in the vertical direction have the same column structure. If the same column structure is used, these blocks are integrated in units of logical columns.

  This will be described with reference to the form example in FIG. As shown in FIG. 69, in the ruled line item information table 1104, a logical column number is set in the column number column RO of the ruled line item. For example, the column number is set to “1” for the ruled line items 10, 11, and 12 of the block 1. In different blocks 2, the ruled line items having the same width as the ruled line number are “17” and “18”. Accordingly, the same column number “1” is set in the ruled line items 17 and 18 of the block 2.

  Next, the nesting information IN is set in the ruled line items constituting the nesting structure. In FIG. 69, nesting information is set in the ruled line items 17 and 18. Similarly, nesting information is also set for ruled line items (for example, ruled line items 31 and 32) constituting the same logical column. Further, an address link HAA is set between ruled line items constituting the same logical column.

  In this case, the nesting information IN includes a nesting flag INF, a hierarchy number KN, a child ruled line item number CRN, and a ruled line item link serial number RLN. In the example of the ruled line items 17 and 18, since the ruled line items 17 and 18 are nested, the nesting flag INF is turned on. Since there is no parent-child relationship between the two ruled line items 17 and 18, “0” is set in the hierarchy number KN. Since there is no parent-child relationship between the two ruled line items 17, 18, "0" is set to the child ruled line item number CRN. Since the ruled line items 17 and 18 are siblings, “0” is set to the link number RLN between ruled line items of the ruled line item 17 and “1” is set to the link number RLN between ruled line items of the ruled line item 18.

  As shown in FIG. 70, in the logical row information table 1102, the logical column number LL of the logical row 2 is changed to “5”. A logical column information table 1103 shown in FIG. 70 is newly created. The logical column information table 1103 includes a logical column number RO, a start coordinate SC, an end coordinate EC, a group number GN, a block number BN, a ruled line item number RIN, an upper horizontal ruled line number ULN, a lower horizontal ruled line number LLN, and a left vertical ruled line. It consists of a number ELN, a right vertical ruled line number RRN, and a first ruled line item address HLA. These are created according to the relationship in the column direction described above.

  Further, as shown in FIG. 71, in the block information table 1101, the number of logical columns of group 3 and block 2 is changed to “5”. Then, the head logical column address HAA of each block is set.

  In this way, the relationship in the column direction of ruled line items between blocks is determined in each group. In accordance with this, the table describing the relationship in the column direction is updated.

  Next, the 1-digit and 3-digit ruled line nesting analysis processing described with reference to FIGS. 20A and 20B will be described. FIG. 72 is a flowchart of the nesting analysis process for 1-digit and 3-digit ruled lines, and FIG.

  (S100) Logical row information is extracted from the logical row table 1102. It is determined whether all logical line information has been extracted. If all logical line information has been extracted, the process ends.

  (S101) A ruled line item to be linked is extracted from the logical line information. It is determined whether all ruled line items have been extracted. If all ruled line items have been extracted, the process returns to step S100.

  (S102) It is determined whether the size of the extracted ruled line item corresponds to the size of the one-digit ruled line item. If applicable, the process proceeds to step S104. If not, the process proceeds to step S103.

  (S103) It is determined whether the size of the extracted ruled line item corresponds to the size of the three-digit ruled line item. If applicable, the process proceeds to step S104. If not, the process returns to step S101.

  (S104) The next ruled line item is extracted from the logical line being referred to. The size of the current ruled line item and the next ruled line item are compared. It is determined whether they match within the error range. If they match, the process proceeds to step S105. If they do not match, the process returns to step S101.

  (S105) The next ruled line number is set as the link ruled line number of the current ruled line number. Then, the process returns to step S101.

  FIG. 73 shows a state after updating the ruled line item table 1104 in the form example of FIG. In FIG. 2, the one-digit ruled line items are ruled line items 22-30, 36-44, and the like. With the above-described processing, these ruled line items 22 to 30 are determined as one-digit ruled line items. In the ruled line item table 1104, the ruled line number to be linked is set to the linked ruled line item numbers AN of the one-digit ruled line items 22 to 30, as shown in a shaded state. By the above-described nesting process, the ruled line numbers 22 to 30 are determined to have a nesting structure, and nesting information IN is set.

  In this way, the vertical size and horizontal size of the 1-digit ruled line item or the 3-digit ruled line item are defined in advance. When an arbitrary number of ruled line items that match these sizes are arranged, they are nested and logically combined as one ruled line item. By using such a result, in the character recognition process, continuous ruled line items can be handled as one character string.

  Next, processing for analyzing a rectangular nested structure having the special structure shown in FIGS. 21A and 21B will be described. FIG. 74 is a flowchart of the nested analysis process other than the main table, and FIGS. 75 and 76 are explanatory diagrams of the table after the nested analysis.

  (S110) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process ends.

  (S111) The structure of the extracted group is determined. A group is determined to be one block and one row. If the group is one block and one row, the process proceeds to step S114. If the group is not one block and one row, it is determined whether the group is one column. If the group is not one column, the process returns to step S110. If the group is one column, it is determined whether the group has a structure of a plurality of blocks. If the group is not a multi-block structure, the process returns to step S110. If the group has a structure of a plurality of blocks, the process proceeds to step S112.

  (S112) The ruled line items constituting the group are extracted. It is determined whether all ruled line item information has been referenced. If all ruled line item information is referred to, the process returns to step S110.

  (S113) In the group, the ruled line items next to the current reference ruled line item are sequentially extracted. Then, the left vertical ruled line number of the current ruled line item and the next ruled line item are compared. If the left vertical ruled line numbers of both ruled line items match, nesting information (parent-child relationship) is set between the ruled line items. Next, the upper horizontal ruled line number of the current ruled line item and the next ruled line item are compared. If the upper horizontal ruled line numbers of both ruled line items match, nesting information (sibling relationship) is set between the ruled line items. Thereby, the nested structure shown in FIG. 21B is detected. Then, the process returns to step S112.

  (S114) If the group is one block and one row, the ruled line items constituting the group are extracted. It is determined whether all ruled line item information has been referenced. If all ruled line item information is referred to, the process returns to step S110.

  (S115) In the group, the ruled line items next to the current reference ruled line item are sequentially extracted. Then, the upper horizontal ruled line number of the current ruled line item and the next ruled line item are compared. When the upper horizontal ruled line numbers of both ruled line items match, nesting information (parent-child relationship) is set between the ruled line items. Next, the left vertical ruled line number of the current ruled line item and the next ruled line item are compared. If the left vertical ruled line numbers of both ruled line items match, nesting information (sibling relationship) is set between the ruled line items. Thereby, the nested structure shown in FIG. 21A is detected. Then, the process returns to step S114.

  In this way, the structure other than the main table on the form is subjected to nested structure analysis processing for each structure. In FIG. 21A, ruled line item 2 has a parent-child relationship with ruled line item 1. The ruled line item 4 has a parent-child relationship with respect to the ruled line item 2. The ruled line item 5 has a parent-child relationship with respect to the ruled line item 3. The ruled line item 3 has a sibling relationship with the ruled line item 2. The ruled line item 5 has a sibling relationship with the ruled line item 4. In FIG. 21B, the ruled line item 2 has a parent-child relationship with respect to the ruled line item 1. The ruled line item 4 has a parent-child relationship with respect to the ruled line item 2. The ruled line item 5 has a parent-child relationship with respect to the ruled line item 3. The ruled line item 3 has a sibling relationship with the ruled line item 2. The ruled line item 5 has a sibling relationship with the ruled line item 4.

  Each table will be described in the form example of FIG. As shown in FIG. 75, in the ruled line item table 1104, the nesting flag INF is set for the ruled line items 1 to 5. For ruled line numbers 2 and 3, the nested hierarchy number KN is set to “1”, and for ruled line numbers 4 and 5, the nested hierarchy number KN is set to “2”. In the child hierarchy number CRN, “2” is set to the ruled line number 1, “4” is set to the ruled line number 2, and “5” is set to the ruled line number 3. The ruled line item link serial number RLN is set to “1” for the ruled line numbers 3 and 5.

  As shown in FIG. 75, in the logical row information table 1102, the number of logical columns “1” is set in the logical row 1 of group 1 and block 1. As shown in FIG. 76, in the logical column information table 1103, the logical columns 2 and 3 are deleted, and the number of ruled line items of the logical column number 1 is changed to “5”. As shown in FIG. 76, in the block information table 1101, the number of logical columns of group 1 and block 1 is changed to “1”.

  In this way, the nesting structure of the part other than the main table is analyzed.

(G) Description of attribute assigning process FIG. 77 is an attribute process flowchart, and FIG. 78 is an explanatory diagram of a table after attribute assignment.

  (S120) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process ends.

  (S121) It is determined whether the group is composed of a plurality of blocks. If it is not composed of a plurality of blocks, the process returns to step S120.

  (S122) Block information is extracted from the block information address of the group information, and a block attribute BZ is assigned to each block. That is, the header block attribute is assigned to the first block, the body block attribute is assigned to the second block, and the footer block attribute is assigned to the third and subsequent blocks. Then, the process returns to step S120.

  In this way, attributes are assigned to the blocks constituting the table structure section so that each block has a meaning. As shown in FIG. 78, in the block information table 1101, the header block attribute “1” is assigned to the first block 1 of the group 3, and the body block attribute “2” is assigned to the second block 2. Then, the footer block attribute “3” is assigned to the third block 3.

(H) Description of Heading Flag Setting Process Next, the heading flag setting process for setting the heading flag for the ruled line items constituting the heading will be described.

  FIG. 79 is a flag setting process flow diagram, and FIG. 80 is an explanatory diagram of a table after flag setting.

  (S130) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process ends.

  (S131) The block information addressed is extracted from the group information. It is determined whether all block information in the group has been extracted. If all block information in the group has been extracted, the process returns to step S130.

  (S132) It is determined whether the extracted block is a header block. If it is a header block, the logical row information of the block is extracted one record at a time. The ruled line item information in the logical line is extracted for each record, and the heading flag is turned on. Thereby, a heading flag is set for all ruled line items constituting the header block. Then, the process returns to step S131.

  (S133) It is determined whether the extracted block is a footer block. If it is not a footer block, the process proceeds to step S134. If it is a footer block, the logical row information of the block is extracted one record at a time. The ruled line item information in the logical line is extracted for each record, and the heading flag is turned on. As a result, the heading flag is set for all ruled line items constituting the footer block. Then, the process returns to step S131.

  (S134) It is determined whether the extracted block is a body block. If it is not a body block, the process returns to step S131. If it is a body block, the first logical row information of the block is extracted for each record. The ruled line item information in the extracted first logical line is extracted for each record, and the heading flag is turned on. As a result, the heading flag is set for all ruled line items constituting the first logical line of the body block. Then, the top logical column information of the corresponding block is extracted. The ruled line item information in the extracted first logical column is extracted for each record, and the heading flag is turned on. As a result, the heading flag is set in the ruled line item of the first logical column. Then, the process returns to step S131.

  If this headline is extracted, the characters used can be restricted, and character recognition processing specialized for the characters (words) can be performed. Therefore, it is possible to recognize the wording of the heading part with high accuracy. As shown in FIG. 80, in the ruled line item table 1102, a header flag MF is set for the ruled line item estimated to be a header in accordance with the rule of FIG.

(I) Description of Character Recognition Process for Heading Part Next, in order to determine the type of the heading, character recognition is performed on a character string existing in the ruled line item in which the heading flag is set.

  FIG. 81 is a flowchart for headline recognition processing, and FIG. 82 is an explanatory diagram of a table after headline recognition.

  (S140) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process ends.

  (S141) Extract block information addressed from group information. It is determined whether all block information in the group has been extracted. If all block information in the group has been extracted, the process returns to step S140.

  (S142) It is determined whether the block information is a body block. If it is a body block, it returns to step S141 because it is a data portion.

  (S143) If it is not a body block, there is a possibility of a heading part. For this reason, the logical row information addressed is extracted from the block information. It is determined whether all logical row information has been extracted. If all the logical lines have been extracted, the process returns to step S141.

  (S144) The ruled line item information addressed is extracted from the logical line information.

  (S145) Then, it is determined whether the header flag of the ruled line item information is on. If the heading flag is not on, the process returns to step S144.

  (S146) The image of the character string data portion in the ruled line item is cut out. Then, the extracted image is recognized using the headline wording dictionary 1003 (see FIG. 25). As shown in FIG. 25, the headline wording dictionary 1003 is registered with words (transfer destinations and the like) existing as headings and their images. Therefore, character recognition can be performed with higher accuracy than when a general character recognition dictionary is used.

  (S147) The heading name of the matched dictionary record is extracted. Records with matching heading names are extracted from the heading table (see FIG. 24). That is, the heading table shown in FIG. 24 is referred to using the heading name obtained from the recognition result as a key. As shown in FIG. 24, attribute information (character recognition category, character font, etc.) corresponding to the heading name is registered in the heading table. Then, a record (attribute information) matching the heading name is extracted. Then, attributes (character recognition category CC, character font CF) are set in the ruled line item information. Then, the process returns to step S144.

  In this way, the headline part is recognized using the limited character recognition dictionary. As shown in FIG. 82, in the case of the form example 1 shown in FIG. 2, the character recognition category CC and the character font CF are set to the ruled line item for which the header flag in the ruled line item table 1104 is on.

(J) Description of Data Part Attribute Setting Processing Next, in order to determine the data part attribute, the attribute of the ruled line item for which the heading flag is not set is determined.

  FIG. 83 is an attribute setting process flowchart, and FIG. 84 is an explanatory diagram of a table after attribute setting.

  (S150) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process ends.

  (S151) Extract block information addressed from group information. It is determined whether all block information in the group has been extracted. If all block information in the group has been extracted, the process returns to step S150.

  (S152) The logical column information addressed is extracted from the block information. It is determined whether all logical column information has been extracted. If all logical columns have been extracted, the process returns to step S151.

  (S153) The ruled line item information addressed is extracted from the logical column information. Then, it is determined whether the header flag of the ruled line item information is on. If the heading flag is not on, the process proceeds to step S155.

  (S154) If the heading flag is on, the character recognition category and character font of the ruled line item are stored. Then, the process returns to step S153.

  (S155) If the heading flag is not on, it is a ruled line item of the data portion. It is determined whether the ruled line item information is nested information. If it is nested information, the process proceeds to step S156. If it is not nesting information, the character recognition category and character font saved in step S154 are set to the ruled line item being referred to. Thereby, the attribute of the heading part of the same logical column is set in the data part. Then, the process returns to step S153.

  (S156) If it is nested information, the character recognition category and the character font saved in step S154 are set to the ruled line item being referenced from the ruled line item at the same level (hierarchical number link serial number) as the ruled line item being referenced. . Then, the process returns to step S153.

  In this manner, the recognition characteristics such as the character recognition category and the character font are copied from the ruled line item of the heading for the ruled line items other than the heading among the ruled line items constituting the logical column. As shown in FIG. 84, in the case of the form example in FIG. 2, the character recognition category CC and the character font of the ruled line items in the same logical column are included in the ruled line items 3, 5, 7 etc. CF is set. For this reason, the data name attribute of the data portion is determined by the logical column information.

(K) Description of Character Recognition Process for Data Part Next, in order to recognize the character of the data part, the character recognition is performed for the ruled line item for which the heading flag is not set.

  FIG. 85 is a character recognition processing flowchart, and FIG. 86 is an explanatory diagram of a character recognition result.

  (S160) The character recognition result notification area management unit 1200 shown in FIG. 86 is set in the memory. The management unit 1200 includes a recognition result heading number and a recognition result heading area address.

  (S161) Group information is extracted from the group information table 1100. It is determined whether all group information has been extracted. If all group information has been extracted, the process ends.

  (S162) Extract block information addressed from group information. It is determined whether all block information in the group has been extracted. If all block information in the group has been extracted, the process returns to step S161.

  (S163) The logical column information addressed is extracted from the block information. It is determined whether all logical column information has been extracted. If all logical columns have been extracted, the process returns to step S162.

  (S164) The ruled line item information addressed is extracted from the logical column information.

  (S165) It is determined whether the header flag of the ruled line item information is on. If the heading flag is on, the process returns to step S164.

  (S166) If the heading flag is not on, the character string existing in the ruled line item is recognized using the general character recognition dictionary 1004 (see FIG. 1). At this time, the character recognition dictionary 1004 is provided separately for each recognition category such as kanji and numbers. Therefore, according to the recognition attribute of the data part described above, a recognition dictionary is selected from the general dictionary and character recognition is performed. For this reason, the data portion can also be recognized with high accuracy.

  Further, a recognition result heading area 1201 shown in FIG. 86 is set. An area 1201 includes a heading name, the number of recognition result data, a recognition result record length, and a recognition result data address. Then, the recognition result data area 1202 is set. That is, the recognition result is set in the area indicated by the recognition result data address.

  In this way, the obtained recognition result is set in the recognition result notification area. Thereby, the data name of the data part, the number of data, and the character recognition result are obtained.

  In this way, the relation between the row and the column of each ruled line item is determined, and classified into a heading part and a data part corresponding thereto. Then, the heading part is character-recognized using a heading recognition dictionary to obtain a heading name. The data name of the data part corresponding to the heading part is found from the relationship of the structure of the logical column. For this reason, it is not necessary to previously define the format of each item of the form.

  Furthermore, the recognition attribute is obtained from the heading name by the heading table. Accordingly, the data portion is recognized as characters. For this reason, the table structure of the form can be analyzed, and the accuracy of character recognition results in the data portion can be improved.

  In the above-described embodiment, the kanji, alphabetic characters, and numerals have been described as examples of entry or print characters for each form. However, only numeric characters or alphabetic characters may be used. Further, the data name, character recognition category, and character font are detected as the attributes of the ruled line item, but only the data name may be used.

  Although the embodiments of the present invention have been described above, various modifications are possible within the scope of the present invention, and these are not excluded from the scope of the present invention.

  As described above, the present invention has the following effects.

  (1) Since the arrangement of ruled line items in a form is detected and the table structure of the form is automatically analyzed, the trouble of registering the form format can be saved.

  (2) Since the form format is automatically registered, it is possible to save re-registration even if the format is changed.

1000 Reader 1001 Processor 1002 Memory 1003 Heading Dictionary 1004 General Dictionary 1005 Ruled Line Extraction Unit 1006 Table Structure Analysis Unit 1007 Attribute Determination Unit 1008 Character Recognition Unit

Claims (2)

In the form recognition method to recognize the table structure of the form,
Extracting a minimum rectangle composed of vertical and horizontal ruled lines of the form from the image of the form;
Classifying into logical rows constituting each row of the table from the connection relation in the row direction of the form of the minimum rectangle;
Combining the logical rows having the same and the same row structure into one block;
Extracting a relationship of blocks having the same column structure between the blocks connected above and below the table, and combining the blocks having the same column structure in units of logical columns;
From the table structure analyzed into the logical row, the block, and the logical column, the block having the largest number of rows is determined as a body portion, and blocks located above and below the block are defined as a header portion, a footer. A step of determining as a part;
The logical line of the header part is a heading part, the character of the heading part is character-recognized in units of the logical column, and the attribute of the heading part of the logical column unit is determined.
And determining the attribute of the minimum rectangle in the logical column of the body part from the attribute of the heading part in units of the logical column. A reader that reads the image of the form,
A memory for storing a heading dictionary for storing the wording and image data of the heading part of the form, and a general dictionary;
Extract a minimum rectangle composed of vertical and horizontal ruled lines of the form from the image of the form, analyze the table structure of the form from the physical arrangement of the minimum rectangles on the form, and from the analyzed table structure A processing unit for determining an attribute of the minimum rectangle,
The processing unit is
From the connection relation in the row direction of the form of the smallest rectangle, it is classified into logical rows constituting each row of the table, the logical rows having the same and the same row structure are grouped into one block, and the table The relationship between the blocks having the same column structure is extracted between the blocks connected above and below, and the blocks having the same column structure are grouped in units of logical columns,
From the table structure analyzed into the logical row, the block, and the logical column, the block having the largest number of rows is determined as a body portion, and blocks located above and below the block are defined as a header portion, a footer. A logical line of the header part is used as a heading part, and characters of the heading part are recognized in units of the logical column using the heading dictionary, and the heading part of the logical column unit is recognized. A form processing apparatus characterized by determining an attribute.

Images