JP6430919B2 - Ruled line frame correction method, ruled line frame correction apparatus, and ruled line frame correction program - Google Patents

Description

  The present invention relates to a ruled line frame correction method, a ruled line frame correction apparatus, and a ruled line frame correction program.

  In business, forms created with electronic files or paper are used in various scenes. A form is often composed of item names and item values for the item names. In that case, the form can be structurally expressed by logically associating the item name and the item name or the item name and the item value.

  Conventionally, using information such as the position and size of the ruled line frame of a form, between item names and item names (hereinafter, between item names), or between item names and item values (hereinafter, item names-item values) A method of automatically estimating the logical relationship between the two (2) and outputting the estimation result as tree structure data is known (for example, Non-Patent Document 1). According to this method, it is possible to link data semi-automatically between other systems and forms using the output tree structure data.

Atsuko Takagi, Nawa for many years, Tsutomu Maruyama, “Examination of extraction methods for cross-sectional data manipulation techniques for electronic forms”, IEICE technical report LOIS2014-11, July 17, 2014

  However, in the conventional method, when item names or item values are described in a form that does not satisfy the prescribed requirements, the logical relationship between item names and between item names and item values cannot be accurately estimated There was a problem that there was.

  For example, in the conventional method, there is a case where the logical relationship is estimated on the assumption that one item name or item value is described in one ruled line frame. In this case, describing one item name or item value in one ruled line frame is one of the requirements for estimating a logical relationship using a conventional method.

  At this time, for example, when both the item name and the item value are described in one ruled line frame as an example of the case where the requirement that one item name or item value is described in one ruled line frame is not satisfied There is. In this case, the conventional method cannot recognize an accurate item name or item value, and may not be able to accurately estimate the logical relationship between item names and between item names and item values.

  In addition, as another example of the case where the requirement that one item name or item value is described in one ruled line frame is not satisfied, one item name or item value may be described over a plurality of ruled line frames. . Also in this case, the conventional method cannot recognize an accurate item name or item value, and may not be able to accurately estimate the logical relationship between item names and between item names and item values.

  The ruled line frame correction method of the present invention extracts a ruled line frame from a form, and acquires at least the type or thickness of the ruled line, the character string in the frame, and the fill color in the frame as the ruled line frame information for each ruled line frame. If the ruled line frame information of the plurality of ruled line frames satisfies a predetermined ruled line frame combination condition, a combination step of combining the plurality of ruled line frames and a process by the combined step are executed. Thereafter, when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame dividing condition, a dividing step of dividing the ruled line frame, and a process by the dividing step are executed, and then the ruled line frame A deletion step of deleting the ruled line frame when the ruled line frame information satisfies a preset ruled line frame deletion condition.

  The ruled line frame correction apparatus of the present invention extracts a ruled line frame from a form, and acquires at least the type or thickness of the ruled line, the character string in the frame, and the fill color in the frame as the ruled line frame information for each ruled line frame. When the acquisition unit and the ruled line frame information of the plurality of ruled line frames satisfy a predetermined ruled line frame combination condition, processing is performed by the combination unit that combines the plurality of ruled line frames and the combination unit. Thereafter, when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame dividing condition, a division unit that divides the ruled line frame, and processing performed by the dividing unit, And a deletion unit that deletes the ruled line frame when the ruled line frame information satisfies a preset ruled line frame deletion condition.

  According to the present invention, even when item names or item values are described in a form that does not satisfy a predetermined requirement, a logical relationship between item names and between item names and item values is accurately estimated. it can.

FIG. 1 is a diagram illustrating an example of a form file and tree structure data. FIG. 2 is a diagram illustrating a configuration example of the data structure extraction device. FIG. 3A is a diagram illustrating a configuration example of the data structure extraction unit. FIG. 3B is a diagram illustrating an example of the ruled line frame to be corrected. FIG. 4A is a flowchart of the processing procedure of the data structure extraction apparatus. FIG. 4B is a flowchart of the graph generation process of S2 in FIG. FIG. 4C is a flowchart illustrating the ruled line frame correction process in S3 of FIG. FIG. 4-4 is a flowchart of the tree structure estimation process in S5 of FIG. 4-1. FIG. 5A is a flowchart illustrating an example of the identification process of the operation interface in S131 of FIG. 4-2. FIG. 5-2 is a flowchart illustrating an example of the form format information acquisition process in S132 of FIG. 4-2. FIG. 5C is a flowchart illustrating an example of the node generation process in S133 of FIG. FIG. 5-4 is a diagram for explaining an example of the node generation processing. FIG. 5-5 is a flowchart illustrating an example of the property information acquisition process in S134 of FIG. 4-2. FIG. 6 is a diagram showing an example of property information and tree structure data in the form database. FIG. 7A is a flowchart illustrating an example of the adjacent edge generation process in S135 of FIG. FIG. 7B is a diagram for explaining an example of the adjacent edge generation process. FIG. 7C is a flowchart illustrating an example of processing for obtaining adjacent edges between nodes in S1353 of FIG. FIG. 7D is a flowchart illustrating an example of processing for checking the adjacency relationship between the nodes in S1354 of FIG. FIG. 8 is a flowchart illustrating an example of the inclusion edge generation process in S136 of FIG. FIG. 9 is a flowchart illustrating an example of a processing procedure of the item name registration unit. FIG. 10A is a flowchart illustrating an example of a process for classifying the node cluster in S4 of FIG. FIG. 10-2 is a diagram illustrating an example of a node cluster. FIG. 10C is a flowchart illustrating an example of clustering processing of another node Y starting from an arbitrary node X in S1383 in FIG. FIG. 11 is a flowchart illustrating an example of the item name assignment process. FIG. 12A is a flowchart illustrating an example of the partial tree pattern generation process in S140 of FIG. 4-4. FIG. 12-2 is a diagram for explaining the hierarchy of inclusion nodes. FIG. 12C is a flowchart illustrating an example of the acquisition process of the partial tree pattern in S1405 of FIG. FIG. 12-4 is a flowchart illustrating an example of a tree structure conversion process for C (X, k) in S14056 and S14060 of FIG. 12-3. FIG. 12-5 is a diagram for explaining assignment of item attributes and modification of adjacent edges according to the above-described tabular / enumerated type estimation rules. FIG. 13 is a flowchart showing an example of the tree structure data construction process in S141 of FIG. 4-4. FIG. 14A is a flowchart illustrating an example of the tree structure data selection processing in S142 of FIG. 4-4. FIG. 14B is a flowchart illustrating an example of tree structure data selection processing in accordance with the tree structure data selection rule of S1424 in FIG. FIG. 15 is a flowchart illustrating an example of the form structure construction process in S6 of FIG. 4-1. FIG. 16A is a diagram illustrating an example of a form structure rule. FIG. 16B is a diagram illustrating an example of a form. FIG. 16C is a diagram for explaining the generation of the adjacent edge of the node X. FIG. 16D is a diagram for explaining the check of the adjacent edge of the node X. FIG. 16-5 is a diagram for explaining the inclusive edge generation of the node X. FIG. 16-6 is a diagram illustrating an example of a tree structure conversion process. FIG. 16-7 is a diagram for explaining an example of setting of the table type / enumeration type. FIG. 17 is a flowchart illustrating an example of ruled line frame combination processing. FIG. 18 is a flowchart illustrating an example of the combining process A in S233 of FIG. FIG. 19 is a flowchart illustrating an example of the combining process B in S234 of FIG. FIG. 20 is a diagram for explaining the combining process A in S233 and the combining process B in S234 in FIG. FIG. 21 is a flowchart illustrating an example of a ruled line frame dividing process. FIG. 22 is a flowchart illustrating an example of the division process A in S244 of FIG. FIG. 23 is a diagram for explaining the division processing A in S244 of FIG. FIG. 24 is a diagram for explaining the division processing A in S244 of FIG. FIG. 25 is a flowchart illustrating an example of the division process B in S245 of FIG. FIG. 26 is a diagram for explaining the dividing process B in S245 of FIG. FIG. 27 is a flowchart illustrating an example of the division process C in S246 of FIG. FIG. 28 is a diagram for explaining the division process C in S246 of FIG. FIG. 29 is a flowchart illustrating an example of ruled line frame deletion processing. FIG. 30 is a diagram for explaining ruled line frame deletion processing. FIG. 31 is a flowchart illustrating an example of ruled line frame addition processing. FIG. 32 is a flowchart showing an example of the additional process A in S263 of FIG. FIG. 33 is a diagram for explaining the additional processing A in S263 of FIG. FIG. 34 is a diagram for explaining the additional processing A in S263 of FIG. FIG. 35 is a diagram for explaining the additional processing A in S263 of FIG. FIG. 36 is a flowchart illustrating an example of the addition process B in S264 of FIG. FIG. 37 is a diagram for explaining the additional processing B in S264 of FIG. FIG. 38 is a flowchart showing another example of the ruled line frame correction process in S3 of FIG. 4-1. FIG. 39 is a flowchart showing another example of the ruled line frame correction process in S3 of FIG. 4-1. FIG. 40 is a diagram illustrating a computer that executes a ruled line frame correction program.

  Hereinafter, embodiments (embodiments) for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment.

(Overview)
First, a form file handled by the data structure extraction apparatus 10 will be described with reference to FIG.

  The form file is composed of one or more sheets, and the sheet includes a table (form) indicating item names and item values corresponding to the item names as indicated by reference numerals 101 and 102. There may be an inclusive relationship between the item names in the form or between the item name and the item value. For example, in the form indicated by reference numeral 101, the item name 14 includes item values 14-1 to 14-4 (inclusive in the vertical direction), and the item name 19 corresponds to each item value corresponding to the item name 14 to item name 17. (Lateral inclusion). In the form indicated by reference numeral 102, the item name 22 includes item values corresponding to the item name 20 and the item name 21 (vertical inclusion), and the item name 23 includes the item name 22, the item name 20 and the item. Each item value corresponding to the name 21 is included (inclusive in the horizontal direction). In other words, a form may have both a vertical inclusion relation and a horizontal inclusion relation.

  The data structure extraction device 10 interprets the logical structure of the form and converts the tree structure data composed of the item name and item value nodes even when the form has a vertical or horizontal logical relationship in this way. Extract. For example, the data structure extraction apparatus 10 extracts the tree structure data indicated by reference numeral 103 from the form indicated by reference numeral 101 and extracts the tree structure data indicated by reference numeral 104 from the form indicated by reference numeral 102.

  By the way, if the item name and item value of the form are not described by a description method that satisfies a predetermined requirement, the data structure extraction apparatus 10 may not be able to accurately interpret the logical structure. Therefore, in this embodiment, the data structure extraction device 10 interprets the logical structure after correcting the part so that the requirement is satisfied when there is a part described by a description method that does not satisfy the predetermined requirement in the form. Extract tree structure data.

(Constitution)
The configuration of the data structure extraction apparatus 10 will be described with reference to FIG. The data structure extraction device 10 includes a data structure extraction unit 11, a storage unit 12, and a ruled line frame correction unit 20.

  When the data structure extraction unit 11 receives an input of a form file from a terminal (for example, a personal computer, a smartphone, etc.), the data structure extraction unit 11 refers to the form structure rule 121 (details will be described later) and extracts the tree structure data of the form file. And registered in the form database (form structure information storage unit) 122.

  The storage unit 12 includes a form structure rule 121 and a form database 122. The form structure rule 121 stores various rules that the data structure extraction unit 11 refers to when extracting the tree structure data from the form file. The form structure rule 121 includes, for example, a node generation rule, meta information generation rule, adjacent edge generation rule, inclusion edge generation rule, node cluster generation rule, tree structure generation rule shown in FIG. b) include adjacent edge check rules, inclusion edge check rules, tree structure condition rules, table type / enumeration type estimation rules, tree structure selection rules, and the like. Details of these rules will be described later.

  The form database 122 stores form configuration information (see FIG. 6) including tree structure data extracted by the data structure extraction unit 11.

  The ruled line frame correction unit 20 passes the form to the data structure extraction unit 11 after correcting the part so as to satisfy the requirement when there is a part described by a description method that does not satisfy the predetermined requirement. Then, the data structure extraction unit 11 extracts tree structure data based on the form corrected by the ruled line frame correction unit 20.

(Data structure extractor)
Next, the data structure extraction unit 11 of the data structure extraction apparatus 10 will be described in detail with reference to FIG. The data structure extraction unit 11 includes a graph generation unit 13, a node cluster unit 138, a tree structure estimation unit 14, and a form structure construction unit 143. Note that the item name database 123 of the storage unit 12 may or may not be equipped, and the case of being equipped will be described later.

  The graph generation unit 13 generates node information of nodes indicating item names and item values of the form file. Further, this node information includes information (adjacent edge) indicating adjacency relation between nodes and information (inclusion edge) indicating inclusion relation. Further, the graph generation unit 13 acquires property information that is attribute information of the form file from the form file.

  The node cluster unit 138 classifies the nodes into node clusters based on the connectivity of adjacent edges of each node indicated in the node information.

  The tree structure estimation unit 14 generates a partial tree pattern from the node group classified into the node cluster. Then, the tree structure estimation unit 14 generates the tree structure data of the form file by combining the generated partial tree pattern groups so that there is no overlapping of nodes.

  The form structure construction unit 143 integrates the tree structure data of the form file output from the tree structure estimation unit 14 and the property information of the form file output from the graph generation unit 13 and registers them in the form database 122.

(Graph generator)
The graph generation unit 13 includes an operation interface identification unit 131, a form format information acquisition unit 132, a node generation unit 133, a property information acquisition unit 134, an adjacent edge generation unit 135, and an inclusion edge generation unit 136. The item name registration unit 137 may or may not be equipped, and will be described later.

  The operation interface identification unit 131 identifies the type of form file and determines an operation interface for operating the form file. Then, the operation interface identification unit 131 outputs information indicating the determined operation interface (operation interface information) to the form format information acquisition unit 132. The operation interface is an interface for acquiring form file information, and is, for example, an API (Application Programming Interface), a COM (Component Object Model), or an OLE (Object Linking and Embedding). Further, the operation interface identification unit 131 outputs the file information of the form file (for example, property information such as application type, creation date, addition date, size, file attribute, etc. of the form file) to the property information acquisition unit 134.

  The form format information acquisition unit 132 acquires document information and format information of each sheet (page) of the form file by using a form operation interface. The document information is, for example, property information about a form, title, style number, creator, creation date, number of pages, number of characters, classification, keyword, and the like. The format information includes, for example, property information related to the format constituting the form, character information (for example, character string, character type, etc.), ruled line information (for example, ruled line start and end positions, ruled line type, ruled line thickness, etc. Cell information). The format is, for example, a character string on the form, the type of ruled line, the thickness of the ruled line, the range enclosed by the ruled line, the node connection information, the node color information, etc. The cell information is, for example, a form file The height, width, upper left coordinates, lower right coordinates of the cells constituting the sheet, the connection state of the cells, the fill color of the cells, and the like.

  The node generation unit 133 generates a node indicating the item name or item value of the form according to the form structure rule 121. For example, based on the form format information obtained by the form format information acquisition unit 132, the node generation unit 133 extracts a portion surrounded by ruled lines on the form sheet as a node. Then, the node generation unit 133 outputs information about the generated node (node information) to the adjacent edge generation unit 135. In addition, the node generation unit 133 extracts information on a portion not surrounded by ruled lines on the sheet as metadata (non-node information) and outputs the metadata to the property information acquisition unit 134.

  The property information acquisition unit 134 outputs form property information (form file information, document information, non-node information) to the form structure construction unit 143.

  The adjacent edge generation unit 135 refers to the form format information and node information of the form, and generates an adjacent edge indicating the adjacent relationship between the nodes of the form. The adjacent edge generation unit 135 adds the generated adjacent edge to the node information of the node.

  The inclusion edge generation unit 136 refers to the adjacent edge of each node indicated in the node information and the format information of the form, and includes the inclusion edge indicating the vertical or horizontal inclusion relation between the nodes based on the position and size of each node. Is generated. The inclusion edge generation unit 136 adds the generated inclusion edge to the node information of the node.

(Tree structure estimation part)
The tree structure estimation unit 14 includes a partial tree pattern generation unit 140, a tree structure data construction unit 141, and a tree structure selection unit 142. The item name assigning unit 139 may or may not be equipped, and will be described later.

  The partial tree pattern generation unit 140 generates a partial tree pattern for the node group classified into the node cluster. Specifically, the subtree pattern generation unit 140 corrects adjacent edges so that the node group classified into the node cluster satisfies the characteristics of the tree structure on the form, and each node is an item name or item value. A subtree pattern is generated by setting attributes.

  In addition, in the modification of the adjacent edge, the subtree pattern generation unit 140 has a one-to-one correspondence between the item name and the item value based on the item attribute and the arrangement position of each node of the node group. It is estimated whether the enumeration type is a relationship or a table type in which the item name and the item value are a one-to-many relationship, and the adjacent edge of the node group is corrected based on the estimation result. Then, the generated partial tree pattern is output to the tree structure data construction unit 141. Details of the generation of the partial tree pattern will be described later.

  The tree structure data construction unit 141 generates the tree structure data of the form by combining the partial tree patterns so that there is no overlapping of nodes. The generated tree structure data is output to the tree structure selection unit 142. The tree structure data is, for example, represented by converting the logical structure {item name,..., Item name, item value} of the form into a tree structure as indicated by reference numerals 103 and 104 in FIG.

  The tree structure selection unit 142 selects tree structure data according to the form structure rule 121 when a plurality of pieces of tree structure data are generated for one form. The selected tree structure data is output to the form structure construction unit 143.

  According to such a data structure extraction device 10, even when vertical and horizontal logical structures are mixed in a form, tree structure data can be extracted with high accuracy.

(Rule line frame correction part)
The ruled line frame correction unit 20 includes an acquisition unit 21, a correction unit 22, and a correction rule input unit 27. The correction unit 22 includes a combining unit 23 that combines ruled line frames, a dividing unit 24 that divides ruled line frames, a deleting unit 25 that deletes ruled line frames, and an adding unit 26 that adds ruled line frames. Is provided. The acquisition unit 21 acquires ruled line frame information from the graph generation unit 13 from the node information and the format information to which the adjacent edge is added by the adjacent edge generation unit 135. Further, the acquisition unit 21 acquires the metadata including the ruled line frame information from the graph generation unit 13 as necessary.

  The ruled line frame information is information related to a rectangle composed of node information of each node and ruled lines corresponding to each node. The method of acquiring the ruled line frame information by the acquisition unit 21 is not limited to the method of acquiring from the graph generation unit 13. For example, the acquisition unit 21 may acquire ruled line frame information from an application standard object structure by referring to a form file, or may read a form as an image and acquire it from image information. Further, the ruled line frame information includes, for example, the coordinate position, width, and height of the rectangle, the fill color within the frame, the character string within the frame, the type and thickness of the ruled lines on the four sides. For a character string not surrounded by a ruled line frame, the ruled line frame may be held internally in all directions. The acquisition unit 21 extracts a ruled line frame from the form, and acquires at least a ruled line type or thickness, a character string in the frame, and a fill color in the frame as ruled line frame information for each ruled line frame.

  The correction rule storage unit 28 stores a correction rule that is a combination of a correction process that is performed when a condition for performing correction and a condition are satisfied, that is, an action. The correction rule storage unit 28 uses, as correction rules, for example, a rule that combines ruled line frame combination processing and ruled line frame combination conditions, which are conditions for performing the combination processing, and ruled line frame dividing processing and division processing. A division rule that combines a ruled line frame division condition, a rule that deletes a ruled line frame and a rule that deletes the ruled line frame deletion condition, and a condition that adds and adds a ruled line frame An additional rule that is combined with a ruled line frame additional condition is stored.

  Ruled line frame combination conditions, ruled line frame dividing conditions, and ruled line frame deletion conditions include, for example, that the type of ruled line is a specific type set in advance, and the thickness of the ruled line is included in a range of specific thickness set in advance That the character string in the frame is a specific character string set in advance, the fill color in the frame is a specific color set in advance, or a combination of a plurality It is.

  Each unit of the correction unit 22 refers to the correction rule and performs an action corresponding to each condition on the ruled line frame that satisfies the condition. When the ruled line frame information of a plurality of ruled line frames satisfies a preset ruled line frame combining condition, the combining unit 23 combines the plurality of ruled line frames. Further, after the processing by the combining unit 23 is performed, the dividing unit 24 divides the ruled line frame when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame dividing condition. Further, after the processing by the dividing unit 24 is executed, the deletion unit 25 deletes the ruled line frame when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame deletion condition.

  In addition, after the processing by the deletion unit 25 is executed, the adding unit 26 adds a ruled line to the area when the area information of the area satisfies a preset ruled frame addition condition. At this time, the acquisition unit 21 extracts a region in which a character string is described from the form based on metadata or the like, and as region information for each region, the presence / absence of ruled lines in the upper, lower, left, and right directions of the region, the character string In addition, at least the fill color of the area is acquired, and if there is a ruled line in any of the upper, lower, left, and right directions of the area, the type or thickness of the ruled line is acquired.

  Each unit of the correction unit 22 corrects the ruled line frame by changing the node information. The correction rule storage unit 28 is manually or automatically input from the correction rule input unit 27. The node information corrected by the correction unit 22 is transferred to the graph generation unit 13, the node cluster unit 138, the tree structure estimation unit 14, or the like.

  Here, an example of a ruled line frame that is corrected by the ruled line frame correction unit 20 will be described with reference to FIG. FIG. 3B is a diagram illustrating an example of the ruled line frame to be corrected. First, as illustrated in FIG. 3A, the deletion unit 25 deletes the ruled line frame when the character string of the ruled line frame starts with “*”. Further, as illustrated in FIG. 3B, the dividing unit 24 includes both a character representing a check box and a character string representing an item name of the check box in the character string in the ruled line frame. In this case, the ruled line frame is divided into a ruled line frame corresponding to the check box and a ruled line frame corresponding to the item name.

  As shown in FIG. 3C, when the character strings in the ruled line frame are predetermined character strings A and B, the dividing unit 24 divides the ruled line frame for each character string. Further, as shown in FIG. 3D, the connecting unit 23 uses a dotted line as the ruled line type between the first ruled line frame and the second ruled line frame adjacent to the first ruled line frame. In some cases, the first ruled line frame and the second ruled line frame are combined. As shown in FIG. 3E, the deletion unit 25 deletes the ruled line frame when the character string in the frame of the ruled line frame is empty and the fill color is gray.

(Processing procedure)
Next, the processing procedure of the data structure extraction device 10 will be described with reference to FIG. When the data structure extraction unit 11 of the data structure extraction apparatus 10 receives an input of a form file (S1), the graph generation unit 13 generates a node graph indicating item names and item values of the form file (S2). That is, the graph generation unit 13 generates node information indicating the adjacent relationship and the inclusion relationship between the nodes for each node constituting the form. The graph generation unit 13 also acquires form property information.

  The ruled line frame correction unit 20 corrects the ruled line frame by changing the node information generated by the graph generation unit 13 (S3). The node information corrected by the ruled line frame correction unit 20 is transferred to the node cluster unit 138.

  Next, the node cluster unit 138 classifies each node into a node cluster based on the connectivity of adjacent edges indicated in the node information of each node corrected by the ruled line frame correction unit 20 (S4). Then, the tree structure estimation unit 14 generates a partial tree pattern for each node group classified into the node cluster, and generates tree structure data by combining the generated partial tree patterns (S5: tree structure estimation). Thereafter, the form structure construction unit 143 integrates the tree structure data generated in S5 and the property information of the form acquired in the graph generation unit 13 and registers them in the form database 122 (S6: form structure construction).

  In this way, the data structure extraction device 10 can generate the tree structure data of the form from the form file. Further, the data structure extraction unit 11 can register in the form database 122 information obtained by integrating the tree structure data of the form and the property information of the form.

(Graph generation)
Next, the graph generation process of S2 in FIG. 4A will be described in detail with reference to FIG. First, the operation interface identification unit 131 of the graph generation unit 13 identifies the operation interface of the form file (S131), and the form format information acquisition unit 132 acquires the format information of the form file (S132), and generates a node. The unit 133 generates node information constituting the form file (S133: node generation). Also, the property information acquisition unit 134 aggregates the file information, document information, and non-node information of the form as the property information of the form, and outputs it to the form structure construction unit 143 (S134: acquisition of property information). Thereafter, the adjacent edge generation unit 135 refers to the format information and node information of the form, generates an adjacent edge indicating the adjacent relationship between the nodes of the form, and adds the generated adjacent edge to the node information of the node. (S135: Adjacent edge generation). Thereafter, the inclusion edge generation unit 136 refers to the adjacent edge of each node indicated in the node information and the form format information, generates an inclusion edge of each node, and adds the generated inclusion edge to the node information of the node. (S136: inclusion edge generation).

  By doing in this way, the graph production | generation part 13 can produce | generate the node information which shows the adjacent relationship and inclusion relation between each node about each node which comprises a form.

(Rule border correction)
Next, the ruled line frame correction process in S3 of FIG. 4A will be described in detail with reference to FIG. The acquisition unit 21 of the ruled line frame correction unit 20 acquires ruled line frame information from the graph generation unit 13 (S21). In addition, when performing the ruled line frame addition process by the adding unit 26, the acquiring unit 21 acquires the ruled line frame information including metadata, which is information of a portion not surrounded by the ruled line, from the graph generating unit 13. Further, the correction unit 22 reads the correction rule from the correction rule storage unit 28 (S22).

  When the ruled line frame information satisfies the ruled line frame combination condition, the combining unit 23 combines the ruled line frame corresponding to the ruled line frame information (S23). When the ruled line frame information satisfies the ruled line frame division condition, the dividing unit 24 divides the ruled line frame corresponding to the ruled line frame information (S24). When the ruled line frame information satisfies the ruled line frame deletion condition, the deleting unit 25 deletes the ruled line frame corresponding to the ruled line frame information (S25). Then, when the metadata satisfies the ruled line frame addition condition, the adding unit 26 adds the ruled line frame to a portion corresponding to the metadata (S26). Then, the ruled line frame correction unit 20 outputs the corrected node information (step S27).

  As described above, the ruled line frame correction unit 20 corrects the ruled line frame satisfying a predetermined condition, so that the tree structure estimation unit 14 and the like can accurately estimate the logical relationship between item names and between item names and item values. It becomes like this. A specific example of processing of each unit of the correction unit 22 will be described later.

(Tree structure estimation)
Next, the tree structure estimation process of S5 of FIG. 4-1 will be described in detail with reference to FIG. The subtree pattern generation unit 140 of the tree structure estimation unit 14 generates a subtree pattern for the node group classified into the node cluster in S4 (S140). Then, the tree structure estimation unit 14 constructs tree structure data by combining the partial tree patterns generated in S140 (S141). Thereafter, if there are a plurality of tree structure data constructed in S141, the tree structure selection unit 142 selects one tree structure data from these (S142).

  By doing in this way, the tree structure estimation part 14 can produce | generate the tree structure data reflecting the adjacent relationship and inclusion relation between each node of a form.

(Operation interface identification)
Next, an example of the operation interface identification process in S131 of FIG. 4-2 will be described with reference to FIG. In the following description, for example, CSV (Comma-Separated Values), JSON (JavaScript (registered trademark) Object Notation), XML (eXtensible Markup Language), or the like is used as a description language of data at the time of output to a specified destination. . The output to the designated destination may be output for each folder, or may be performed after data compression such as ZIP or CAB format.

  First, when the operation interface identification unit 131 reads the file information of the form file (S1311), the operation interface identification unit 131 identifies the type of the form file (S1312). For example, the operation interface identification unit 131 identifies the type of application used in the form file from the property information and extension of the form file included in the file information and the magic number inherent in the application. Further, the operation interface identification unit 131 outputs the read file information to the property information acquisition unit 134. Then, the operation interface identifying unit 131 determines an operation interface according to the type of the form file specified in S1312 (S1313). If the operation interface can be determined (Yes in S1314), the operation interface identification unit 131 outputs the operation interface information to the designated destination (here, the form format information acquisition unit 132) (S1315), and the operation interface cannot be determined. If (No in S1314), for example, an error message “do not register information as a form” is returned to the user (S1316).

  In this way, the operation interface identification unit 131 can determine the operation interface of the form file.

(Get form format information)
Next, an example of the form format information acquisition process in S132 of FIG. 4-2 will be described with reference to FIG. The form format information acquisition unit 132 reads the operation interface information of the form file (S1321). When the form file is read (S1322), the form information for each page (sheet) is acquired from the form file (S1323). Document information is acquired from the file (S1324). Then, if the acquired format information and document information include ruled line information (Yes in S1325), the form format information acquisition unit 132 outputs the acquired format information and document information to the designated destination (S1326). For example, the format information is output to the node generation unit 133, and the document information is output to the property information acquisition unit 134. If the acquired format information and document information do not have ruled line information (No in S1325), the form format information acquisition unit 132 returns, for example, an error message “not register information as a form” to the user (S1327). .

  In this way, the form format information acquisition unit 132 can acquire format information and document information from the form file.

(Node generation)
Next, an example of the node generation processing in S133 of FIG. 4-2 will be described using FIGS. 5-3, 5-4, and 16-2. The node generation unit 133 reads the format information (S1331), reads the form structure rule 121 (S1332), and acquires the node information from the format information according to the node generation rule of the form structure rule 121 (see FIG. 16-1). (S1333). For example, as shown in FIG. 5-4, the node information includes a character string (for example, “person in charge”) surrounded by a ruled line, upper left coordinates (px1, py1) and lower right coordinates ( px2, py2), fill color (for example, white), ruled line type (for example, solid line), ruled line size (for example, 1 pt), and the like. In addition, information on the height and width of the cell surrounded by the ruled line may be included.

  When the node generation unit 133 acquires the node information from the format information as described above, the node generation unit 133 outputs the node information to the adjacent edge generation unit 135 (S1334: output node information). On the other hand, information other than the node information (non-node information) among the information included in the format information is output to the property information acquisition unit 134 (S1335: output non-node information).

  Whether the information included in the format information is node information (node information) or non-node information (non-node information) is determined as follows. For example, in accordance with the form structure rule 121, the node generation unit 133 uses “name” or the like enclosed by ruled lines of the form as a node in the form shown in FIG. ”Etc. as non-nodes. Note that the node generation unit 133 may use a region surrounded by a ruled line and painted with the same background color as a node, or may use a region surrounded by a solid ruled line as a node. Further, the node generation unit 133 may determine a character string to be a non-node in advance, and an area including the character string may be a non-node, or an area to be determined as a non-node in advance (for example, at the top or bottom of a form) A region where the character string is arranged in the region may be determined as a non-node. Then, the node generation unit 133 acquires the portion determined to be a node as node information, and acquires the portion determined to be a non-node as non-node information.

  In this way, the node generation unit 133 can extract nodes (node information) necessary for generating the tree structure data of the form file. In addition, the node generation unit 133 can extract the attribute information of the form file as non-node information.

(Acquire property information)
Next, an example of the property information acquisition process in S134 of FIG. 4-2 will be described with reference to FIG. The property information acquisition unit 134 reads the file information (S1341), reads the document information (S1342), reads the non-node information (S1343), and reads the form structure rule 121 (S1344). Then, the property information acquisition unit 134 generates form meta information from the non-node information in accordance with the read form structure rule 121 (S1345). The form meta information is information accompanying the form, such as the form title, date, and style.

  For example, the property information acquisition unit 134 follows the meta information generation rule of the form structure rule 121 (see (a) in FIG. 16A), and delimiters for character strings of non-node information (for example, “:” and “/”). Etc.) and form meta information based on the contents of the character string. For example, if there is non-node information “name: xxx”, form meta information “name” is generated. Further, the property information acquisition unit 134 knows a character string (for example, “style”, “name”), special characters (for example, “〒”, “TEL”), data type, format, etc. included in the non-node information. If so, the form meta information may be generated based on the data type or format. For example, if the data type such as date and employee number is known in advance, the property information acquisition unit 134 generates form meta information such as “date” and “employee number” based on the data type.

  After S1345, the property information acquisition unit 134 aggregates each piece of attribute information (that is, file information, document information, and form meta information) as property information (S1346) and outputs it to the form structure construction unit 143 (S1347: property information). output).

  In this way, the property information acquisition unit 134 can acquire the property information of the form file.

  In S1347, for example, the property information is output to the form structure construction unit 143 in a state in which file information, document information, and form meta information are associated with each other as shown in FIG. Thereafter, the form structure construction unit 143 integrates the property information with the tree structure data (reference numerals 601 and 602) generated by the tree structure estimation unit 14 and outputs the result to the form database 122.

(Adjacent edge generation)
Next, an example of the adjacent edge generation process of S135 of FIG. 4-2 will be described using FIGS. 7-1 and 7-2. The adjacent edge generation unit 135 reads the node information generated in S133 of FIG. 4-2 (S1351), and reads the form structure rule 121 (S1352). Then, the adjacent edge generation unit 135 obtains an adjacent edge between the nodes in accordance with the form structure rule 121 (S1353), and checks the adjacent relationship (S1354). Thereafter, the adjacent edge generation unit 135 adds the adjacent edge (adjacent edge information) generated in S1353 to the node information (S1355), and the adjacent edge generation unit 135 includes the node information to which the adjacent edge information is added as an inclusion edge generation unit. It outputs to 136 (S1356).

  For example, the adjacent edge generation unit 135 first refers to the node information of each node, and generates information (adjacent vector) indicating adjacent nodes in the upper, lower, left, and right directions for each node. . When there is an adjacent node in the node, the adjacent node index information is held in the adjacent vector. When there is no adjacent node, the fact (for example, “0”) is described in the adjacent vector. For example, if there is no adjacent node above, below, and left in the “person” node shown in FIG. 7B, but the “name” node and “affiliation” node are adjacent on the right, the adjacent edge generation unit 135 The adjacent edge information indicating that is added to the node information of the “person in charge” node.

  In this way, the adjacent edge generation unit 135 can generate an adjacent edge indicating the adjacent relationship between the nodes.

(Process to find adjacent edges between nodes)
Next, an example of processing for obtaining adjacent edges between nodes in S1353 of FIG. 7-1 will be described using FIGS. 7-3 and 16-3. When the adjacent edge generation unit 135 defines the unsearched nodes X and Y with reference to the node information read in S1351 of FIG. 7-1 (S13532), the node X and the node Y are in the form structure rule related to the adjacent relationship. It is determined whether or not 121 (adjacent edge generation rule in FIG. 16A (a)) is satisfied (S13533). In S13533, when the adjacent edge generation unit 135 determines that the form structure rule 121 related to the adjacent relationship is satisfied for the node X and the node Y (Yes in S13533), the adjacent edge is extended in the adjacent direction to the nodes X and Y ( S13534). If the adjacent edge generation unit 135 determines that the adjacent relationship between all the nodes has been checked (Yes in S13535), the adjacent edge generation unit 135 outputs adjacent edge information (S13536). On the other hand, when it is determined in S13533 that the adjacent edge generation unit 135 does not satisfy the form structure rule 121 related to the adjacent relationship with respect to the node X and the node Y (No in S13533), S13534 is skipped and the process proceeds to S13535. On the other hand, if the adjacent edge generation unit 135 determines that there is one that has not yet checked the adjacent relationship between the nodes (No in S13535), the process returns to S13532.

  The above-mentioned adjacent edge generation rule is, for example, for the node X and the node Y shown in FIG. 16C, (1) the y coordinate range of the node X includes the y coordinate range of the node Y, and (2 The rule is that when the end point of the x coordinate of the node X coincides with the start point of the x coordinate of the node Y, the node X and the node Y are considered to be adjacent. That is, for node X and node Y shown in FIG. 16C, the upper left coordinate of node X is X (x1, y1), the lower right coordinate is X (x2, y2), and the upper left coordinate of node Y is Y (x1, y1). y1), where Y (x2, y2) is the lower right coordinate, (1) X (*, y1) ≦ Y (*, y1) and X (*, y2) ≧ Y (*, y2) , (2) When X (x2, *) = Y (x1, *), the adjacent edge generation unit 135 regards the node X and the node Y as having an adjacent relationship.

  Although the above is the right (horizontal) adjacent edge generation rule, the downward (vertical) adjacent edge generation rule is defined similarly. As described above, the adjacent edge generation unit 135 generates adjacent edges in the right and down directions of the node, so that the left and upper adjacent edges of the paired nodes (adjacent to the node) can also be acquired.

  In this way, the adjacent edge generation unit 135 can generate an adjacent edge based on the positional relationship between the nodes.

(Process to check adjacency)
Next, an example of processing for checking the adjacency relationship between nodes in S1354 of FIG. 7-1 will be described with reference to FIGS. 7-4 and 16-4. The adjacent edge generation unit 135 reads the adjacent edge information output in S13536 (S13541), and defines the adjacent direction k for the unconfirmed node X (S13542). The adjacent direction k indicates either the horizontal direction or the vertical direction. For example, k = 0 in the horizontal direction and k = 1 in the vertical direction. Next, the adjacent edge generation unit 135 satisfies the form structure rule 121 (adjacent edge check rule of (b) in FIG. 16A) of the node X in the k direction (No in S13543), all nodes, all If the adjacent direction has been confirmed (Yes in S13545), adjacent edge information is output (S13546). On the other hand, if the adjacent edge in the k direction of the node X does not satisfy the form structure rule 121 (adjacent edge check rule in FIG. 16B (b)) (Yes in S13543), the adjacent edge is deleted (S13544). The process proceeds to S13545. If any node or any adjacent direction is not confirmed in S13545 (No in S13545), the process returns to S13541.

  The above adjacent edge check rule is, for example, for the node X shown in FIG. 16-4. (1) The y coordinate of all nodes located on the right side of the node X is within the range of the y coordinate of the node X (one However, if there is a node that does not satisfy, it is not established), and (2) when there is no more than one node to the left of the node X, it is considered that the modification of the adjacent edge relating to the node X is not necessary. For example, as shown by reference numeral 1601 in FIG. 16-4, if the node located on the right side of the node X is only the node Y and the y coordinate of the node Y is included in the y coordinate of the node X, the adjacent edge is generated. The unit 135 determines that the condition (1) described above is satisfied. On the other hand, as indicated by reference numeral 1602, if the nodes located on the right side of the node X are the nodes Y and Z and the y coordinate of the node Y is not included in the y coordinate of the node X, the condition (1) described above is satisfied. Judge that it does not meet. If the node located on the left side of the node X is only the node W as indicated by reference numeral 1603, the adjacent edge generation unit 135 determines that the condition (2) described above is satisfied. On the other hand, as indicated by reference numeral 1604, when there are two nodes W and V located on the left side of the node X, the adjacent edge generation unit 135 determines that the condition (2) described above is not satisfied.

  Although the above is the adjacent edge check rule in the right direction (horizontal direction), the adjacent edge check rule in the downward direction (vertical direction) is similarly defined.

  In this way, the adjacent edge generation unit 135 can correct the generated adjacent edge if it includes an inappropriate adjacent relationship as the adjacent relationship on the form.

(Included edge generation)
Next, an example of the inclusion edge generation process in S136 of FIG. 4-2 will be described with reference to FIGS. 8 and 16-5. The inclusion edge generation unit 136 reads node information including the adjacent edge information generated in S135 of FIG. 4-2 (S1361), and reads the form structure rule 121 (S1362). When the inclusion edge generation unit 136 defines the unsearched nodes X and Y and the adjacent direction k (S 1363), the inclusion-related form for an arbitrary node X and another node Y and the k direction (vertical / horizontal). It is determined whether or not the structure rule 121 (the inclusion edge generation rule in FIG. 16A (a)) is satisfied (S1364). Here, if the inclusion edge generation unit 136 satisfies the form structure rule 121 regarding the inclusion relation for an arbitrary node X, another node Y, and the k direction (vertical / horizontal) (Yes in S1364), the nodes X, Y An inclusion edge is stretched between them (S1365). On the other hand, the inclusion edge generation unit 136 generates a form structure rule 121 (inclusion edge generation rule in FIG. 16A in FIG. 16A) regarding an inclusion relationship for an arbitrary node X, another node Y, and the k direction (vertical / horizontal). If not satisfied (No in S1364), S1365 is skipped and the process proceeds to S1366. If the inclusion relationship between all the nodes has been confirmed in S1366 (Yes in S1366), inclusion edge information is added to the node information (S1367), and the node information is output to the node cluster unit 138 (S1368). On the other hand, if the inclusion relationship between any of the nodes has not been confirmed (No in S1366), the process returns to S1363.

  Note that the above-described inclusion edge generation rule includes, for example, (1) two or more nodes adjacent to the right side of the node X. (2) The y-coordinates of the nodes adjacent in the right direction of the node X are all within the range of the y-coordinates of the node X. (3) Any one of the y coordinates of the nodes adjacent to the right side of the node X overlaps with the y coordinate of the start point of the node X, and any one of the y coordinates of the nodes adjacent to the right side of the node X is the node X. It overlaps with the y coordinate of the end point. When all the three conditions are satisfied, this is a rule that considers that there is an inclusive relationship between the node X and a node adjacent to the node X in the right direction (and a series of nodes adjacent to the node).

  According to the inclusion edge generation rule, for example, the node X indicated by reference numeral 161 in FIG. 16-5, the node adjacent to the node X in the right direction (node Y, Z, U), and adjacent to the node in the right direction For a series of nodes (nodes W, V, T), the inclusion edge generation unit 136 considers that there is an inclusion relationship. In other words, in the above example, the inclusion edge generation unit 136 considers that the node X, the nodes Y, Z, U, W, V, and T have an inclusion relationship.

  For example, when the node U is missing from the node group indicated by reference numeral 161 in FIG. 16-5 (see reference numeral 162), the above conditions (1) and (2) are satisfied. ) Does not satisfy the condition that “one of the y-coordinates of nodes adjacent to the right of node X overlaps the y-coordinate of the end point of node X”. , Z, W, V, and T are regarded as having no inclusive relation.

  The above is the right (horizontal) inclusion edge generation rule, but the downward (vertical) inclusion edge generation rule is defined similarly. For example, the inclusive edge generation rule in the downward direction (vertical direction) has (1) two or more nodes adjacent in the downward direction of the node X. (2) All the x coordinates of the nodes adjacent in the downward direction of the node X are within the range of the x coordinates of the node X. (3) Any one of the x coordinates of the nodes adjacent to the right side of the node X overlaps with the x coordinate of the start point of the node X, and any one of the x coordinates of the nodes adjacent to the right side of the node X is the node X. It overlaps with the x coordinate of the end point of. When all the three conditions are satisfied, this is a rule that considers that there is an inclusive relationship between the node X and a node adjacent to the node X in the downward direction (and a series of nodes adjacent to the node).

  By doing in this way, the inclusion edge production | generation part 136 can produce | generate the inclusion edge which shows the inclusion relationship of each node.

(Process to classify into node cluster)
Next, an example of processing for classifying the node cluster in S4 of FIG. 4A will be described with reference to FIGS. 10A, 10B, and 10C. The node cluster unit 138 reads the node information generated by the graph generation process of S2 of FIG. 4A (S1381), and reads the form structure rule 121 (S1382). Then, the node cluster unit 138 performs clustering of another node Y starting from an arbitrary node X according to the form structure rule 121 (the node cluster generation rule shown in FIG. 16A (a)) (S1383). Thereafter, the node cluster unit 138 checks whether or not there is a non-clustered node (S1384), and if there is a non-clustered node (Yes in S1384), selects the non-clustered node as the node X ( S1385), the process returns to S1383. On the other hand, if there is no node that has not been subjected to node clustering (No in S1384), the node cluster unit 138 outputs the node cluster to the tree structure estimation unit 14 (S1386).

  Note that the above node cluster generation rule is, for example, a rule that regarding a neighboring edge of a certain node, a node group connected to the node is regarded as a node cluster. In addition, in the above rules, for example, a node that is divided by a predetermined ruled line (for example, a thick line or a red ruled line) among a group of nodes connected to the node is regarded as another node cluster, A rule may be combined in which nodes of a solid color (for example, gray) are regarded as the same node cluster.

  The node cluster unit 138 physically separates the tables in the sheet of the same form file, for example, as shown in FIG. 10-2 by performing classification into node clusters according to such a node cluster generation rule. Can be divided into node clusters 1 and 2. As a result, the tree structure estimation unit 14 can generate separate tree structure data for physically separated forms (tables) in the same sheet.

(Clustering processing of another node Y starting from an arbitrary node X)
Note that the process of S1383 in FIG. 10A is performed, for example, according to the process procedure illustrated in FIG. First, when the node cluster unit 138 reads the node information of the node X (S13831), the node cluster unit 138 searches for another node Y from the node X (S13832), and when the node Y cannot be found (No in S13833), It is determined whether or not the node Y has been clustered (S13834). If the node Y has not been clustered (No in S13834), the node Y is classified into the same cluster (node cluster) as the node X and has been classified into the node Y. Is set (S13835). Thereafter, the process proceeds to S13836. When the node Y is found in S13833 (Yes in S13833), the process proceeds to S13839. If the node Y has been clustered (Yes in S13834), the process proceeds to S13836.

  In S13836, when the node cluster unit 138 has searched all the nodes (Yes in SS13836), the node cluster unit 138 sets a classified flag for the node X (S13837), and outputs the node cluster (S13838). On the other hand, when there is a node that has not been searched yet (No in S13836), an unsearched node is defined as node Y (S13839), and the process returns to S13832.

  In this way, the node cluster unit 138 can classify each node into a node cluster.

(Partial tree pattern generation processing)
Next, an example of the sub-tree pattern generation process of S140 of FIG. 4-4 will be described using FIGS. 12-1 to 12-5, FIG. 16-6, and FIG. 16-7. The partial tree pattern generation unit 140 reads the node information of the node cluster classified in S4 of FIG. 4A (S1401), and reads the form structure rule 121 (S1402). Then, the subtree pattern generation unit 140 acquires a set of inclusion nodes in the node cluster, and sets {φ} to the searched inclusion node set (S1403). The inclusion node is a node that includes other nodes in the inclusion relationship between nodes. For example, node X in the node group indicated by reference numeral 161 in FIG. 16-5 corresponds to the inclusion node.

  Next, the inclusion edge generation unit 136 classifies C (X, k) for each hierarchy of inclusion nodes. Further, m is set as the maximum hierarchy, and “0” is set to n (hierarchy of the inclusion node) (S1404).

  Note that the above C (X, k) indicates a set of nodes included in the k direction by the node X which is an included node. For example, the node set included in the vertical direction by the node X is C (X, 1), and the node set included in the horizontal direction is C (X, 0). The above hierarchy indicates a hierarchy when the inclusion node has a nested structure. When there is no inclusion node to be nested, the hierarchy is “1”, and in other cases, “the number of the nested structures + 1”. . For example, since the node cluster shown in FIG. 12B has one nested inclusion node, the hierarchy is “2”.

  Next, the subtree pattern generation unit 140 acquires a subtree pattern for C (X, k) at level n (S1405), and adds a level n inclusion node to the searched inclusion node set (S1406). Then, the subtree pattern generation unit 140 increments the value of n (S1407). If n> m and there is no unsearched inclusion node (Yes in S1408), the subtree pattern of the inclusion node set is represented by a tree. The data is output to the structure data construction unit 141 (S1409). On the other hand, if n> m is not satisfied or there is an unsearched inclusion node (No in S1408), the process returns to S1405.

  By doing in this way, the subtree pattern generation unit 140 can acquire a subtree pattern for each hierarchy even when the inclusion node has a nested structure in the node cluster.

(Partial tree pattern acquisition process)
Next, an example of the sub-tree pattern acquisition process in S1405 of FIG. 12A will be described with reference to FIG. The partial tree pattern generation unit 140 reads C (X, k) of level n (S14051), and reads the form structure rule 121 (S14052). Then, the subtree pattern generation unit 140 sets {φ} to the subtree pattern of C (X, k) (S14053), and if n is 1 or more (Yes in S14054), C (X, k) C (X ′, k) is regarded as a dummy node for the inclusion node (node X ′) in the nearest hierarchy included in (S14055). For example, the subtree pattern generation unit 140 regards the node group indicated by reference numeral 1201 among the node groups shown in FIG. 12-2 as one node (dummy node) as indicated by a balloon 1202 in FIG. 12-3. If n is not 1 or more in S14054 (No in S14054), the process proceeds to S14059.

  Next, the subtree pattern generation unit 140 performs a tree structure conversion process on C (X, k) (S14056), and if a subtree pattern can be acquired (Yes in S14057), the subtree pattern of C (X, k). The subtree pattern is added to (S14058). On the other hand, if the subtree pattern cannot be acquired in S14057 (No in S14057), the process proceeds to S14059.

  Next, the subtree pattern generation unit 140 considers that there is no inclusion node included in C (X, k) (S14059), and performs a tree structure conversion process on C (X, k) as in S14056 (S14060). If the subtree pattern can be acquired (Yes in S14061), the subtree pattern is added to the subtree pattern of C (X, k) (S14062). Then, the subtree pattern generation unit 140 outputs a C (X, k) subtree pattern (S14063). On the other hand, if the partial tree pattern cannot be acquired (No in S14061), S14062 is skipped and the process proceeds to S14063.

  In this way, the subtree pattern generation unit 140 allows both the subtree pattern for each hierarchy and the subtree pattern of the entire node cluster when the containing nodes are nested in the node cluster. A tree pattern can be acquired.

(Tree structure conversion processing for C (X, k))
Next, an example of a tree structure conversion process for C (X, k) in S14056 and S14060 in FIG. 12-3 will be described with reference to FIG. The partial tree pattern generation unit 140 reads C (X, k) (S140601), and reads the form structure rule 121 related to tree structure generation (S140602). Then, the subtree pattern generation unit 140 corrects adjacent edges of each node and sets item attributes in accordance with the form structure rule 121 (tree structure generation rule in FIG. 16A) relating to tree structure generation ( S140603).

  Note that the tree structure generation rule includes, for example, the following four conditions (1) to (4). That is, (1) there is only one adjacent edge in the lower layer (in this case, the right side) except for the inclusion node. If there are two or more adjacent edges in the lower order (in this case, the right side) excluding the inclusion node, the subtree pattern generation unit 140 cuts the adjacent edges. For example, the subtree pattern generation unit 140 cuts adjacent edges connecting the node a and the nodes b and c shown in (1) of FIG.

  (2) All nodes below the inclusion node have inclusion nodes at the top. After (1), if there is a node that does not have a higher-order (in this case, left side) adjacent edge except for the included node, the subtree pattern generation unit 140 sets the included node (node X) and the adjacent edge in the k direction. Tighten. For example, the subtree pattern generation unit 140 extends adjacent edges connecting the node X and the nodes b and c shown in (2) of FIG.

  (3) The item attribute when there is a lower adjacent edge is “item name”, and the item attribute of a node that is not present is “item value”. After (2), the subtree pattern generation unit 140 sets the item attribute of the node having no lower adjacent edge to “item value”. For example, the subtree pattern generation unit 140 sets the item attributes of the nodes a, d, e, and f shown in (3) of FIG. 16-6 to “item value”.

  (4) When the adjacent edge includes a portion in which two or more nodes are connected, the table type or enumeration type or adjacent edge should not be stretched (incompatible tree structure). After (1) to (3), when an adjacent edge includes a portion in which two or more nodes are connected, a group of nodes including inclusion nodes according to a table type / enumeration type estimation rule (details will be described later) Estimate (determine) whether the table type or enumeration type. For example, if the subtree pattern generation unit 140 finds a series of node groups in which two or more adjacent edges are connected to each other as indicated by reference numeral 1610 in (4) of FIG. The node group to be indicated is subject to estimation (judgment) whether it is a table type or an enumeration type according to a table type / enumeration type estimation rule (details will be described later).

  After S140603 in FIG. 12-4, the subtree pattern generation unit 140 selects an unsearched node Z from the nodes having the item attribute “item value” set in S140603 (S140604), and from the node Z to the node X. (S140605). A route from the node Z to the node X at this time is defined as route (X, Z). Then, according to the table type / enumeration type estimation rule (see (b) of FIG. 16A), the subtree pattern generation unit 140 determines whether there is a route (route (X, Z)) that becomes the table type / enumeration type. Is determined (estimated) (S140606), and if there is a route (route (X, Z)) that becomes a table type / enumeration type (Yes in S140606), the form structure rule 121 (table type) regarding the table type / enumeration type estimation According to the enumeration type estimation rule), a table type and an enumeration type are set (S140607). Then, the process proceeds to S140609. On the other hand, if there is no route (route (X, Z)) that becomes a table type / enumeration type (No in S140606), items other than the node Z on the route (route (X, Z)) that becomes a table type / enumeration type. The attribute is set to “item name” (S140608), and the process proceeds to S140609.

  The table type / enumeration type estimation rule includes, for example, the following four conditions (1) to (4). (1) The nodes included in the node X have a grid-like connection relationship. For example, when the nodes included in the node X have a grid-like connection relationship as shown in (1) of FIG. 16-7, the subtree pattern generation unit 140 selects these nodes as a table type / enumeration type estimation target. And

  (2) In the case where the condition of (1) is satisfied, if a node of “item name” exists in a node not adjacent to the node X, it is determined as “enumeration type” or “incompatible with tree structure”. For example, as shown in (2) of FIG. 16-7, the sub-tree pattern generation unit 140, when a node of “item name” exists in a node that is not adjacent to the node X, is “enumerated type” or “incompatible with tree structure”. Estimated.

  (3) In the case where the condition of (1) is satisfied but the condition of (2) is not satisfied, if the end point of each node included in the node X overlaps with any end point of another adjacent node, “table type” ". For example, as shown in FIG. 16-7 (3), the subtree pattern generation unit 140 displays a “table type” if the end points of each node included in the node X overlap with any of the end points of other adjacent nodes. ".

  (4) When the conditions of (1) and (2) are satisfied, if the number of nodes from the node excluding the node X to the terminal node is an even number, it is estimated as “enumeration type”, and if it is an odd number, “ It is estimated that the tree structure is incompatible. For example, if the number of nodes from the node excluding the node X to the terminal node is an even number (four) as shown in (4) of FIG. If it is an odd number (three), it is determined that the tree structure is incompatible. That is, as shown in (4) of FIG. 16-7, in the case of “enumeration type”, “item name” and “item value” have a pair structure, but “item name” is a pair “ When there is no “item value”, the tree structure of the form is unnatural, and the partial tree pattern generation unit 140 estimates that the node group is “incompatible with the tree structure”. As described above, the subtree pattern generation unit 140 estimates whether the node group is a table type, an enumeration type, or a tree structure in the first place.

  Then, the subtree pattern generation unit 140 assigns table type item attributes and corrects adjacent edges for the node group estimated as the table type in accordance with the above table type / enumeration type estimation rule (FIG. 12-5). (See (a).) For the node group estimated to be an enumeration type, assignment of enumeration type item attributes and correction of adjacent edges are performed (see (b) of FIG. 12-5).

  After S140607 in FIG. 12-4, the subtree pattern generation unit 140 searches for the node Z selected in S140604 when C (X, k) can be converted into a table type or an enumeration type in S140607 (Yes in S140609). (S140610). If all nodes having the item attribute “item value” have been searched (Yes in S140611), the subtree pattern generation unit 140 outputs a subtree pattern of C (X, k) (S140612). If there is a node that has not been searched for in the node whose attribute is “item value” (No in S140611), the process returns to S140604.

  On the other hand, in S140609, the subtree pattern generation unit 140 does not satisfy the tree structure condition for C (X, k) when C (X, k) cannot be converted into a table type or an enumeration type (No in S140609). Is determined (S140613). In this case, the subtree pattern is not output.

  In this way, the subtree pattern generation unit 140 corrects adjacent edges of each node while considering whether the node group (C (X, k)) included in the included node is a table type or an enumerated type. And set item attributes. Also, the subtree pattern generation unit 140 does not output a node group that does not satisfy the tree structure condition (characteristics of the tree structure as a form) as a subtree pattern. As a result, the subtree pattern generation unit 140 can generate a subtree pattern with high accuracy.

(Tree structure data construction process)
Next, the tree structure data construction processing of S141 of FIG. 4-4 will be described using FIG. The tree structure data construction unit 141 reads the partial tree pattern of each inclusion node generated in S140 (S1411), reads the node information of the node cluster (S1412), and reads the form structure rule 121 (S1413). Then, after setting {φ} in the set of tree structure data (S1414), the tree structure data construction unit 141 obtains a combination of subtree patterns so that there is no overlapping of nodes (S1415). Thereafter, the tree structure data construction unit 141 selects an unconfirmed partial tree pattern combination from the partial tree pattern combination group (S1416). That is, a combination of sub-tree patterns not subjected to the processing of S1419 described later is selected. Thereafter, the tree structure data construction unit 141 refers to the node information of the node cluster read in S1412, and adds a missing node for the combination of the partial tree patterns selected in S1416 (S1417).

  Next, the tree structure data construction unit 141 generates a tree structure with reference to the inclusion edges and adjacent edges of the combination of subtree patterns generated by the processing up to S1417 (and nodes added thereto) (S1418). .

  Then, the tree structure data construction unit 141 sets the tree structure condition for the tree structure generated in S1418 in accordance with the form structure rule 121 (the tree structure condition rule in FIG. 16B). It is determined whether or not it is satisfied (S1419). If the conditions of the tree structure are satisfied (Yes in S1419), the tree structure is added to the tree structure data (S1520). Thereafter, if the combination of all the partial tree patterns is confirmed (Yes in S1521), the tree structure data construction unit 141 outputs the tree structure data to the tree structure selection unit 142 (S1522).

  When the tree structure data construction unit 141 determines that the tree structure generated in S1418 does not satisfy the tree structure condition (No in S1419), the process returns to S1416. Also, when there is a combination of partial tree patterns that have not yet been confirmed in S1521, the tree structure data construction unit 141 returns to S1416.

  The tree structure condition rule includes, for example, the following three conditions (1) to (3). That is, (1) the edge between nodes of “item name” is one-to-many. (2) The edge between the “item name” node and the “item value” node is 1: 1. However, in the case of a node group determined as a table type by the subtree pattern generation unit 140, there is a one-to-many. (3) The “item value” node has no lower nodes. When the above three conditions are satisfied, the tree structure data construction unit 141 determines that the tree structure satisfies the tree structure condition.

(Tree structure data selection process)
Next, the tree structure data selection processing of S142 of FIG. 4-4 will be described using FIG. The tree structure selection unit 142 reads the tree structure data output from the tree structure data construction unit 141 (S1421). If the read tree structure data is not one type (No in S1422), the form structure rule 121 relating to tree structure data selection is read (S1423), and the form structure rule 121 relating to tree structure data selection (FIG. 16-1). In accordance with (b) tree structure data selection rule), tree structure data selection processing is performed (S1424), and the selected tree structure data is output (S1425). On the other hand, if the read tree structure data is one type (Yes in S1422), the tree structure selection unit 142 outputs the read tree structure data (S1425).

  An example of tree structure data selection processing according to the tree structure data selection rule in S1424 of FIG. 14A will be described with reference to FIG.

  For example, as shown in FIG. 14A, the tree structure selection unit 142 displays difference information between a plurality of tree structure data to the user, and selects tree structure data using the information corrected by the user. To do. That is, the tree structure selection unit 142 takes the difference of the node information of each tree structure data (S14241), and notifies the user of the result (for example, notification that there is “abnormal”) (S14242). After S14242, the tree structure selection unit 142, for example, includes difference information of tree structure data (for example, item attributes of nodes obtained by comparing a plurality of generated tree structure data, inclusion relations at inclusion edges, (Information indicating the difference in the adjacent direction or the like at the adjacent edge) and then receiving an input of the tree structure correction information from the user through GUI (Graphical User Interface) or the like (S14243), the tree structure selecting unit 142 The tree structure data is corrected based on the structure correction information, and the corrected tree structure data is selected (S14244).

  In this way, when a plurality of tree structure data is generated, the tree structure selection unit 142 can output the tree structure data with corrections desired by the user.

  For example, as shown in FIG. 14B, when the tree structure selection unit 142 processes two or more form files, if a plurality of pieces of tree structure data are generated from one form file. Alternatively, the processing may be requested from the user after all (or a certain) form file has been processed. For example, if the tree structure selection unit 142 takes a difference in node information of each tree structure data when a plurality of tree structure data is output (S14245), the tree structure data difference information is output (cached) (S14246). ), During this process, a message “Do not register information as a form (that is, do not register the tree structure data in the form database 122)” is displayed to the user (S14247). Then, the tree structure selection unit 142 determines whether or not all the form file groups have been processed (S14248). If there is an unprocessed form file (No in S14248), the unprocessed form file (form) is selected. Process (S14249). That is, the processing after S14245 is performed. When the tree structure selection unit 142 determines that all the form file groups have been processed (Yes in S14248), the tree structure selection unit 142 reads the difference information of the tree structure data output in S14246 (S14250), and notifies the user of the result. This is performed (S14251). For example, the tree structure selection unit 142 notifies the user of difference information of tree structure data for each form file. After that, the user determines whether to execute the process of FIG. 14A on the difference information of the tree structure data of these form files, or not to convert the form file into the tree structure data.

  In this way, the tree structure selection unit 142 can execute processing of all form files without affecting the selection / conversion processing of other form files when processing two or more form files. Further, when a plurality of tree structure data is generated, the tree structure selection unit 142 can collectively display the difference information to the user.

  Further, for example, the tree structure selection unit 142 may select the tree structure data having the simplest structure among a plurality of tree structure data, as illustrated in FIG. For example, the tree structure selection unit 142 calculates a value indicating the complexity of the structure for each tree structure data (S14252). Then, the tree structure selection unit 142 selects tree structure data having the simplest structure (S14253). For example, the tree structure selection unit 142 counts the number of hierarchies of inclusion relationships in each tree structure data, and selects the tree structure data having the smallest hierarchy of inclusion relationships. Then, the tree structure selection unit 142 notifies the user of the result of selecting tree structure data (S14254).

  The tree structure data is selected by paying attention to the complexity of the structure of the tree structure data in this way, when tree structure data having an extremely complicated structure is generated by the tree structure data construction unit 141. This is because it is estimated that the data is likely to be different from the actual logical structure of the form file. That is, when a plurality of tree structure data is generated as described above, the tree structure selection unit 142 selects a tree structure data having the simplest structure, so that the tree structure closer to the logical structure of the actual form file can be obtained. Structure data can be selected.

(Form structure construction process)
Next, the form structure construction process in S6 of FIG. 4A will be described with reference to FIG. When the form structure construction unit 143 reads the tree structure data output by the tree structure estimation unit 14 (S1431), the form structure construction unit 143 determines whether or not the tree structure data of all node clusters has been acquired (S1433), and If the tree structure data has already been acquired (Yes in S1433), the node clusters are integrated (S1434). Then, the form structure construction unit 143 reads the property information output from the property information acquisition unit 134 (S1435), and adds the tree structure data to the form configuration including this property information (S1436). This is form configuration information.

  For example, the form structure construction unit 143 integrates the tree structure data generated by the tree structure estimation unit 14 into a form structure including property information (file information + document information + form meta information) as shown in FIG.

  Thereafter, when the form structure construction unit 143 confirms that the tree structure data has been acquired for all pages (all sheets) of the form file (Yes in S1437), the form structure information generated in S1436 is output to the form database 122 (S1438). ). On the other hand, if there is a page for which the tree structure data of the form file has not yet been acquired (No in S1437), the form structure construction unit 143 performs the node generation process in S133 of FIG. 4-2 in the node generation unit 133 (S133). ). If there is a node cluster that has not yet acquired tree structure data in S1433 (No in S1433), the form structure construction unit 143 selects an unacquired node cluster (S1440), and the tree structure estimation unit 14 The tree structure estimation process of S4-1 of 4-1 is performed (S5).

  In this way, the form structure construction unit 143 can register information (form structure information) obtained by integrating the tree structure data of each node cluster with property information in the form database 122.

  According to the data structure extraction apparatus 10 described above, even if the vertical and horizontal logical structures are mixed in the form file, the tree structure data of the form file can be accurately extracted. In addition, the data structure extraction unit 11 can register information in which the tree structure data of the form file is associated with the property information in the form database 122.

(Other embodiments)
The data structure extraction apparatus 10 registers the item name extracted from the form file in the item name database 123, and when receiving a new form file, refers to the item name database 123 and refers to the node of the node cluster. You may make it provide the item attribute of.

  Such a data structure extraction apparatus 10 includes an item name registration unit 137, an item name database 123, and an item name assignment unit 139 indicated by broken lines in FIG.

  When the item name registration unit 137 acquires the node information from the node generation unit 133, the item name registration unit 137 extracts a character string from a node that is likely to be a node of the item name according to the form structure rule 121 relating to the item name determination, and the item name database 123 Register with.

  The item name database 123 stores a character string extracted by the item name registration unit 137 (a character string often used for an item name).

  The item name assigning unit 139 refers to the item name database 123 and assigns an item attribute (“item name” or “item value”) to each node of the node cluster.

  According to such a data structure extraction device 10, it is possible to assign item attributes to each node of the node cluster with high accuracy. As a result, the data structure extraction device 10 can generate accurate tree structure data. In addition, the time required for the tree structure data generation process can be reduced.

(Item name registration process)
An example of the processing procedure of the item name registration unit 137 will be described with reference to FIG. The item name registration unit 137 reads the node information of the form output from the node generation unit 133 (S1371), and reads the form structure rule 121 related to the item name determination (S1372). Then, when the item name registration unit 137 determines that the node information of the form satisfies the form structure rule 121 regarding the determination of the item name (Yes in S1373), the item name registration unit 137 regards the form as a template and forms the form structure regarding the determination of the item name. Character string information is extracted from each node according to the rule 121 (S1374). Thereafter, the item name registration unit 137 omits the sentence structure from the extracted character string (S1375), and registers the extracted character string in the item name database 123 (S1376). On the other hand, when the item name registration unit 137 determines that the node information of the form does not satisfy the form structure rule 121 relating to the determination of the item name (No in S1373), the process ends.

  For example, the rule regarding the determination of the item name is a rule that it is determined that character string information is extracted from node information when any one of the following conditions (1) to (3) is satisfied. (1) A node whose character string information is empty (null) is equal to or greater than a threshold value (for example, for nodes of 50% or more of the entire form file). (2) The fill color specified for the node, or any one of the colors other than white and transparent is used above the threshold (for example, for nodes of 50% or more of the total number of form files). (3) A user defines a form file as a template, that is, a form in which a character string is assigned only to a node corresponding to an item name. In the case of (1) or (3), regarding the registration of the item name, the item name registration unit 137 registers a character string of a node whose character information is not empty as an item name, and in the case of (2), The item name registration unit 137 further includes a rule of registering a designated fill color or a character string of any fill color other than white or transparent as an item name.

  In this way, the item name assigning unit 139 can register character string information that is more likely to be an item name in the item name database 123.

(Field name assignment process)
An example of the processing procedure of the item name assigning unit 139 will be described with reference to FIG. The item name assignment unit 139 reads the node information of the node cluster (S1391), and reads the item name list of the item name database 123 (S1392). Next, the item name assigning unit 139 sets an unconfirmed node of the node cluster as the node X (S1393), and if the character string of an arbitrary node X exists on the item list (Yes in S1394), the item attribute of the node X "Item name" is assigned to (S1395), and if the character string of node X does not exist on the item list (No in S1394), the item attribute of node X is not assigned (S1396). After S1395 and S1396, if it is determined that the item name assignment unit 139 has confirmed all the nodes (that is, the processing after S1393 has been executed) (Yes in S1397), the node information of the node cluster is obtained from the subtree pattern generation unit 140. (S1398). On the other hand, if there is an unconfirmed node in the item name assignment unit 139 (No in S1397), the process returns to S1393.

  By doing in this way, the item name assignment part 139 can assign an item attribute to each node of the node cluster.

(Specific example of correction processing of ruled line frame correction unit)
Here, a specific example of the ruled line frame correction processing in the ruled line frame correction unit 20 will be described. In S23 to S26 of FIG. 4C, each unit of the correction unit 22 first reads ruled line frame information and correction rules, and performs correction processing for each correction rule. Then, each unit of the correction unit 22 reflects the update of the node information in the correction process in the ruled line frame information, and outputs the ruled line frame information to the next processing unit. For example, the ruled line frame information updated by the correction process of the dividing unit 24 is output to the deleting unit 25. Then, the deletion unit 25 performs correction processing using the ruled line frame information output from the dividing unit 24. Hereinafter, specific examples of processing of each unit of the correction unit 22 will be described in detail.

(Rule border frame processing)
First, a specific example of ruled line frame combination processing will be described with reference to FIG. FIG. 17 is a flowchart illustrating an example of ruled line frame combination processing. As shown in FIG. 17, first, the combining unit 23 acquires a combination rule group from the correction rules (S231). Next, when the ruled line frame combination condition of a certain combination rule in the combination rule group is a condition related to the type or thickness of the ruled line (the type and thickness of the ruled line in S232), the combining unit 23 performs the combination process A. Execute (S233). If the ruled line frame combination condition of the combination rule is a condition related to the character string or the fill color in the frame (character string and color in the frame in S232), the combining unit 23 executes the combination process B (S234). ).

  At this time, when the process is completed for all the combination rule groups (Yes in S235), the process is terminated. On the other hand, if the processing has not been completed for all the combination rule groups (No in S235), an unprocessed combination rule is acquired (S236), and the process returns to S232.

(Combining process A)
Here, the combining process A will be described with reference to FIG. FIG. 18 is a flowchart illustrating an example of the combining process A in S233 of FIG. As illustrated in FIG. 18, first, the combining unit 23 acquires a combination rule to be processed (S 23301). For example, a ruled line frame combination condition of the combination rule is “there is a dotted line on one side of the ruled line frame”, and the action is “join adjacent ruled line frames with dotted lines”.

Next, the combining unit 23 refers to the ruled line frame information and acquires a ruled line frame X that matches the ruled line frame combining condition (S23302). Then, the combining unit 23 acquires the ruled line direction d that matches the condition (S23303). Then, the combining unit 23 acquires ruled line frame groups Y ₁ ,..., Y _m adjacent in the d direction from the ruled line frame X (S23304).

Here, when the condition regarding the character string or the color is further set in the combining rule, the combining unit 23 determines whether or not the ruled line frames X, Y ₁ ,..., Y _m comply with the condition (S23305). ). When the ruled line frames X, Y ₁ ,..., Y _m comply with the conditions (Yes in S23305), the combining unit 23 combines the ruled line frame X and the ruled line frame groups Y ₁ ,..., Y _m in the d direction (S23306). ). If the ruled line frames X, Y ₁ ,..., Y _m do not comply with the conditions (No in S23305), the combining unit 23 returns to step S23302.

When the ruled line frames are combined, the combining unit 23 sets the combined ruled line frame X as the ruled line frame Z (S23307), and deletes the ruled line frame group Y ₁ ,..., Y _m (S23308). Further, the combining unit 23 updates adjacent edges around the ruled line frame Z (S23309). If there is another ruled line frame that matches the ruled line frame combination condition (Yes in S23310), the process returns to S23302. On the other hand, if there is no other ruled line frame that matches the ruled line frame combination condition (No in S23310), the process ends.

(Combining process B)
Next, the combining process B will be described with reference to FIG. FIG. 19 is a flowchart illustrating an example of the combining process B in S234 of FIG. As illustrated in FIG. 19, first, the combining unit 23 acquires a combination rule to be processed (S2341). For example, the rule for combining ruled lines in the combination rule is “the character string of the ruled line frame is“ (1) ”or“ application date ””, and the action is “ruled lines if the character string that matches the conditions is adjacent” To combine the frames.

Next, the combining unit 23 refers to the ruled line frame information, and acquires ruled line frame groups X, Y ₁ ,..., Y _m that match the ruled line frame combining condition (S 23402). For example, it is assumed that the character string of the ruled line frame X is “(1)” and the character string of the ruled line frame group Y ₁ ,..., Y _m is “application date”. The coupling unit 23, line border groups X and line border group _Y 1, ..., if a and _{Y m} are adjacent (Yes in S23403), ruled line frame group _Y 1, ..., adjacent direction _{d 1} of _{Y m} ,..., _Dm are acquired (S23404).

Then, the combining unit 23 combines the ruled line frame X and the ruled line frame group Y _i (i = 1,..., M) (S23405). Then, the combining unit 23 sets the combined ruled line frame X as the ruled line frame Z (S23406), and deletes the ruled line frame group Y ₁ ,..., Y _m (S23407). Further, the combining unit 23 updates adjacent edges around the ruled line frame Z (S23408). If there is another ruled line frame that matches the ruled line frame combination condition (Yes in S23409), the process returns to S23402. On the other hand, if there is no other ruled line frame that matches the ruled line frame combination condition (No in S23409), the process ends.

Note that the ruled line frames before combining are classified as shown in FIG. 20 according to the adjacent direction of adjacent ruled line frame groups Y ₁ ,..., Y _m . FIG. 20 is a diagram for explaining the combining process A in S233 and the combining process B in S234 in FIG. At this time, the format information of the ruled line frame Z in the ruled line frame combination process is expressed as follows, for example.
• Range X, Y ₁ , ..., Y _m range combined • Upper left coordinate (when d is right or lower) Upper left coordinate of X (when d is left or upper) Upper left coordinate of Y _{1 •} Lower right coordinates (if d is right or down) Y lower right coordinates (when d is above or left) of the _m lower-right coordinates, string of X (when d is the right or below) X of the string + Y ₁ character column + ... + Y (when d is above or left) string _m Y ₁ string + ... + Y _m fill color of a character string, fill color border frame X string + X of

(Rule frame division processing)
Next, a specific example of the ruled line frame dividing process will be described with reference to FIG. FIG. 21 is a flowchart illustrating an example of a ruled line frame dividing process. As shown in FIG. 21, first, the dividing unit 24 acquires a division rule group from the correction rules (S241). Next, when the ruled line frame division condition of a certain division rule in the division rule group is a condition related to the character string in the frame (character string in the frame in S242), the dividing unit 24 performs the dividing processes A, B, Perform any of C. Further, when the ruled line frame division condition of the division rule is not a condition related to the character string in the frame (other than that in S242), the dividing unit 24 executes either of the dividing processes A and B.

  Specifically, when the ruled line frame division criterion of the division rule uses an adjacent ruled line (uses an adjacent ruled line in S243), the dividing unit 24 executes a dividing process A (S244). When the ruled line frame division criterion of the division rule uses the number of divisions (the number of divisions is used in S243), the division unit 24 executes the division process B (S245). Further, when the ruled line frame division criterion of the division rule uses the character string delimiter (uses the character string delimiter in S243), the dividing unit 24 executes the division process C (S246).

  Then, when the processing has been completed for all the division rule groups (Yes in S247), the processing ends. On the other hand, if the processing has not been completed for all the division rule groups (No in S247), an unprocessed division rule is acquired (S248), and the process returns to S242.

(Division process A)
Here, the division process A will be described with reference to FIG. FIG. 22 is a flowchart illustrating an example of the division process A in S244 of FIG. As shown in FIG. 22, first, the dividing unit 24 acquires a division rule to be processed (S24401). For example, as shown in FIG. 23, the ruled line frame dividing condition of the dividing rule is “the character string of the ruled line frame is“ name address ””, and the action is “new line code following the ruled line frame in the right direction”. To divide. " In addition, as to which ruled line frame the division number and line segment interval are determined, as shown in FIG. 24, it differs depending on the reference direction. 23 and 24 are diagrams for explaining the division process A in S244 of FIG.

Next, the dividing unit 24 refers to the ruled line frame information and acquires a ruled line frame X that matches the ruled line frame dividing condition (S24402). Then, the dividing unit 24 acquires a direction d as a reference for dividing the ruled line frame X (S24403). Then, the dividing unit 24 obtains the division number n and the line segment intervals L ₁ ,..., L _n from the ruled line frame adjacent in the d direction from the ruled line frame X (S24404). Then, division unit 24, a character string is divided in accordance with the divided reference string line border X, _S 1 the divided character string, ..., and _{S n} (S24405).

At this time, if the number of divided character strings is not n (No in S24406), the dividing unit 24 outputs an error (step S24413), and the process returns to S24402. On the other hand, when the number of character string divisions is n (Yes in S24406), the dividing unit 24 generates ruled line frames Z ₁ ,..., Z _n obtained by dividing the ruled line frame X (S24407). Then, division unit 24, line border _Z 1, ..., a _{Z n,} respectively strings _S 1, ..., and stores the _{S n} (S24408).

Then, the dividing unit 24 adds ruled line frames Z ₁ ,..., Z _n to the node information (S24409), and deletes the ruled line frame X from the node information (S24410). Furthermore, the dividing unit 24 updates adjacent edges around the ruled line frame Z (S24411). If there is another ruled line frame that matches the ruled line frame dividing condition (Yes in S24412), the process returns to S24402. On the other hand, if there is no other ruled line frame that matches the ruled line frame combination condition (No in S24412), the process ends.

(Division process B)
Next, the division process B will be described with reference to FIG. FIG. 25 is a flowchart illustrating an example of the division process B in S245 of FIG. As shown in FIG. 25, first, the dividing unit 24 acquires a division rule to be processed (S24501). For example, the rule for dividing the ruled line frame of the division rule is “the character string of the ruled line frame is“ name address ””, and the action is “following the ruled line frame in one of the adjacent directions with two line feed codes. To divide. " Also, as shown in FIG. 26, the line segment interval is determined following a ruled line frame in a direction in which there are two adjacent ruled line frames. FIG. 26 is a diagram for explaining the dividing process B in S245 of FIG.

Next, the dividing unit 24 refers to the ruled line frame information, and acquires a ruled line frame X that matches the ruled line frame dividing condition (S24502). Then, the dividing unit 24 divides the character string in accordance with the character string division criterion on the ruled line frame X, and sets the divided character strings as S ₁ ,..., S _n (S 24503). Then, the dividing unit 24 acquires a direction d in which the number of adjacent lines to the ruled line frame X is n (S24504). If the direction d cannot be acquired, or if n is 1 (Yes in S24505), the process returns to S24502.

On the other hand, the direction d can be acquired, if n is not 1 (No at S24505), the dividing unit 24, a line segment distance _L 1 from the line border adjacent to the direction d from line border X, ..., acquires the _{L n} ( S24506). Then, the dividing unit 24 generates ruled line frames Z ₁ ,..., Z _n obtained by dividing the ruled line frame X (S24507). Then, division unit 24, line border _Z 1, ..., a _{Z n,} respectively strings _S 1, ..., and stores the _{S n} (S24508).

Then, the dividing unit 24 adds the ruled line frames Z ₁ ,..., Z _n to the node information (S24509), and deletes the ruled line frame X from the node information (S24510). Further, the dividing unit 24 updates adjacent edges around the ruled line frame Z (S24511). If there is another ruled line frame that matches the ruled line frame dividing condition (Yes in S24512), the process returns to S24502. On the other hand, if there is no other ruled line frame that matches the ruled line frame combination condition (No in S24512), the process ends.

(Division process C)
Next, the division process C will be described with reference to FIG. FIG. 27 is a flowchart illustrating an example of the division process C in S246 of FIG. As shown in FIG. 27, first, the dividing unit 24 acquires a division rule to be processed (S24601). For example, as shown in FIG. 28, the ruled line frame dividing condition of the dividing rule is “the character string of the ruled line frame is an item name and a check box”, and the action is “divide between the item name and the check box” It is. FIG. 28 is a diagram for explaining the division process C in S246 of FIG.

Next, the dividing unit 24 refers to the ruled line frame information and acquires a ruled line frame X that matches the ruled line frame dividing condition (S24602). Then, the dividing unit 24 divides the character string in accordance with the character string division criteria on the ruled line frame X, and sets the divided character strings as S ₁ ,..., S _n (S 24603). When n is 1 (Yes in S24604), the process returns to S24602.

On the other hand, when n is not 1 (No in S24604), the dividing unit 24 acquires line segment intervals L ₁ ,..., L _n for dividing the ruled line frame X according to the length of the character string (S24605). Then, the dividing unit 24 generates ruled line frames Z ₁ ,..., Z _n obtained by dividing the ruled line frame X (S24606). Then, division unit 24, line border _Z 1, ..., a _{Z n,} respectively strings _S 1, ..., and stores the _{S n} (S24607).

Then, the dividing unit 24 adds ruled line frames Z ₁ ,..., Z _n to the node information (S24608), and deletes the ruled line frame X from the node information (S24609). Further, the dividing unit 24 updates adjacent edges around the ruled line frame Z (S24610). If there is another ruled line frame that matches the ruled line frame dividing condition (Yes in S24611), the process returns to S24602. On the other hand, if there is no other ruled line frame that matches the ruled line frame combination condition (No in S24611), the process ends.

(Rule line frame deletion processing)
Next, a specific example of ruled line frame deletion processing will be described with reference to FIG. FIG. 29 is a flowchart illustrating an example of ruled line frame deletion processing. As shown in FIG. 29, first, the deletion unit 25 acquires a deletion rule group from the correction rules (S251). Next, when the ruled line frame deletion condition of a deletion rule in the deletion rule group is a condition related to the type and thickness of the ruled line or the fill color in the frame (the type of ruled line, the thickness, and the color in the frame in S252) ), The deletion unit 25 acquires a ruled line frame region that satisfies the condition as shown in FIG. 30 (S253). FIG. 30 is a diagram for explaining ruled line frame deletion processing. As shown in FIG. 30, the types of ruled lines specified in the condition include, for example, a dotted line and a thick line, and the fill color specified in the condition includes gray.

  Next, the deletion unit 25 acquires a ruled line frame group that satisfies the condition (S254). Then, the deletion unit 25 deletes the ruled line frame group (S255) and updates the node information (S256). If the process has been completed for all deletion rule groups (Yes in S257), the process ends. On the other hand, if the processing has not been completed for all deletion rule groups (No in S257), an unprocessed deletion rule is acquired (S258), and the process returns to S252.

(Rule frame addition processing)
Next, a specific example of ruled line frame addition processing will be described with reference to FIG. FIG. 31 is a flowchart illustrating an example of ruled line frame addition processing. As shown in FIG. 31, first, the adding unit 26 acquires an additional rule group from the correction rules (S261). Next, when the ruled line frame addition condition of an additional rule in the additional rule group is a condition regarding the presence / absence, type, and thickness of a ruled line (presence / absence of ruled line, type, thickness) in S262, the adding unit 26 An additional process A is executed (S263). When the ruled line frame addition condition of the addition rule is a condition related to the fill color or character string in the frame (character string and color in the frame in S262), the adding unit 26 executes the addition process B ( S264).

  If the process has been completed for all the additional rule groups (Yes in S265), the process ends. On the other hand, when the process has not been completed for all the additional rule groups (No in S265), an unprocessed additional rule is acquired (S266), and the process returns to S262.

(Additional processing A)
Here, the additional process A will be described with reference to FIG. FIG. 32 is a flowchart showing an example of the additional process A in S263 of FIG. As illustrated in FIG. 32, first, the adding unit 26 acquires an additional rule to be processed (S26301). For example, as shown in FIG. 33, the ruled line frame addition condition of the addition rule is “there is a ruled line in any direction of the character string”, and the action is “add a ruled line frame”.

  Next, the adding unit 26 refers to the ruled line frame information, and acquires a character string range that matches the ruled line frame adding condition (S2632). If there is no ruled line in any of the acquired character string range in the vertical and horizontal directions (No in S26303), it is determined whether there is another character string range that matches the ruled line frame addition condition (S26416). On the other hand, if there is a ruled line in any one of the upper, lower, left, and right directions of the acquired character string range (Yes in S26303), the adding unit 26 acquires the direction d with the ruled line (S26304).

If d is above (above in S26306), addition unit 26, the upper left coordinates _(x 2, _{y 2)} of the line border to get the line border set B satisfying y = _{y 2} for (S26307). Also, when d is below (under S26306), addition unit 26, lower right coordinates _(x 2, _{y 2)} of the line border to get the line border set B satisfying y = _{y 2} for (S26308). Also, if d is the left (left in S26306), addition unit 26, the upper left coordinates _(x 2, _{y 2)} of the line border to get the line border set B satisfying x = _{x 2} for (S26309). Also, if d is the right (right in S26306), addition unit 26, lower right coordinates _(x 2, _{y 2)} of the line border to get the line border set B satisfying x = _{x 2} for (S26310). An example in the case where d is on is shown in FIG. 34, and an example in the case where d is on the left is shown in FIG. 34 and 35 are diagrams for explaining the additional processing A in S263 of FIG.

  Then, the adding unit 26 obtains a connected graph G for the ruled line frame set B (S26311), and obtains a line segment range L (s, g) including (x, y) from G (S26312). Further, when d is up or down (up or down in S26313), a ruled line frame Z whose height is the height of the character string and whose width is L (s, g) is created and added to the node information (S26314). ). If d is left or right (left or right in S26313), a ruled line frame Z having a width of the character string and a height of L (s, g) is created and added to the node information (S26314). .

  If there is another character string range that matches the ruled line frame addition condition (Yes in S26316), the process returns to S2632. On the other hand, if there is no other character string range that matches the ruled line frame combination condition (No in S26316), the process ends.

(Additional processing B)
Here, the additional process B will be described with reference to FIG. FIG. 36 is a flowchart illustrating an example of the addition process B in S264 of FIG. As shown in FIG. 36, first, the adding unit 26 acquires an additional rule to be processed (S26401). For example, as shown in FIG. 37, the ruled line frame addition condition of the addition rule is “a character string is painted in gray”, and the action is “add a ruled line frame”.

  Next, the adding unit 26 refers to the ruled line frame information, and acquires a character string range that matches the ruled line frame adding condition (S26402). If there is a ruled line in any of the upper, lower, left, and right directions of the acquired character string range (Yes in S26403), the adding unit 26 executes an adding process A (S26404). On the other hand, if there is no ruled line in any of the acquired character string range in the vertical and horizontal directions (No in S26403), a ruled line frame Z is generated from the coordinate position, height, and width of the character string (S26405), and the ruled line is included in the node information. A frame Z is added (S26406).

  If there is another character string range that matches the ruled line frame addition condition (Yes in S26407), the process returns to S26402. On the other hand, if there is no other character string range that matches the ruled line frame combination condition (No in S26407), the process ends.

(effect)
The acquisition unit 21 extracts a ruled line frame from the form, and acquires at least a ruled line type or thickness, a character string in the frame, and a fill color in the frame as ruled line frame information for each ruled line frame. Further, when the ruled line frame information of a plurality of ruled line frames satisfies a preset ruled line frame combining condition, the combining unit 23 combines the plurality of ruled line frames. Further, after the processing by the combining unit 23 is performed, the dividing unit 24 divides the ruled line frame when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame dividing condition. Further, after the processing by the dividing unit 24 is executed, the deletion unit 25 deletes the ruled line frame when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame deletion condition.

  As a result, even if there is a part in which the item name or item value is described in a form that does not satisfy the prescribed requirements, the description method is corrected so that the part satisfies the requirement. And the logical relationship between the item name and the item value can be accurately estimated.

  Further, the corrected ruled frame is classified into item names and item values corresponding to the item names, and tree-structured data in which the item names and item values are associated is generated and output. Thereby, tree structure data in which the logical relationship between the item name and the item value is accurately estimated can be acquired.

  The acquisition unit 21 further extracts an area in which the character string is described from the form, and acquires at least the presence / absence of ruled lines in the upper, lower, left, and right directions of the area, the character string, and the fill color of the area as area information for each area. If a ruled line exists in either the top, bottom, left, or right direction of the area, the type or thickness of the ruled line is acquired. When the area information of the area satisfies a ruled line frame addition condition set in advance, the adding unit 26 adds a ruled line to the area. Thereby, the logical relationship between the item name and the item value can be estimated including the character string in which the ruled line frame is not described.

(Other embodiments)
As shown in FIG. 38, the ruled line frame correction unit 20 may recognize the input operation instruction (step S22a) and perform processing according to the operation instruction. FIG. 38 is a flowchart showing another example of the ruled line frame correction process in S3 of FIG. 4-1. For example, the operation instructions include tree structure data output, form output, and ruled line frame graph output. The operation when the operation instruction is tree structure data output is as described above. If the operation instruction is a form output, the ruled line frame correction unit 20 outputs the form using the node information obtained as a result of the correction process performed by the correction unit 22 (step S29a). When the operation instruction is ruled line frame graph output, the ruled line frame correction unit 20 outputs the node information as it is as a ruled line frame graph (step SS29b).

  Further, as shown in FIG. 39, the correction unit 22 of the ruled line frame correction unit 20 may sort the correction rules in advance in the order of the combination rule, the division rule, the deletion rule, and the addition rule. FIG. 39 is a flowchart showing another example of the ruled line frame correction process in S3 of FIG. 4-1. In this case, each unit can obtain the correction rule without searching.

  When form output is performed, the ruled line frame correction unit 20 sets a flag for the corrected node information, and the ruled line frame corresponding to the node information for which the flag is set is included in the node information. With reference to the coordinate information, the configuration of the ruled line frame, the character string, the fill color, and the like are rewritten.

[System configuration, etc.]
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed in each device is realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or hardware by wired logic. Can be realized as

  In addition, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

(program)
As an embodiment, the ruled line frame correction method may be implemented by a ruled line frame correction apparatus having the same function as the ruled line frame correction unit 20. In this case, the ruled line frame correction apparatus can be implemented by installing a ruled line frame correction program for executing the above ruled line frame correction as package software or online software in a desired computer. For example, the information processing apparatus can function as a ruled line frame correction apparatus by causing the information processing apparatus to execute the ruled line frame correction program. The information processing apparatus referred to here includes a desktop or notebook personal computer. In addition, the information processing apparatus includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), and slate terminals such as PDA (Personal Digital Assistants).

  The ruled line frame correction apparatus can also be implemented as a server apparatus that uses a terminal device used by a user as a client and provides the client with the above-described service related to ruled line frame correction. For example, the ruled line frame correction apparatus is implemented as a server apparatus that provides a ruled line frame correction service that receives ruled line frame information as input and outputs corrected ruled line frame information as output. In this case, the ruled line frame correction apparatus may be implemented as a Web server, or may be implemented as a cloud that provides the above-described ruled line frame correction service by outsourcing. As described above, the present invention can be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a network.

  FIG. 40 is a diagram illustrating a computer that executes a ruled line frame correction program. As shown in FIG. 40, the computer 1000 includes, for example, a memory 1010, a CPU (Central Processing Unit) 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network. Interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100, for example. For example, a mouse 1110 and a keyboard 1120 are connected to the serial port interface 1050. For example, a display 1130 is connected to the video adapter 1060.

  Here, as shown in FIG. 40, the hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each piece of information described in the above embodiment is stored in, for example, the hard disk drive 1090 or the memory 1010.

  Further, the ruled line frame correction program is stored in the hard disk drive 1090 as a program module in which a command executed by the computer 1000 is described, for example. Specifically, a program module describing each process executed by the ruled line frame correction apparatus described above is stored in the hard disk drive 1090.

  Further, data used for information processing by the ruled line frame correction program is stored as, for example, the hard disk drive 1090 as program data. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the hard disk drive 1090 to the RAM 1012 as necessary, and executes the above-described procedures.

  Note that the program module 1093 and the program data 1094 related to the ruled line frame correction program are not limited to being stored in the hard disk drive 1090. For example, the program module 1093 and the program data 1094 are stored in a removable storage medium and the CPU 1020 via the disk drive 1100 or the like. It may be read out. Alternatively, the program module 1093 and the program data 1094 related to the control program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and are transmitted via the network interface 1070. It may be read by the CPU 1020.

DESCRIPTION OF SYMBOLS 10 Data structure extraction apparatus 11 Data structure extraction part 12 Memory | storage part 13 Graph production | generation part 20 Ruled line frame correction | amendment part 21 Acquisition part 22 Correction | amendment part 23 Combining part 24 Dividing part 25 Deletion part 26 Addition part 27 Correction rule input part 28 Correction rule storage part 121 Form structure rule 122 Form database 123 Item name database 131 Operation interface identification unit 132 Form format information acquisition unit 133 Node generation unit 134 Property information acquisition unit 135 Adjacent edge generation unit 136 Inclusion edge generation unit 137 Item name registration unit 138 Node cluster unit 139 Item Name Allocation Unit 140 Partial Tree Pattern Generation Unit 141 Tree Structure Data Construction Unit 142 Tree Structure Selection Unit 143 Form Structure Construction Unit

Claims (8)

An acquisition step of extracting a ruled line frame from the form and acquiring at least a ruled line type or thickness, a character string in the frame, and a fill color in the frame as ruled line frame information for each ruled line frame;
A combination step of combining the plurality of ruled line frames when the ruled line frame information of the plurality of ruled line frames satisfies a preset ruled line frame combining condition;
A division step of dividing the ruled line frame when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame division condition after the processing by the combining step is executed;
After the processing by the dividing step is executed, when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame deletion condition, a deletion step of deleting the ruled line frame;
A ruled line frame correction method comprising:   After executing the combining step, the dividing step, and the deleting step, the ruled line frame is classified into item names of the form and item values corresponding to the item names, and the item names are associated with the item values. 2. The ruled line frame correction method according to claim 1, further comprising a tree structure data output step of generating and outputting data of a tree structure. The obtaining step further extracts an area in which a character string is described from the form, and as area information for each area, the presence / absence of ruled lines in the upper, lower, left, and right directions of the area, the character string, and the area Get at least the fill color, and if there is a ruled line in either the top, bottom, left, or right direction of the area, get the type or thickness of the ruled line,
After the process by the deletion step is executed, the method further includes an adding step of adding a ruled line to the area when the area information of the area satisfies a preset ruled line frame addition condition. The ruled line frame correction method according to claim 1 or 2.   In the combining step, at least one of the first ruled line frame and the second ruled line frame adjacent to the first ruled line frame is a specific type in which the type of ruled line is set in advance, The thickness is included in a preset specific thickness range, the character string in the frame is a specific character string set in advance, and the fill color in the frame is a specific color set in advance The said 1st ruled line frame and said 2nd ruled line frame are couple | bonded when at least 1 is satisfy | filled that there exists, The one of Claim 1 to 3 characterized by the above-mentioned. Ruled line frame correction method.   The dividing step includes at least one of the ruled line frame being a specific character string in which a character string in the frame is preset, and a fill color in the frame being a predetermined color in advance. 5. The ruled line frame correction method according to claim 1, wherein the ruled line frame is divided when the two are satisfied.   In the deleting step, the ruled line frame is a specific type in which the type of ruled line is preset, the thickness of the ruled line is included in a predetermined thickness range, and the character string in the frame Is a specific character string set in advance, and if at least one of the fill color in the frame is a specific color set in advance, the ruled line frame is deleted. 6. The ruled line frame correction method according to claim 1, wherein the ruled line frame is corrected. An acquisition unit that extracts a ruled line frame from the form, and acquires at least a ruled line type or thickness, a character string in the frame, and a fill color in the frame as ruled line frame information for each ruled line frame;
When the ruled line frame information of the plurality of ruled line frames satisfies a preset ruled line frame combining condition, a combining unit that combines the plurality of ruled line frames;
After the processing is executed by the combining unit, when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame dividing condition, a dividing unit that divides the ruled line frame;
After the processing is performed by the dividing unit, when the ruled line frame information of the ruled line frame satisfies a preset ruled line frame deletion condition, a deleting unit that deletes the ruled line frame;
A ruled line frame correction apparatus comprising:   A ruled line frame correction program for causing a computer to function as the ruled line frame correction device according to claim 7.

Images