JP2005043990A - Document processor and document processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely generate a structured document tagged with XML or HTML from printed document of various forms. <P>SOLUTION: The document processor comprises a layout analysis part 10 for extracting sentence areas and layout information of the sentence areas, and extracting character line areas for every sentence area; a character recognition part 11 for extracting character patterns from the character line areas and converting them to character codes; a document logic element extraction part 13 for extracting document logic elements from the sentence areas in reference to a database having information for extracting document logic elements and information for the contents of the document logic elements; a document logic element reading order determination part for arranging the document logic elements in the reading order; a document structure analysis part 14 for hierarchically grouping the document logic elements based on information showing the identity of kinds of document logic elements and the layers of document logic elements; and a tree structure extraction part for forming a tree structure showing the logic relation between document logic elements and character codes based on the grouped information. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a document processing apparatus and a document processing method for processing a document.
[0002]
[Prior art]
By importing printed documents such as newspaper articles, magazines, scientific and technical literature, books, office documents, official documents, etc. from a scanner or other input device to a computer as a document image, and recognizing and encoding characters from the captured image information, There is a demand for digitizing and reusing the contents of printed documents.
[0003]
In this case, a print document is captured as an image by an image scanner into a computer, and “layout structure” and “logical structure” are extracted and associated from the captured image, and further, character recognition processing is performed and output in the reading order. It is common to perform processing. For example, a technique for extracting a chapter structure and a list structure over a plurality of pages using a framework of a probability grammar is known (for example, see Non-Patent Document 1).
[0004]
Also known is a technology that automatically extracts general-purpose document logic elements such as chapter headings, paragraphs, bullets, mathematical formulas, footnotes, headers, and footers, assigns the reading order to them, and then outputs character recognition results. (For example, refer to Patent Document 1).
[0005]
[Non-Patent Document 1]
Kenishi: “Interpretation of document logical structure using probabilistic grammar”, IEICE D-II, Vol. J79-D-II, no. 5, pp. 687-697, (1996-5)
[0006]
[Patent Document 1]
Japanese Patent Laid-Open No. 11-250041
[0007]
[Problems to be solved by the invention]
In the conventional technology, a printed document is captured as an image by an image scanner to a computer, and "layout structure" and "logical structure" are extracted and associated from the captured image, and further, character recognition processing is performed and output in the reading order. The process of doing is common. However, all of the conventional techniques only extract a specific logical element from a printed document under a specific layout condition, and the user can flexibly obtain desired logical information through detailed analysis over various printed documents. It was difficult to extract. In particular, in a document in which itemized items are described hierarchically, it is difficult to accurately extract the hierarchical itemized structure without contradiction, and to correctly convert the result into a structured document described in HTML or XML. Met.
[0008]
The present invention has been made paying attention to such problems, and the object of the present invention is to generate a structured document tagged with XML or HTML from various forms of printed documents with high accuracy. An object of the present invention is to provide a document processing apparatus and a document processing method capable of performing the above.
[0009]
[Means for Solving the Problems]
In order to solve the above-described problem, the present invention is a document processing apparatus, which extracts a text region indicating a region where a text is described from an input document image, and layouts the text region in the document image. A layout analysis unit for extracting layout information indicating a character line, a character line region extraction unit for extracting a character line region constituting the text region for each text region, and a character code by extracting a character pattern from the character line region A character recognition unit for converting to a document, information for extracting document logical elements constituting a document in the sentence area, and information on the contents of the document logical elements, and a structure for obtaining a desired tree structure Document logical element extraction for extracting a document logical element from a text region extracted by the layout analysis unit with reference to a first document model indicating a database describing a method A document logical element reading order determining unit that arranges the document logical elements extracted by the document logical element extracting unit in the reading order, information indicating the identity of the document logical elements and the hierarchy of the document logical elements And hierarchically grouping the document logical elements rearranged in the reading order by the document logical element reading order determining unit based on the second document model for structuring the document logical elements A logical element grouping unit; and a tree structure creating unit that creates a tree structure representing a logical relationship between the document logical element and the character code based on the information grouped by the document logical element grouping unit.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The document processing apparatus according to the present embodiment extracts and edits information from a plurality of document images input from a scanner or the like, and generates a tagged document using, for example, XML technology. First, a document image generating unit configured by a scanner or the like converts a plurality of paper documents into images to generate a document image.
[0011]
Each image is binarized by a scanner or the like and converted into a binary image. In addition, after generating multi-valued images during scanning, the document “Suzuki, Kubota:“ Binarization of low-resolution document images using stroke extraction based on interpolation and convexity determination ”, IEICE, Pattern Recognition and Understanding A multi-valued image may be converted into a binary image by a known technique based on “Study Group, Technical Report, PRMU99-231, pp1-8, 2000”. The binary image may be further converted into a binary image in which the inclination of the image is corrected using “Document Image Inclination Detection Method” of Japanese Patent Laid-Open No. 05-174183. Hereinafter, a binary image whose inclination is corrected by these processes will be referred to as an input image. The plurality of input images are sequentially input to a document processing apparatus described later.
[0012]
FIG. 1 is a block diagram showing a configuration of a document processing apparatus according to the present invention. The document processing apparatus includes a layout analysis unit 10, a character recognition unit 11, a document logical element extraction unit 13, a document model database 16, a document logical element extraction result editing unit 12, a document structure analysis unit 14, and a data format. And a conversion unit 15.
[0013]
In such a configuration, first, an input image is input to the layout analysis unit 10. The layout analysis processing unit 10 performs layout analysis on an input image input from the outside, and extracts partial areas having different properties such as a text area, a table area, a figure area, and a photo / picture area as a layout element group. Each extracted area can be expressed by a rectangle circumscribing the area. For example, the text area is represented by a rectangle 50 that circumscribes the text area TB as shown in FIG. In this case, the circumscribed rectangle 50 can be expressed by the position coordinates (x1, y1) of the upper left corner and the position coordinates (x2, y2) of the lower right corner of the circumscribed rectangle 50, as shown in FIG.
[0014]
In layout analysis, vertical writing and horizontal writing text areas are separated and output as different areas. One sentence area is not extracted across columns. However, in the layout analysis, areas corresponding to document logical elements such as paragraphs (paragraphs), bulleted lists (lists), mathematical expressions, and chapter headings may not be extracted.
[0015]
In each sentence area TB (or table area), character line areas Str are extracted in order as shown in FIG. 2 (Str1, Str2, Str3, Str4, Str5). In each character line area Str, as shown in FIG. Thus, the character areas Ch are similarly ordered and extracted (Ch1, Ch2, Ch3, Ch4, Ch5).
[0016]
The text area TB (or table area), the character line area Str, and the character area Ch can be described (represented) in a hierarchical manner, for example, by a tree structure as shown in FIG.
[0017]
The layout analysis unit 10 can be realized by a configuration disclosed in, for example, “Image processing method and image processing apparatus” of Japanese Patent Laid-Open No. 9-167233. In this case, character recognition processing is performed in the layout analysis unit 10 and each character in the text area is coded. Alternatively, the character recognition process may be performed immediately after the layout analysis to code each character in the text area TB.
[0018]
The character recognition unit 11 is, for example, “Ariyoshi:“ Extracting Characters from a Japanese Printed Document by Dynamic Hypothesis Generation / Verification ””, IEICE Technical Report, PRU 93-47, pp. 33-40, 1993. The character recognition is realized by the configuration disclosed in FIG. In this case, after cutting out each character area Ch from the character line area Str obtained by the layout analysis, the character pattern in the character area Ch is converted into character code information. The character recognition unit 11 outputs the character recognition results converted into the character codes to the document logical element extraction unit 13 in a state in which they are arranged in the reading order. In addition, the state of being arranged in the reading order means that the character codes are arranged from left to right in the character line in the case of a horizontal writing character line, and in the character line in the case of a vertical writing character line. Means that the character codes are arranged from top to bottom.
[0019]
Further, the character recognition unit 11 also outputs the layout analysis result subjected to the layout analysis by the layout analysis unit 10 to the document logic element extraction unit 13.
[0020]
The document logic element extraction unit 13 applies a document model (details will be described later) created in advance and stored in the document model database 16 to the layout analysis result and the character recognition result input from the character recognition unit 11. Document logical elements such as titles, authors, abstracts, headers, footers, chapter headings, paragraphs, bullets, table elements / columns, and chart captions are extracted. Here, these extracted document logical elements are referred to as document logical element extraction results.
[0021]
The document logical element extraction result editing unit 12 is a part for an operator to change or give an element name or hierarchy information to the document logical element extraction result extracted by the document logical element extraction unit 13.
[0022]
The document structure analysis unit 14 assigns the reading order to the document logical elements, rearranges the document logical elements in the reading order, refers to a document model created in advance from the document model database 16, and sets each document logical element. The document structure is extracted as a tree structure. The reading order determination method can be realized by, for example, a configuration disclosed in Japanese Patent Application Laid-Open No. 11-250041 “Document processing apparatus and document processing method”.
[0023]
The data format conversion unit 15 converts a document logical element described as a tree structure and a character recognition result into a structured document described in XML or HTML, and obtains a final output.
[0024]
Hereinafter, the document model database 16 will be described in detail.
[0025]
Conventional OCR (Optical Character Recognition) generates plain text from a document image by outputting character recognition results in the order of reading. However, an actual document may have a hierarchical structure such as chapters, sections, and bullets. In an XML / HTML document, such a document is expressed by a hierarchical tree structure. Therefore, when converting a document image into an XML / HTML document, it is necessary to extract some hierarchical structure from the set of document logical elements described above. At this time, if the structuring method is different, the tree structure of the processing result may be greatly different, and if the operator's desired tree structure cannot be obtained, there is a problem that a large correction is required after document conversion. .
[0026]
Therefore, in this embodiment, the operator designates the structuring method in advance in the document model database 16 in the form of a document model, and the system interprets this document model to generate a tree structure desired by the operator, and then converts it into text. Convert a document to a structured document.
[0027]
By using such a document model, document logical elements can be extracted from a document image with high accuracy, and the hierarchical structure of the document can be extracted with high accuracy. As a result, it is possible to efficiently and accurately generate a document with XML / HTML tags from a printed document.
[0028]
5 to 7 are diagrams showing specific examples of the user interface when the operator designates the structuring method.
[0029]
FIG. 5 shows a user interface for defining a knowledge / model of a structuring method related to document elements (details of the model contents will be described later).
[0030]
FIG. 6 shows a user interface for defining a knowledge / model of a method for extracting logical elements (details of the model contents will be described later).
[0031]
FIG. 7 shows a user interface for defining a knowledge / model related to the contents of logical elements (details of the model contents will be described later).
[0032]
The contents defined by using these user interfaces are stored in the document model database 16 of FIG. 1 and used appropriately when inputting a document.
[0033]
The document model used is
(1) Knowledge about the contents of document logic elements
(2) Knowledge about document logic element extraction method
(3) Knowledge of how to structure chapter headings and bullet structures
Etc.
[0034]
Knowledge about the contents of the document logic element described in (1) is defined using the user interface shown in FIG. First, as logical element names, “header”, “footer”, “document heading”, “chapter heading 1”, “chapter heading 2”, “chapter heading 3”, “chapter heading 4”, “chapter heading 5”, “ "Chapter Heading 6", "Chapter Heading 7", "Chapter Heading 8", "Chapter Heading 9", "Paragraph", "Ordered Bullet", "Unordered Bullet", "Define Bullet", "Figure Select one of “Caption”, “Figure Footnote”, “Table Caption”, “Table Footnote”, “Formula”, “Footnote”.
[0035]
The selection in the user interface of FIGS. 6 to 8 is performed by the user clicking a white circle at the head of each logical element name with a pointing device such as a mouse and changing it to a black circle. When the selection item has a white circle at the beginning, the item is not selected, and when the selection has a black circle, the item is selected.
[0036]
In the “hierarchy level”, hierarchy information is set using an arbitrary integer of 0 or more.
[0037]
In “Language”, select either “Japanese” or “English”.
[0038]
In “Character Line Direction”, select either “horizontal writing” or “vertical writing”.
[0039]
The upper limit and the lower limit of the size information are set in “mm” for “character size”, “block height”, and “block length”.
[0040]
Set upper and lower limits for "number of character lines" and "number of characters".
[0041]
Set the indentation amount in mm in “Indentation Width”.
[0042]
Information for determining as heading information is set in “heading information”. The heading information includes “Part 1”, “Chapter 1”, “Article 1”, “1.”, “(1)”, “▲ 1 ▼” Logical elements using numbers such as “I”, “(I)”, “(a)”, “A.”, “One”, and symbols such as “•”, “O”, “*”, “*” It is identification information. Here, these heading descriptions are expressed by two means of “definition by numerical sequence” and “definition by keyword information”.
[0043]
In “Heading Information”, “Define by Number Sequence”, the heading description is “numeric”, “Kanji”, “Roman Number”, “Circle Number”, “Alphabet”, “Kana”, “Greek Letter”, “Symbol”, It is described with 13 types of information: “period”, “hyphen”, “open parenthesis”, “close parenthesis”, “space”. At this time, by assigning “0 (none)” or “an integer greater than or equal to 1” to each piece of information, the heading description is expressed by a numerical sequence composed of 13 numbers and arranged in the order described above.
[0044]
For example, the heading description “(1)” is expressed as “100000000010” by assigning 1 to “number”, 1 to “open parenthesis”, 1 to “close parenthesis”, and 0 to other information. To do. The heading description “1.2.1” is expressed as “30000000020001” by assigning 3 to “number”, 2 to period, 1 to space, and 0 to other information.
[0045]
As described above, in the present invention, the headings of bullets and chapter headings (for example, “1.” in “1. Introduction” is used as the heading of the chapter heading) are numbers, Chinese numerals, Roman numerals, and round numerals. , Alphabets, kana, Greek letters, symbols, periods, hyphens, open parentheses, brailles, and spaces. In this case, for each character type, by assigning a number of either “0 (meaning none)” or “an integer greater than or equal to 1 (meaning the number of character types)”, the heading description is expressed by 13 numerical values. It shall be expressed by a combination of information.
[0046]
In addition, numerical information is composed of 13 numerical information by arranging numbers, Chinese numerals, Roman numerals, round numerals, alphabets, kana, Greek letters, symbols, periods, hyphens, open parentheses, braces, and spaces in this order. The heading description is expressed by the sequence information. For example, the heading description of “1.” is 1 for “number”, 0 for “Kanji number”, 0 for “Roman number”, 0 for “Round number”, 0 for “Alphabet”, 0 for “Kana”, Assign 0 to "Greek", 0 to "Symbol", 1 to "Period", 0 to "Hyphen", 0 to "Open parenthesis", 0 to "Close parenthesis", 0 to "Space" Therefore, it can be expressed as 1000000010000000.
[0047]
In the present invention, such a method is used to specify a chapter heading or item heading description in an input document. According to the present invention, since “1. Introduction” and “2. Principle” can be similarly expressed as “10000000100000”, it is possible to cope with the variety of heading descriptions of chapter headings and bullets.
[0048]
By introducing such a definition method, it is not necessary to cover all itemized heading descriptions that can appear when defining a document model.
[0049]
In the “definition by keyword information” of “heading information”, a heading description that cannot be expressed by a numerical sequence as described above is expressed as a character string as it is. However, it is complicated to cover all heading descriptions like “Article 1”, “Article 2”,..., “Article 99” at this time. Therefore, it is possible to simplify and express a portion where the number changes like “Article” with a blank or the like.
[0050]
In addition, chapter headings in the form of an arbitrary character string inserted between the open parenthesis “[” and the close parenthesis “]” and bullets with definitions can be flexibly specified by adopting keyword definitions that allow such omission. It becomes possible.
[0051]
When keyword matching is performed, it is possible to define the keyword matching status in the character line to be processed. In this case, the distance from the beginning position of the character line to the beginning position of the keyword matching result can be set in mm units as the “keyword matching position”, and the keyword matching result exceeding this value can be regarded as invalid. . In addition, it is possible to set the occupancy rate of the keyword in the character line, and it is possible to regard the keyword collation result indicating the occupancy rate below that value as invalid. Furthermore, the conditions of “keyword matching location” and “keyword occupancy” can be applied in combination, and when both conditions need to be satisfied, “AND determination” is selected in “keyword matching condition” When either one of the conditions is satisfied, “OR determination” is selected.
[0052]
Knowledge regarding the document logical element extraction method described in (2) is defined using the user interface shown in FIG. In the interface of FIG. 6, “header”, “footer”, “document heading”, “chapter heading”, “paragraph”, “ordered bullets”, “unordered bullets”, “defined bullets”, “figure” ”,“ Figure caption ”,“ Figure footnote ”,“ Table ”,“ Table caption ”,“ Table footnote ”,“ Formula ”,“ Footnote ”, etc., the extraction method can be designated.
[0053]
In the following, when “Do not extract” is selected, the logical element is extracted as a paragraph. In addition, when “reject after extraction” is selected, it is extracted once at the time of processing, but is not finally output to the tree structure. Furthermore, when “extract based on model” is selected, logical elements are extracted based on the contents defined using the user interface shown in FIG.
[0054]
For the logical element "Header", select "Do not extract", "Extract and reject at the top of the page", "Extract and reject on the right side of the page", or "Extract and reject on the left side of the page" be able to.
[0055]
For the logical element "Footer", select either "Do not extract", "Extract and reject at the bottom of the page", "Extract and reject on the right side of the page", or "Extract and reject on the left side of the page" be able to.
[0056]
In the logical element “document heading”, one of “Do not extract”, “Auto extract”, “Reject after extraction”, and “Extract based on model” can be selected.
[0057]
In the logical element “Chapter heading”, any of “Do not extract”, “Auto extract”, “Reject after extraction”, and “Extract based on model” can be selected.
[0058]
In the logical element “paragraph”, either “automatic extraction” or “extraction based on the model” can be selected.
[0059]
In the logical item “ordered bullets”, one of “Do not extract”, “Auto extract”, “Discard after extraction”, and “Extract based on model” can be selected.
[0060]
In the logical element “unordered list”, one of “Do not extract”, “Auto extract”, “Reject after extraction”, and “Extract based on model” can be selected.
[0061]
In the logical item “Bullet with definition”, one of “Do not extract”, “Auto extract”, “Reject after extraction”, and “Extract based on model” can be selected.
[0062]
For the logical element “diagram”, one of “not extract”, “auto extract”, and “reject after extraction” can be selected.
[0063]
For the logical element "Figure caption", "Do not extract", "Discard after extraction", "Extract at the top of the figure", "Extract at the bottom of the figure", "Extract at the right side of the figure", "Figure "Extract on the left side of" can be selected.
[0064]
For the logical element "Figure footnote", "Do not extract", "Discard after extraction", "Extract at the top of the figure", "Extract at the bottom of the figure", "Extract at the right side of the figure", "Figure "Extract on the left side of" can be selected.
[0065]
For logical element "table", "Do not extract", "Auto extract", "Discard after extraction", "Extract as horizontal text", "Extract as mixed vertical / horizontal text", "Lattice table "Extract as non-grid table", "Extract as non-grid table as vertical text", or "Extract non-grid table as mixed vertical / horizontal text" . In this case, a grid-like table has the property that all horizontal lines constituting the table are equal to the width of the outer frame of the table, and all vertical lines constituting the table are equal to the vertical width of the outer frame of the table. Means a table with Such a table is output according to a hierarchical structure of “table-column-table element”. At this time, the column means a group based on the arrangement of table elements.
[0066]
For the logical element “table caption”, “do not extract”, “reject after extraction”, “extract at the top of the table”, “extract at the bottom of the table”, “extract at the right side of the table”, “table "Extract on the left side of" can be selected.
[0067]
In the logical element table "footnote", "Do not extract", "Discard after extraction", "Extract at the top of the table", "Extract at the bottom of the table", "Extract at the right side of the table", "Table "Extract on the left side of" can be selected.
[0068]
In the logical element “formula”, any one of “not extract”, “extract as a diagram”, and “discard after extraction” can be selected.
[0069]
For the logical element “footnote”, one of “Do not extract”, “Auto extract”, and “Discard after extraction” can be selected.
[0070]
Knowledge regarding the structuring of the logic elements described in (3) is defined using the user interface of FIG.
[0071]
With regard to the “document structuring method”, either “execute” or “do not execute” can be selected. At this time, if “not implemented” is selected, all other definitions are invalidated, and logical elements are output flatly in the reading order. On the other hand, when “Implement” is selected, logical elements are structured based on other definitions and output as a tree structure.
[0072]
With regard to “structured bullets”, “not structured”, “structured based on indentation information”, and “structured based on heading information” can be selected. When bullets are structured based on indentation information, it is possible to set the indentation amount for determining indentation in mm units. The bullet structuring operation will be described later.
[0073]
With regard to “structuring the diagram inside the bulleted list”, it is possible to select either “leave in the bulleted structure” or “out of the bulleted list”. When a figure is left in an itemized list, it is structured in the same way as an adjacent itemized list in a document. When a figure is taken out of the bullet structure, it is output at the same level as the bullet structure after the bullet structure containing the figure in the document.
[0074]
Similarly, regarding “structuring the table inside the bulleted list”, it is possible to select either “leave in the bulleted structure” or “out of the bulleted list”. If a table is left in a bulleted list, it shall be structured in the same way as the adjacent bulleted list in the document. When the table is taken out of the bullet structure, it is output at the same level as the bullet structure after the bullet structure containing the figure in the document.
[0075]
Regarding the extraction of logical elements in the table, it is possible to select either “Do not extract” or “Extract” for “Extracting chapter headings in the table” and “Extracting bullets in the table” . At this time, when “Do not extract” is selected, each logical element is output as a paragraph.
[0076]
With regard to “giving reading order of logical elements in the table”, either “order as horizontal composition” or “order as vertical composition” can be selected. When ordering as horizontal composition, it is assumed that the logical elements at the upper left corner are ordered in the horizontal direction. When ordering as a vertical composition, it is assumed that the logical elements at the upper right end are ordered in the vertical direction.
[0077]
Regarding “ordering of document headings”, “read document headings first” or “automatically order document headings”.
[0078]
The document logical element extraction unit 13 performs the following processing with reference to the document model database 16 described above.
[0079]
First, the layout analysis result holding the character recognition result for each line is decomposed for each character line. Each character line is given information of “normal line”, “indentation line”, “centering line”, or “hard return line”. At this time, the character lines are classified into the above four categories by applying the following conditional expressions to the distance D1 between the text block boundary and the character line head position and the distance D2 between the text block boundary and the character line end position.
[0080]
Conditional expression:
When D1 ≦ TH1 and D2 ≦ TH2, the character line is a normal line.
[0081]
When D1> TH1 and D2 ≦ TH2, the character line is set as an indented line.
[0082]
When D1> TH1 and D2> TH2, the character line is set as a centering line.
[0083]
When D1 ≦ TH1 and D2> TH2, the character line is set as a hard return line.
[0084]
Chapter heading extraction is performed in the document model database 16 ("section heading" shown in FIG. 6 is selected, and "automatic extraction" or "extraction based on model" is selected in the "section heading". (Selected) or perform bulleted matching ("ordered bullets" shown in Fig. 6 are selected and "automatically extracted" or "extract based on model" in this "ordered bullets" Is selected "or" unordered bullets "is selected, and" automatic extraction "or" extract based on model "is selected in" unordered bullets ") Is specified, “heading description” is detected for each line. At this time, if the heading description of the document logical element is expressed by a number sequence in the document model, several characters are similarly expressed by the number sequence from the beginning in each character line, and the number sequence and the document model are defined. The heading description is detected by determining the coincidence of the numerical sequences.
[0085]
When the heading description of a document logical element is expressed by a keyword in the document model, the heading description is detected by performing keyword matching on each character line. At this time, if the keyword is expressed in a simplified manner as described above, a desired document logical element is extracted by performing keyword matching before and after the simplified portion without performing matching.
[0086]
FIG. 8 is a diagram illustrating an example of a document logical element extraction result.
[0087]
When a character line having a heading description and a character line following the head line are in the arrangement relationship shown in FIG. 8, the document logical elements of chapter headings or bullets are extracted by integrating them. That is, as shown in FIG. 8, among the three character lines 60 to 62, the first line 60 is a normal line, and the second and subsequent lines (61, 62) are continuously indented or indented with indentation or centering lines. If the line length of each of the character lines 60 to 62 is the same as or shorter than the line length of the previous line, these are integrated to extract the document logical element 63.
[0088]
As for the chapter headings, element names that make it possible to distinguish chapter headings at different levels such that “1.” is heading 1, “1.1” is heading 2, and “1.1.1” is heading 3. Is granted.
[0089]
As for itemization, when hierarchical information is defined corresponding to the heading description in the document model, hierarchical information is added simultaneously with extraction of logical elements.
[0090]
If a table is mixed in the document and the document model specifies that the table is to be automatically extracted ("Automatic extraction" is selected in the "Figure" shown in FIG. 6), The elements and element columns are extracted from the table, and the hierarchical structure of “table-column-table element” is extracted.
[0091]
When “extract table as text area” is specified in the document model (in the “table” shown in FIG. 6, either “automatic extraction” or the lower five to be extracted as text is selected. In each cell of the table, the text area is extracted by the method described above, and after the text area is similarly decomposed into character lines, each character line is classified into four categories, and the heading description described above is detected. Document logical elements are extracted by integrating the character lines.
[0092]
As a result, it is possible to extract a logical element similar to a normal text area from the table. At this time, when “Do not extract” is selected in “Extract chapter headings inside table” shown in FIG. 7, the document logical elements in the table having the same heading description as the chapter headings outside the table are displayed. Extract as bullets or paragraphs. Also, when “Do not extract” is selected in “Extract bullets inside table” shown in FIG. 7, the logical elements in the table having the same heading description as the bullets outside the table are paragraphs. Extract as
[0093]
The document logic element extracted by the above-described processing is output to a display device such as a display (not shown), and the document logic element extraction result editing unit 12 changes the processing result or adds information to the processing result. In this document logical element extraction result editing unit 12, in order to obtain a highly accurate processing result in the subsequent processing in the document structure analyzing unit 14, if the logical element name or the hierarchy information is incorrect, these are changed. If no hierarchy information is detected, hierarchy information is assigned to this logical element. At this time, different hierarchical information can be given to logical elements having the same heading description and indentation amount, and the tree structure extraction is performed by the document structure analysis unit 14 based on the result.
[0094]
The document structure analysis unit 14 refers to the document model database 16 and performs the following processing.
[0095]
First, the reading order is assigned to the document logical elements, and the document logical elements are rearranged according to the reading order. Further, a grouping process is performed on the document logical elements rearranged in the reading order. At this time, the hierarchical structure of the document is extracted by extracting the chapter group and the bulleted group and extracting the inclusion relation between the groups.
[0096]
When extracting a group of document logical elements, a global group such as a chapter group is preferentially extracted. In chapter group extraction, logical elements existing between chapter headings having the same element name are grouped.
[0097]
In the extraction of bulleted groups, lower grouping processing is performed in ascending order of reading order. At this time, if it is defined in the document model as “structure based on indentation”, the top position of the document logical element (the left end (upper end) of the text block in the case of horizontal writing (vertical writing) elements) Group within chapter groups based on placement relationships. For example, when the head positions of logical elements following a certain bulleted logical element are substantially the same or indented, they are considered to belong to the same group and are grouped. This grouping process ends when a logical element having a smaller starting position coordinate value than the starting position coordinate value of the starting element of the group appears or when a logical element having shallow hierarchical information appears.
[0098]
In each group, hierarchical grouping is performed by recursively performing the same grouping process every time a logical element whose head position is indented by more than the threshold TH3 as compared to the logical element located in front appears. Can be realized. When hierarchical information is given based on the model definition at the time of logical element extraction described above, and hierarchical information that means that the hierarchy is deeper than the adjacent logical elements, indentation within the group The lower group is detected regardless of the position.
[0099]
When “Structure based on heading description” is specified in the document model (“Structure based on heading description” is selected in “Bulk Structuring” in FIG. 7) The bulleted groups are extracted based on the consistency of the heading descriptions detected by the method described above within each chapter group.
[0100]
First, the last element of this chapter group is extracted as a group from the bullets that appear first in the chapter group. Next, if a bullet with a heading description different from the head element appears in the bullet group, grouping is performed until a bullet with the same heading description as the head element appears. Detect bulleted groups that are subordinate to the group. By executing such processing recursively, it is possible to detect a hierarchical itemized group.
[0101]
At this time, as described above, when the hierarchical information is given based on the model definition at the time of the logical element extraction, the lower group is extracted if the hierarchy is different even if the heading description is the same. On the contrary, even if the heading descriptions are different, if the hierarchical information is the same, the grouping process is performed as belonging to the same group.
[0102]
In the document model, when “Output diagram outside bulleted list” is specified (“Outside bullet structure” is selected in “Structure diagram inside bulleted list” in FIG. 7) Or “Exit Bullet Structure” is selected in “Structure Tables in Bullets” in FIG. 7). All figures / tables in a bulleted group retain their reading order. As it is, the same hierarchical information is output in the form given next to the top bulleted group.
[0103]
The document structure analysis unit 14 converts the result obtained by the hierarchical grouping process as described above into a tree structure. When “not structured” is specified in the document model (“not implemented” is selected in “document structured” in FIG. 7), grouping processing of chapter headings and bullets is not performed. A flat tree structure is obtained.
[0104]
When "Structure based on indentation" or "Structure based on heading description" is specified in the document model ("Based on indentation information" in "Bulk structuring" in FIG. 7) Depending on the parameters of the other document model, various tree structures are obtained. “Structure” or “Structure based on heading description” is selected.
[0105]
Hereinafter, the above-described embodiment will be described in more detail using specific examples.
[0106]
FIG. 9 shows an example in which a one-page printed document is converted into a document image. In this case, the logical elements C to M and N1 to N5 can be regarded as constituent elements of the table, but here, they are not output as the elements of the table but are output as the logical elements constituting the logical structure shown in FIG. Consider the case of doing as an example. Further, the logical element A is “heading 1”, the logical element B is “heading 2”, the logical element E and the logical element I are “heading 3”, and the logical elements F, G, H, J, K, L, M, and N1. ˜N5 is regarded as “bullet”. Also, the logic elements C and D are not output as unnecessary logic elements in the final result.
[0107]
First, a layout analysis result shown in FIG. 11 is obtained by a layout analysis process in the layout analysis unit 10. Next, by the document logical element extraction processing in the document logical element extraction unit 13, first, the logical element shown in FIG. 13 is extracted by decomposing the character line shown in FIG. 12 and then identifying the character line and identifying the heading description. Is done.
[0108]
For example, in the document model stored in the document model 16, if the keyword “Chapter” is defined as the heading description of the logical element “Heading 1”, the logical element A is identified as “Heading 1”. Similarly, if the keyword “Section” is defined as the heading description of the logical element “Heading 2” in the document model, the logical element B is identified as “Heading 2”.
[0109]
Further, if “10000000100” and “10000000000001” are defined as the heading description of the “bullet” logical element in the document model, the logical elements F, G, H, J, K, L, and N1 to N5 are “ Identified as "Bullet". In this case, since the “heading 2” of the logical elements E and I and the “bullet” of the logical elements K and L have the same heading description, it has been difficult to distinguish them as different logical elements in the prior art.
[0110]
However, since the logical elements E and I have a smaller block width than the logical elements K and L, in this embodiment, by defining a model that combines the block width and the heading description, the logical elements E and I are discriminated as different logical elements. Is possible.
[0111]
The document model has a heading description of “10000000100000”, a hierarchical level of bullets with a character line height of α mm or more, 1 and a hierarchical level of bullets with a heading description of “100000000010”, “100000000100000”. If the hierarchical level of the itemized list with the text line height of β mm or less is defined as 3 respectively, the logical elements F, G, and H are the ordered itemized item 70 of the hierarchical level 1, logical Elements J, K, and L can be identified as ordered bullets 71 at hierarchical level 2, and logical elements N1-N5 can be identified as ordered bullets 72 at hierarchical level 3.
[0112]
The document structure analysis unit 14 first assigns the reading order to the document logical elements, and then rearranges the document logical elements according to the reading order as shown in FIG. As a result, the logic elements A, B, E, F, G, H, I, J, K, L, M, N1, N2, N3, N4, and N5 are arranged in this order.
[0113]
Next, a hierarchical group of logical elements is detected by performing a grouping process. In the example of FIG. 14, the groups A, B, E, F, G, H, I, J, K, L, M, N1, N2, N3, N4, and N5 of “Heading 1” are first detected. Next, from the “Heading 1” group, the groups B, E, F, G, H, I, J, K, L, M, N1, N2, N3, N4, N5 of “Heading 2” and “Heading 3” E, F, G, H and I, J, K, L, M, N1, N2, N3, N4, and N5 are detected.
[0114]
Then, using the sameness of heading information and hierarchical information as clues, “ordered list” groups F, G, H, J, K, L, M, N1, N2, N3, N4, N5, K, L, M, N 1, N 2, N 3, N 4, N 5, N 1, N 2, N 3, N 4, and N 5 are detected hierarchically from each “Heading 3” group.
[0115]
The group thus obtained is converted into a tree structure shown in FIG. 15 based on the inclusion relationship between the groups.
[0116]
As described above, according to the above-described embodiment, document headings, authors, abstracts, dates, chapter headings, paragraphs, bullets, footnotes, captions, mathematical expressions are converted from document images converted from various printed documents composed of a plurality of pages. Since document logical elements such as headers and footers are automatically extracted, the hierarchical structure between the document logical elements is extracted with high precision so that the document image is represented by a tree structure of the document logical elements. A structured document tagged with XML, HTML, or the like can be generated with high accuracy, and can be automatically input to a computer system.
[0117]
In addition, a complicated bullet structure with a multi-level hierarchical structure can be expressed in a tree structure as intended by the operator, which makes it possible to greatly reduce the complexity of editing structured documents, Operator editing work in computerization can be greatly reduced.
[0118]
(Modification 1)
According to the present invention, the print document shown in FIG. 9 can be converted into an XML document having another logical structure.
[0119]
For example, when each of the logical elements A, B, E, and I shown in FIG. 9 is considered as an itemized list, a document model database 16 having the following model definition is created and a structuring process is performed based on the document model database 16. Thus, the tree structure shown in FIG. 16 can be extracted and then converted into the XML document shown in FIG.
[0120]
An example of the document model definition is as follows.
[0121]
Logical elements having heading descriptions of “Chapter 1”, “Chapter 2”, “Chapter 3”,... Are regarded as itemized items, and this hierarchical information is defined as level 1.
[0122]
Logical elements having heading descriptions of “Section 1”, “Section 2”, “Section 3”,... Are regarded as itemized items, and this hierarchical information is defined as level 2.
[0123]
If logical elements having heading descriptions of “(1)”, “(2)”, “(3)”,... Are regarded as bullets, and the character size (height of the character line) is less than γmm, the hierarchy information Is level 3, and if the height of the character line is λ mm or more, the hierarchical information is level 5.
[0124]
Logical elements having heading descriptions of “1.”, “2.”, “3.”,... Are regarded as itemized items. If the height is less than βmm, the hierarchy information is set to level 6.
[0125]
When the document model is defined as described above, as described above, the level 1 bulleted grouping process, the level 2 bulleted grouping process, the level 3 bulleted grouping process, and the level 4 bulleted grouping process are performed. The hierarchical group shown in FIG. 17 can be extracted by sequentially performing the itemized grouping process, the level 5 itemized grouping process, and the level 6 itemized grouping process in a hierarchical manner.
[0126]
Thereafter, the tree structure shown in FIG. 16 is extracted by the document structure analysis unit 14, and further converted into an XML document having the tree structure shown in FIG. 18 by the data format conversion unit 15.
[0127]
(Modification 2)
Further, in the printed document shown in FIG. 9, a document model database 16 having a model definition is considered so that the logical element A is regarded as a level 1 heading, the logical element B as a level 2 heading, and the logical elements C to N5 as table elements. By creating and executing a structuring process based on this, after extracting the tree structure of FIG. 20 based on the following processing procedure, it is possible to convert it into the XML document shown in FIG.
[0128]
For example, after a horizontal line segment and a vertical line segment are extracted from a document image, an area surrounded by the extracted line segment is extracted as a table element.
[0129]
As a result, as shown in FIG. 19, the document structure analysis unit 14 combines the logical element C into the table element 1, the logical element D into the table element 2, the logical element E into the table element 3, and the logical elements F to H into the table element. 4. Extract logical element I as table element 5 and logical elements J to N5 as table element 6 together.
[0130]
Next, the document structure analysis unit 14 groups the table elements adjacent in the horizontal direction. Specifically, table elements 1 and 2 are defined as a table element group a, table elements 3 and 4 as a table element group b, and table elements 5 and 6 as a table element group c.
[0131]
Next, the document structure analysis unit 14 groups the table element groups a to c and extracts the table group shown in FIG.
[0132]
As a result, the document structure analysis unit 14 extracts the tree structure shown in FIG. The extracted tree structure is converted by the data format conversion unit 15 into an XML document shown in FIG.
[0133]
As described above, the XML document shown in FIG. 22 can be generated from the print document shown in FIG.
[0134]
【The invention's effect】
According to the present invention, a structured document tagged with XML, HTML, or the like can be generated with high accuracy from various types of printed documents.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a document processing apparatus according to the present invention.
FIG. 2 is a diagram illustrating an example of a layout analysis result.
FIG. 3 is a diagram for explaining a circumscribed rectangle.
FIG. 4 is a diagram showing a sentence area, a character line area, and a character area described hierarchically.
FIG. 5 is a diagram showing a user interface for defining a knowledge model of a structuring method related to document elements.
FIG. 6 is a diagram showing a user interface for defining a knowledge model related to the contents of logical elements.
FIG. 7 is a diagram showing a user interface for defining a knowledge / model of a logical element extraction method;
FIG. 8 is a diagram illustrating an example of a document logical element extraction result.
FIG. 9 is a diagram illustrating an example of a document image to be processed.
FIG. 10 is a diagram illustrating an example of a document structure analysis result.
FIG. 11 is a diagram illustrating an example of a layout analysis result.
FIG. 12 is a diagram illustrating an example of document logical element extraction.
FIG. 13 is a diagram illustrating an example of a document logical element extraction result.
FIG. 14 is a diagram showing an example of hierarchical grouping of logical elements in document structure analysis processing.
FIG. 15 is a diagram illustrating an example of processing results of document structure analysis and data format conversion.
FIG. 16 is a diagram illustrating another example of a document structure analysis result.
FIG. 17 is a diagram illustrating another example of hierarchical grouping of logical elements in document structure analysis processing.
FIG. 18 is a diagram illustrating another example of processing results of document structure analysis and data format conversion.
FIG. 19 is a diagram illustrating another example of a document logical element extraction result.
FIG. 20 is a diagram illustrating another example of a document structure analysis result.
FIG. 21 is a diagram illustrating another example of hierarchical grouping of logical elements in document structure analysis processing;
FIG. 22 is a diagram illustrating another example of processing results of document structure analysis and data format conversion.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Layout analysis part, 11 ... Character recognition part, 12 ... Document logic element extraction result edit part, 13 ... Document logic element extraction part, 14 ... Document structure analysis part, 15 ... Data format conversion part, 16 ... Model database.

Claims (8)

A layout analysis unit that extracts a text region indicating a region where a text is described from the input document image, and extracts layout information indicating a layout of the text image in the document image;
For each sentence area, a character line area extracting unit that extracts a character line area constituting the sentence area;
A character recognition unit that extracts a character pattern from the character line area and converts it into a character code;
A database showing a structuring method for obtaining a desired tree structure having information for extracting document logical elements constituting a document in the sentence area and information on contents of the document logical elements. A document logical element extraction unit that extracts a document logical element from a text region extracted by the layout analysis unit with reference to one document model;
A document logical element reading order determining unit that arranges the document logical elements extracted by the document logical element extracting unit in reading order;
Based on the second document model for structuring the document logical elements, the document logical element reading order determining unit has information indicating the identity of the document logical elements and the hierarchy of the document logical elements. A document logical element grouping unit that hierarchically groups document logical elements rearranged in order,
A document processing apparatus comprising: a tree structure creation unit that creates a tree structure representing a logical relationship between document logic elements and character codes based on information grouped by the document logic element grouping unit . The document model related to the method for extracting the document logical element is not extracted, automatically extracted, extracted based on the document model related to the contents of the document logical element, rejected after extraction, the position on the page and other logics. 2. The document processing apparatus according to claim 1, wherein extraction of a document logical element from a table element constituting a table is extracted based on an arrangement relationship with the element. The document model related to the contents of the document logical element is composed of the type of the document logical element, the geometric information of the document logical element, the keyword information existing in the document logical element, and the heading description information of the first line of the document logical element. The document processing apparatus according to claim 1. The keyword information existing in the document logical element is composed of a character string that constitutes the keyword, an arrangement position of the keyword in the character line including the keyword, and an occupancy ratio of the keyword in the character line including the keyword. The document processing apparatus according to claim 3. 4. The document processing apparatus according to claim 3, wherein the heading description information of the first line of the document logical element is composed of a number sequence expressed by the number of character types constituting the heading information. The document logical element extraction unit includes the heading description information of the first line of the document logical element, which is composed of a number sequence represented by the number of character types constituting the heading information, and the character of the first line of the document logical element. 2. The document processing apparatus according to claim 1, wherein the document logical element is recognized by comparing the number of character types extracted from the code information and a number sequence expressed more. The document model relating to the method of structuring the document logical elements relates to information relating to the structuring method of bullets, information relating to the method of structuring diagrams located in the bullets, and to the method of structuring chapter headings and bullets in the table. 2. The document processing apparatus according to claim 1, wherein the document processing apparatus comprises information, information on a method for ordering logical elements in a table, and information on a method for ordering document headings. A layout analysis step of extracting a text area indicating an area in which text is described from the input document image, and extracting layout information indicating a layout in the text image of the text area;
A character line area extracting step for extracting a character line area constituting the sentence area for each sentence area;
A character recognition step of extracting a character pattern from the character line region and converting it into a character code;
A database showing a structuring method for obtaining a desired tree structure having information for extracting document logical elements constituting a document in the sentence area and information on contents of the document logical elements. A document logic element extraction step of extracting a document logic element from the sentence area extracted in the layout analysis step with reference to one document model;
A document logical element reading order determining step for arranging the document logical elements extracted in the document logical element extracting step in the reading order;
Based on the second document model for structuring the document logical elements, the information indicating the identity of the document logical elements and the hierarchy of the document logical elements is read in the document logical element reading order determination step. A document logical element grouping step for hierarchically grouping document logical elements rearranged in order;
A document processing method comprising: a tree structure creation step for creating a tree structure representing a logical relationship between document logic elements and character codes based on the information grouped in the document logic element grouping step .

Images