JPH11232439A - Document picture structure analysis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely and efficiently analyze the structure of a document picture by using content information when the document picture is converted into an electronized document. SOLUTION: For learning the document structure of a whole document, the document picture of a content page is taken in at first, it is extracted in a basic rectangle at every line, a character is recognized and is analyzed. Here, chapter/clause numbers are analyzed, indexes are extracted and the page numbers of the respective indexes are extracted. The document picture of the text page is taken in, several tens of continuous pages are inputted and the basic rectangle is extracted and analyzed against the respective pages. The layout elements of a header, a footer, the page number, chapter/clause, a text, graphic/ table are identified from the layout feature of the extracted basic rectangle. All the elements except for the rectangles identified as the graphic/list are character-recognized. The index is matched with an index candidate extracted in text analysis at every index page extracted in content analysis as a matching processing and more precise index information is analyzed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document image structure analysis technique, and more particularly, to a document image structure analysis technique for recognizing a logical structure of a document and efficiently digitizing the document.

[0002]

2. Description of the Related Art With the spread of personal computers, word processors and the like, digitized documents have begun to be generalized, and electronic publishing has become active due to the ease and efficiency of editing these digitized documents. If a new electronic document is created, a standard markup language such as SGML or ODA may be used in the process.

On the other hand, there is a high demand that existing print documents be digitized and reused. Particularly, in the case of a library having a large collection of books, the collection of books can be efficiently searched and referenced by digitizing the books. The technology for realizing this is a document image structure analysis technology. This document image structure analysis technology acquires not only character recognition and area identification but also various kinds of information, such as attributes (cover, table of contents, figures, etc.), chapter sections, page numbers, from a document image read from a scanner. This is a processing technology that recognizes the logical structure of a document and stores the document efficiently.

[0004] With respect to the document image structure analysis technology, researches have been actively conducted mainly on media conversion such as character recognition and graphic recognition and analysis of the layout structure of a document image. As a study on the layout structure analysis of document images, the enlargement / reduction method [Character region extraction algorithm for mixed-mode communication, IEICE (J), J67-D, 11, pp.1277-1285 (1984-1
1)], Concatenation method [Multimedia document structure processing system, Journal of the Institute of Image Electronics Engineers of Japan, 19, 5, pp. 286-295 (1990-10)
], Region segmentation method based on marginal distribution, etc. [Document image structure analysis by split detection, IEICE (D-II), J74-D-II, 4
, Pp.491-499 (1991-04)], etc., and those using a format definition language [Document comprehension method for automatic filing, Gakushin Ron (D), J71-D, 10, pp.2
050-2058 (1988-10)], Model-based approach [Understanding document image layout based on model, IEICE Transactions on Electronics (D-II), J75-D-II, 10, pp.1673-1681 ] Has been reported. These can be said to realize separation and extraction of document layout elements (columns, figures, tables, character lines, characters, etc.).

[0005]

As shown in FIG. 1, the "document structure" of a document such as a book or a magazine includes a "logical structure" composed of chapter sections, a "linear structure" represented by a page, and an index to the text. It consists of a “reference structure”. The table of contents represents the logical structure of a document most faithfully and concisely, and by analyzing this, the logical structure of the document is obtained. Moreover, since most tables of contents do not include information such as figures and tables, analysis in area division is easy, and analyzing document images of the text based on this logical structure allows more efficient analysis. it is conceivable that. Also, the index information becomes the keyword of the book, and by analyzing the index,
It can be used as a keyword.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a more accurate and efficient document image structure analysis method using index information and index information when converting a document image into an electronic document.

[0007]

In order to achieve the above object, a document image structure analysis method according to the present invention analyzes a text obtained by first taking in a document image of a table of contents page in order to know the document structure of the entire document. (Table of Contents analysis). Next, the document image of the main body page is fetched, layout elements are checked (layout analysis), layout element matching processing is performed using information obtained by the table of contents analysis, and a logical structure, a linear structure, a reference structure, and the like are obtained. An electronic document is obtained (see FIG. 2).

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The outline of the processing performed in the present invention will be described. A document image is fetched for the entire document (table of contents and body page). Table of contents analysis: Extract each line as a basic rectangle, perform character recognition, and then analyze. Here, section number analysis, headline extraction, and page number extraction of each headline are performed. Body analysis: Input dozens of continuous pages and extract and analyze basic rectangles for each page. Here, the header / footer, page number, chapter section,
Identify layout elements such as text, figures and tables. Except for rectangles identified as figures and tables, character recognition is performed on all. Matching processing: For each page of the headline extracted by the table of contents analysis, matching is performed between the headline and the headline candidate extracted by the text analysis, and more accurate headline information is analyzed. Index analysis processing: After extracting an index page as a basic rectangle and performing column determination, an index and a page number serving as a keyword are extracted. In the table of contents and text analysis, the analysis is performed based on layout features (position, width, height, indent, etc.) of the basic rectangle extracted for each page.

With respect to the above processing, preferable and specific processing contents will be further described. 1. Basic Rectangle Extraction As a common process for all pages of a document, an image scanner reads and converts it into a binary image for each page in the case of a table of contents, and for each of the left and right pages in the case of a text. After removing noise and correcting the inclination of the converted image, a basic rectangle serving as a basic unit of a layout element is extracted by the following algorithm. (1) An adjacent black pixel area is obtained by 8-connection labeling, and a minimum rectangular area connected thereto is obtained. Next, the overlapping rectangles of the regions are integrated by their circumscribed rectangles. (2) With respect to the rectangle obtained in (1), a horizontal projection distribution is obtained, and adjacent circumscribing rectangles are integrated with a blank below a certain threshold to extract a rectangle that becomes a character string or a line. By the above processing, the character matrix, figure, table, etc. of each page are extracted as independent basic rectangles.

[0010] 2. Table of Contents Analysis Although there are several types of table of contents layouts, here, as shown on the left side of Table 1, one chapter / section is used for each line, which is used most frequently in documents. In this case, each row unit has one of the formats shown on the right side of Table 1.

2.1 Structural Analysis of Table of Contents In general, the table of contents of a document does not include information such as figures and tables.
After performing the basic rectangle extraction processing, the image is cut out line by line, character recognition is performed in line units, and converted into character codes. The section number, page number, and heading are extracted by the following processing.

2.2 Extraction of section numbers Assuming that the contents of the headings are not all numbers (headings consisting only of numbers are not considered), the first few characters of each line are extracted, and only the numbers are extracted from these, and the numbers are extracted. Characters other than (for example, "Chapter" and "Chapter" of "Chapter?", "1-1",
Remove the separator characters "-" and "." By this processing, headings having no section number are removed, and a pattern shown in Table 2 is obtained.

2.3 Identification of Page Numbers In the same manner as in the section number analysis, the last fixed number of characters is taken from each line, and only the numbers are extracted from these numbers and used as page numbers.
Hereinafter, the section number, heading, and page number of each line of the heading are called a headline set, and for a line without a page number, the page number of the lower line is used as the page number of this heading. Also, the character codes of the separators (“,”, “.”, Blank, etc.) used between the heading part and the page number part of each heading set are removed. With the above processing, the format of the table of contents is as shown on the right side of Table 1.

[0014]

[Table 1]

[0015]

[Table 2]

2.4 Section Number Correction Section number correction is described in Yamada [Conversion method of document image to ODA logical structured document, Preparatory Theory (D-II), J76-D-II, 11,
pp2274-2284 (1993-11)] could correct a section number in a certain m-level hierarchy by a matching method using a primary derived number and a secondary derived number.
However, when recognizing characters, the separator "-"
Matching may fail due to omission of “.” Or “.” Or recognition error. To avoid this problem,
When extracting the section number, the separator character is removed. Here, for the section number snr of a certain m-level hierarchy, the next expected section number is the highest m + 1 level, as shown in FIG. . These are referred to as primary derived numbers of snr, and the entirety of node numbers that can occur next to the primary derived numbers are defined as secondary derived numbers of snr. Hereinafter, the nth derived number is similarly defined.
Since these chapter numbers may include erroneous character recognition or numbers included in headings, the following correction processing is performed here on the assumption that such disturbances occur sporadically.

(1) The first number from the section number list is set as the analysis start point (snr ₀ ), and the number is always 0 or 1, and this is set as the start point. (2) The starting point is defined as snr _i , the section number of the line having the next section number from the table of contents list is defined as snr _{i + 1} , and the node number of the line having the next section number is defined as snr _{i + 2} . (3) If snr _{i + 1} is successful matching with any of the first-order derivation number of snr _i, as a new starting point snr _{i + 1,} repeating (2). (4) If snr _{i + 1} does not match any of the primary derived numbers of snr _i , the following processing is performed. (Case 1): the snr _{i +} snr _{i + 2} where the numbers of what there is an error in the character recognition are present in ₁ compares the secondary derivation number snr _i, the snr _{i + 2} if successful matching Return to (2) as the new starting point. At this time, snr _{i + 1} is also corrected. (Case 2): the difference between the table of contents list number of snr _{i + 1} of the Table of Contents list number and the snr _i that does not exist numbers in the section number part by the error of character recognition is 2, and the primary derivation number of snr _i If it is matched, snr _{i + 1} is set as the new starting point, and the section number is added to the previous table of contents set. (5) If any of (4) fails, the entire processing is regarded as failure. Note that even if this processing fails, only the node number cannot be extracted, and other processing is continuously performed.

3. Text Layout Analysis 3.1 Processing Overview The text page of the document contains layout elements such as headings, text, figures and tables (including photos), headers and footers, and page numbers. Accordingly, the text region of the document is roughly composed of a heading region, a text region, a figure / photograph region, a header / footer region, and a page number region. These areas are bordered
(Field Separator) or a blank area. For area division, an image is first input from a scanner, pre-processed such as binarization and noise removal, and then a rectangle is extracted. Each rectangle is identified from the obtained rectangle by using the feature amount. Hereinafter, details of the processing will be described according to the processing flow.

3.2 Image Input and Extraction of Feature Amount From a document having a plurality of continuous pages, two facing pages are input as one image, binarized, and noise is removed. A bounding rectangle surrounding all black pixels (Boundin
g Rectangle) and calculate the upper left corner of this circumscribed rectangle as (0,
0) Coordinates. This uses relative coordinates as one of basic square features described later, and it is necessary to align print areas. After extracting the basic rectangle for each of the left and right pages, the upper left coordinates (x, y),
Width, height, indent (alignment), and six feature values between the upper and lower rows. The indent is the displacement from the reference position at the left end of the image to the left end of each rectangle, and the line spacing is sorted into the extracted rectangles and the distance from the upper and lower rectangles. Further, character recognition is performed for each of the obtained rectangles, stored as character codes, and a basic rectangle list is created for each page. This is hereinafter referred to as a body list.

3.3 Extraction of Page Number and Header / Footer Since the page number and header / footer are always in the top row or bottom row of each page, the basic rectangles in the top and bottom rows of each page are taken as candidates. The header / footer and page number are extracted by a method described later.

3.3.1 Extraction of Header / Footer The header / footer is provided for each page, and the content is not always the same, but the position of placement is always at the top or bottom line of the page. In addition, the size of the character, that is, the height of the basic rectangle is equal, and the length is longer than the basic rectangle of the page number but shorter than the width of the circumscribed rectangle of the page. Under the above conditions, the header / footer is extracted by the following method.

Using the upper right coordinate of the feature quantity, the basic rectangle in the lowermost row of each continuous page is taken out, height statistics are separately obtained for the upper and lower sides, and the largest number is determined as a candidate. Among all the candidates, those having a length equal to or less than a certain value (basic rectangle of the page number) and those having the same width as the circumscribed rectangle of the page (basic rectangle of the text) are removed. The numbers of the upper and lower rows are calculated separately, and the larger one is set as a header / footer candidate. The basic rectangle of the header / footer candidate obtained by the above method is removed from the body list.

3.3.2 Extraction of page number In the same manner as in the extraction method of the header / footer, the basic rectangle in the top and bottom lines is examined. If the height is equal and the content is a number, this is regarded as a page number candidate. . Depending on the document, the page number is next to the header / footer. When the basic rectangle is cut out, the page number may be in the same basic rectangle as the header / footer. The outermost numbers (the left page is the first character string, and the right page is the last character string) of the candidates are taken out as page number candidates. Although the page number may be wrong due to a character recognition error, since the page numbers are in a continuous numerical order, if the number is N based on the page number of a certain page, Assuming that the page number is N-1 and the next page number is N + 1, matching is performed up to the last page, and the matching rate is 7
When it is 0% or more, the correct page number order is set, and a page number that does not match this page number order is corrected. Finally, for a page from which no page number has been extracted, a basic rectangle located at the same position as the page number of another extracted page is set as a page number candidate, and all page number candidates are removed from the text list.

3.4 Body Identification Since the body is the largest rectangle in the document, statistics of the height and the line spacing are obtained from the feature values of all the basic rectangles in the body list, and the most basic rectangle is determined as a body area candidate. . Since there are three basic rectangular patterns as shown in FIG. 4, a pattern meeting one of the following conditions is set as a text region candidate. Pattern A: There is an indent and a length to the right end of the text area. Pattern B: rectangle having the same width as the circumscribed rectangle of the page Pattern C: no indentation. In the case of a basic rectangle matching the pattern B and the pattern C, it is assumed that the previous basic rectangle belongs only to the pattern A or the pattern B. The rectangle is the basic rectangle at the end of the previous page. After the above processing is completed, those recognized as the text area are removed from the text list.

3.5 Identification of Heading All of the basic rectangles left in the text list are set as heading candidates. This includes a part of the text that failed to be identified, but is finally identified by matching with the headline.

4. 4.1 Matching process 4.1 Matching process between the table of contents heading and the body heading In the matching process, the basic rectangle in which the text among the heading candidates obtained from the body layout analysis most closely matches the heading text obtained by the table of contents analysis Is the heading. Specifically, a set of a heading set including a chapter number, a heading, and a page number is extracted from the heading set list created from the table of contents, and compared with the heading candidate at the page number. And At the time of comparison, a character string obtained by character recognition is used for the section number and the heading of the heading set. This matching rate is calculated by the method shown in the following equation. (Number of matching characters in two comparison strings) / (number of characters in long string)

5. 5. Index Analysis Processing 5.1 Column Determination After extracting the basic rectangle of the index page, the following two types of index pages are column-determined. 1. When there is a separator If there is a separator that separates columns, the column part can be separated by examining the coordinates. 2. In the case where there is no separator In the case where there is no separator, first, all pages are histogramd in the Y coordinate direction, and are divided into rows by density. Next, character recognition is performed for each line, and the obtained result is checked whether it is a character or a number. Then, a character string area that is not a number is set as a character part of the index information, and a part that is a number is set as a page number. This separates the columns of the index page. 5.2 Index Analysis Since this analysis method employs a method of sequentially analyzing each index row, regions existing in the same column are collected, and the collected columns are rearranged in descending order. Next, the column area is divided into characters on each line and divided into an index portion and a numeric portion as a page number, and an index information list is created.

[0028]

As described above, according to the present invention, it is possible to automatically analyze the logical structure of a target document to be stored, and it becomes unnecessary to input secondary information from a keyboard. The input is greatly simplified, since it is greatly reduced. Further, by analyzing the logical structure of the document, it is easy to construct a large hypertext system.

[Brief description of the drawings]

FIG. 1 shows the document structure of a document.

FIG. 2 shows a processing block diagram of the present invention.

FIG. 3 shows feature values of a basic rectangle.

FIG. 4 shows basic rectangular patterns A, B and C of the text.

FIG. 5 is an explanatory diagram of section number correction;

FIG. 6A is an explanatory diagram of section number correction;

FIG. 6B is an explanatory diagram of section number correction;

Claims (3)

[Claims] When an electronic document is created from a document having at least a table of contents page and a body page, each document image of the table of contents page and body page is fetched, and the document structure of the entire document shown in the table of contents page is analyzed. Document image structure analysis that separates and extracts layout elements by performing page layout analysis and uses the document structure information of the entire document obtained from the table of contents page analysis to perform matching processing on the body page layout elements Method. 2. The document image structure analysis method according to claim 1, wherein when analyzing the document structure of the entire document shown in the table of contents, a chapter number correction is performed in response to a character recognition error of the chapter number. 3. A document image analysis system which analyzes index information when digitizing a document including an index, divides the index into page numbers of the index, and uses the index as a keyword when searching for a book.