JP3472038B2 - Logical identification of document elements - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention
Compare the white area required with the structural model to the columns in the document.
Document elements of the input document by analyzing the structure
Regarding the classification of (components), in particular the specific text
Acceptable color for a book, document type, or print
Statements in columns using a structural model that represents the system layout
Method and apparatus for logically identifying document elements
You. [0002] Documents to be Solved by the Related Art
A function that logically identifies an element (component) in an image
The next method is, for example, "A pro
to type Document Image Analysis System for technica
l Journal ", pp10-21, Computer, July 1992.
ing. Until now, documents have been analyzed based on the column structure.
Many methods have been developed, but all have column structures
Was based on general estimates of like this
In an accurate estimation-based method, the actual column structure
Question when there are parts or exceptions that differ from the general column structure
There was a title. Usually different source documents have different exceptions
So we can't afford to ignore such exceptions. In addition, the document structure does not depend on the context.
Page layout of the original document that can be expressed by grammar rules
Document structure can also be described by identifying documents
(See Naji above). However, this method is complicated
Different structure from image level to pixel level
Requires strict grammar rules expressed in fineness. Of the original document
Representing all levels of the document structure is a complex and difficult task.
It is a business, and simpler expressions are desired. Japanese Patent Application No. 7-243213 discloses a background area.
Method and apparatus for splitting a document by parsing
Has been disclosed. This method takes into account page layout
Without document, based on key white areas
Extract the rectangular document element. Also, Japanese Patent Application No. 7-243212 discloses a
Page images by matching site space patterns
A method for logically tagging is disclosed. This method uses the ratio
Suitable for identifying document pages with relatively fixed layout
You. However, various elements were arranged in various orders
To apply this method to a flexible layout,
List all layouts
The problem of having to express in the corresponding document structure
is there. There's nothing you can't do, but
Is a tedious process, again a simpler
Expression is desired. SUMMARY OF THE INVENTION The present invention is directed to different types
Use a single column structure model for each of the documents
, Add a “heading” to the document element in each column of the input document.
Logical tags such as "
Provide an accurate and efficient way to identify document elements
You. In the present invention, between document elements and between document elements
The sequence of the main white areas in the document
By matching with Dell, the color of the input document
The system structure is analyzed. [0007] One aspect of the present invention is a document element in a document image.
A method for logically identifying a statement, wherein at least one
At least one column containing one document element
At least one structural model corresponding to the original document
Generating, wherein each structural model comprises at least
At least one element with one column representation
At least one document element of the source document that has a representation
The first key background area in the document image
Identifying a region, wherein the first primary background region comprises a sentence.
The document image into at least one column,
Identify a second key background area in each of the columns
A second primary background area identified, comprising the step of identifying
To main background area pattern for each column of document image
For each column of the document image
At least one matching main background area pattern
Main step with generating column strings
At least one column string that matches the pattern
To determine the best column string from the list
And the best column string has at least one
Best match the column expression of the
At least one sentence in the document image based on the tring
Logically identifying each of the write elements
You. The present invention also provides a document element in a document column.
The size and orientation of the main white area separating the
Allows users to develop structural models using information
The relationship between document elements in the document image column.
Provide a system that can be expressed. The main white area is the “white space”
A rectangular area whose minimum size is predetermined
ing. White space is the background of the document image,
Area or non-text area. Normal documents are
Black and / or colored on white background area
Because these images are formed, these background areas are
It is called "it space", but in the interpretation of the present invention
Is when the document is formed in a colored background area, or
Form text with white outline on black or color background
These colors are non-image areas even if
(Or black) background area defined as "white space"
I do. With the present invention, including document elements
Analyze the document image part and coherent group,
That is, there is no need to detect connected components from document elements.
No. The image on the document is scanned and the electronic or
A digital representation is formed. Area containing the document element
May have other document
These documents, if separated from the element area
The element regions are considered to be independent of each other.
Document elements include headings, text,
Major white areas including information such as graphics
These are rectangular areas separated from each other by areas. [0011] The structural model is a possible tolerance in the original document.
It represents a specific possible column layout.
Structural models have two types of regular expressions: column tables
It is represented by the present and element expressions. Element of structural model
Portrayal of the original document prior to the actual tagging process.
Users or creators provide offline,
Or from a real training sample from the original document
It is determined. Column representation of the structural model is actually tagged
User or creator of the original document prior to the process
Generated offline. Acceptable power for original document
The ram layout is shown in the structural model as a column representation.
Is done. Element representations are identified and
Statement for each element type used in the ram representation
All possible white space in the main white area
Sequence. The document element identification system according to the present invention comprises:
Key white area sequence or pattern of the input document image
All of the parts of the
Extract possible matches. Next, input the input document
For each column of the page, the main image of the input document page image
At least one column string that indicates a white area
Generate. In addition, the main white area of the input document image
All pairs of matching element expressions to match
Compare the combination with the column representation of the structural model. plural
A set of column strings or matching element expressions
If the match is continuously tested against the column representation
The best match (ie, the closest match)
Touching). Based on a comparison with the input document image, the closest
Logic related to the document element of the matching column expression
Using tags, the corresponding document elements of the input document image
And logically identify document elements. Alternatively, the corresponding document file of the input document image
Elements are split and output for printing or storage
Is done. For other objects and effects of the present invention, see the drawings.
Further clarification from the following detailed description of the preferred embodiments
Be certain. FIG. 1 shows a document element of the present invention.
1 shows a preferred embodiment of the identification system 100. documents
The element identification system 100 extracts the document white area.
Output system 110, main white area selecting means 120,
Memory 130, processor 140, structural model definition means
150, column string selecting means 160, column expression
Comparing means 170, logical tag assigning means 180, and
Including document element extraction means 190, all of which
They are connected to each other via a bus means 105. Shown in FIG.
The document element identification system 100 is
It is preferred to run on pewter 300, but of course
Dedicated computer, microprocessor-based or
Microcontroller-based systems, ASICs, etc.
Hard wires for integrated circuits, discrete element circuits, etc.
Electronic circuit, field programmable gate array
Such as programmable logic devices (PLDs)
But it can be done. FIG. 2 shows the document element identification system of FIG.
Of the document white area extraction system 110 of the
3 shows a preferred embodiment. As shown, document white area
The region extraction system 110 includes a continuous component
Bounding box generation means 250, and main white area extraction
Outgoing means 240, all of which are bus means 105
Connected to. First, the scanner 210 or the memory 13
0, the document image data is input to the connected element identifying means 260.
Power. The memory 130 is included in the general-purpose computer 300.
Internal memory, disk drive, CD-RO
In the form of M, EPROM, etc.
It may be arranged outside the computer 300. Scanner 21
0 to the connected element identifying means 260
Before inputting, it may be stored in the memory 130 first. Sentence
The writing image data can be a binary (binary) image or
Connected element identification means 260 in the form of digital signals
Is input to Each set of 1 bit or more is
Indicates whether a particular pixel is on or off. The connection element identification means 260 stores the document image data
When the data is received, all the link elements in the document image are
Detect element. FIG. 4 is an example of a document image 400. Communicating
Connected element 410 is surrounded by “off” pixels (white pixels)
A series of adjacent “on” pixels (black pixels)
Is done. Detecting connected component 410 in document image 400
Systems are well known in the art.
You. Identify the connected component 410 of the document image 400
Then, each bounding element is
Generate a bounding box 420 for each 410. In the industry
As is well known, bounding box 420 includes connecting element 410
Is the smallest rectangular box that completely encloses. Connection required
A system for generating a bounding box 420 from a prime 410
Systems are also well known in the art. Document image data including bounding box information
Is sent to the main white area extracting means 240. Major
The white area extracting means 240 is shown in FIGS.
As shown in FIG.
Extract the white area. The document white region extraction system 110
In a preferred embodiment, the primary white region extraction means 240
Are the two sections shown in FIG. 3, namely the vertical extractor
241 and a horizontal extraction unit 242. Vertical extractor
241 and the horizontal extraction unit 242 each have a primary (prime
Iv) white region extracting means 243, comparing means 244,
An erasing unit 245 and a grouping unit 246 are provided.
Are connected to the bus means 105. Vertical extractor 241 and horizontal
The extraction unit 242 includes the same components and operates in a similar manner.
Make. As shown in FIG. 5, the horizontal extractor 242 is
, Primary (primitive) white areas 430-1 to 430-4
30-10, and extract the main white area 46 in the horizontal direction.
0 is formed. Similarly, as shown in FIG.
21 is a primary white area 430-11 to 430-19
To form a main white area 460 in the vertical direction
You. Formation of Horizontal Main White Area 460
Of the horizontal primary white areas 430-1 to 430-10
Adjacent areas can be grouped together according to specific rules.
One or more groups merged horizontally and grouped horizontally
This is effected by making it a primary white area. As well
In addition, the formation of the main white region 460 in the vertical direction is
Next to the next white area 430-11 to 430-19
Areas are grouped together according to specific rules
Is effected by merging into
One or more primary white areas grouped in
it can. Glue vertical and horizontal primary white areas
And merged, the horizontal primary white area 430 and
In the primary white area, grouped horizontally and horizontally,
It has a width wider than the threshold width 440 and a threshold height 45
Identify areas with heights greater than zero. Similarly, the vertical primary
Primary white grouped vertically with white area 430
Height above the threshold height of 450 '
Identifies an area wider than the threshold width 440 '.
These identified areas are the primary white areas. As shown in FIG. 7, various document elements
470 is defined by a vertical major white area 460.
And group them into a column 405. In the example of FIG.
The writing image 400 has two columns 405-1 and 405-2.
Divided into However, the logical identification of each divided area
Has not been done yet. In other words, from here,
Document elements are identified using one or more structural models.
I need to separate it. The document white area extraction system 110
Therefore, the main white area was identified as described above.
If the main white area selection means 120 determines
Select various regions of the default region. Structural model definition means 1
50, the user can specify the structural model column expression and element.
It may be possible to input a statement expression. Original document creation
When a structural model is given in advance by an adult
In this case, the structural model definition means 150 is not required. Columns
The tring selection means 160 selects a selected item in the document image column.
In the column of the input page using the main white area
Column streams corresponding to the primary white region sequence
Identify Column string selecting means 160
Also performs element candidate extraction processing, and
Document representation in the main white area
Match with the sequence. Column expression comparison means 17
0 is the selected major white area in the document image column
Character string or column stream generated from
Is compared with the column representation of the corresponding structural model. Mosquito
Column expression ratio performed by ram expression comparison means 170
The comparison process consists of the structure matching process and the best match
A column expression selection process for identifying a column expression.
Next, the logical tag assigning means 180
Logical tags based on the column structure
Assign to areas between white areas that need to be
Document element whose output means 190 is logically tagged
Extract 470. Alternatively, logical tag assigning means 1
In place of 80 or added to logical tag assigning means 180
Then, a document element extracting means can be used. Argument
Logical tag assigning means 180, document element extracting means 1
90 and details of the structural model definition means 150
Are described in Japanese Patent Application Nos. 7-243212 and 7-2.
43213. The document white area extraction system 110
One of the systems that performs the document white area extraction function described above
However, the present invention is not limited to this. As shown in FIG. 7, the document image 400 is optional.
Of document elements 470. Sentence of input document image
Key existing between or within document elements
Spatial (geometric) relationships between white areas and each original document
Spatial defined in the structural model selected for
By comparing with the (geometric) relationship, the document image 4
00 document element 470 is logically identified. Document image
The geometric relationship between the main white areas of column 405 is
If it matches the relationship represented by the modeling model column expression,
Document image color of document element 470 of document image 400
The system layout is specified. Next, the document image 400
Extract document element 470 by major white area
I do. The extracted document element 470 contains the structure of the original document.
Logical element of the corresponding element type in the model column representation
Assign Logical tags represent different types of document elements.
470, for example, "heading", "sentence"
Chapters and figures. Logical element for document element 4
70, the document element 470
Are logically identified. However, the document element of the document image 400
Before logically identifying 470, at least
And one structural model, the document element identification system
Must be supplied to the system 100. Each structural model is
The document element type corresponding to the source document and the corresponding source
The spatial (geometric) relationships between the document elements of the document
Including. The structural model is based on the column layout that meets the criteria.
Out structure is described, two types of rules
Expression, ie, column expression and element expression
It is. The column expression is the source text in any source document.
That meet all possible criteria contained in each column of the
Represents a element type sequence. Element representation
Is available for each element type in the source document
The main white area sequence that meets various criteria.
The definition of the element representation can be included in the column representation. each
Different known column layouts for each source document
Rules for each particular source document.
A model is provided. In a preferred embodiment of the present invention
Means that the column expression of the structural model is one-column
n) Although it is the expression of page layout, the system of the present invention
Method and method, two columns, three columns, and more generalized n
Also applicable to column page layout. Implementation of the present invention
In the embodiment, the original document is stored in the document element identification system 100.
Is assumed to be supplied.
Generate a formula document image and logically add it to the document element
Tag it. The document elements are arranged in order from the top of the column to the bottom.
Be placed. The position of each document element in the column is
According to the column layout rules of the structural model of the original document
Is determined. Columns that meet the criteria for each source document column
Layouts are represented by at least one column expression.
You. Each column has a top and a bottom, and each source document is
Allowable document elements such as "Heading" and "Text"
Type set. In a preferred embodiment,
The user of the document is within the tolerance of the structural model of the original document
Rule set that defines the column layout of
Each original text from the creator or source of the original document.
The column expression of the book may be obtained. In the following, the element type and column
Here is an example of a regular expression. Actual column or structural model
The regular expression given to the column expression of
Is represented. Column = ＾ F * (H? T |
F) (HT) * F * ＄ (1) As can be seen from equation (1), the proper document element
The set of font types is “Heading (H)”, “Figure
(F) "and" text (T) ". In addition, the original document
The following rules apply as the rules for. 1. Headings are
It is not located at the bottom of the column. 2. For each column,
Place at least one text block. 3. Te
Make sure that the text block is located below the heading block.
You. 4. Sequences containing one or more figures must be
Touch the top or bottom of the The symbols H, T and F are
The respective element type of the document element
It means "heading", "text", and "figure". Sign
“＾” indicates the top of the column, that is, the start of the column expression.
Express. The symbol “＄” is the bottom of the column, that is, the column
Indicates the end of the expression. The sign "?" Indicates that the previous expression is
Indicates optional. The sign "*" precedes
Indicates that the expression can be repeated an indefinite number of times or does not appear
You. That is, "*" indicates that the preceding expression does not appear at all.
Indicates that it may not appear or may appear more than once. "|"
Means logical OR. Parentheses () indicate the operator's "natural"
Disregarding the order of priority, “*” is at the top and element at the second
Type connection and end with "|"
You. FIG. 8 shows three original documents 401 that meet the criteria.
Column layouts 405-3, 405-4, 405-
5 is shown. Each acceptable column 405-3, 405-4,
Element sequence corresponding to each of 405-5
Are represented by the following equations (2) to (4). Column = FHT (2) Column = HTFF (3) Column = FHF (4) Using a similar process in the structural model of each original document
The element representation of the identified element type
Justify. Element for each element type in the original document
The default representation is preferably generated by the user of the original document.
Each element type has the same logical tag,
To one of the categories of document elements that have
Respond. All element types are shown in the following equation (5).
Are represented by regular expressions. Element = AW * B (5) where “Element” is “text”, “heading”
Represents an element type such as
*, B indicates a regular expression. "A" is the document element
Main white area located above (Above)
Character code assigned according to the size of
Is a regular expression that matches. "B" is
The main e located below the document element (Below)
Characters assigned according to the size of the white area
This is a regular expression that matches the dictator codes.
“*” Is a repetition operator. "W" is the document element
Main white area located inside the client (Within)
Character code assigned according to the size of
Is a regular expression that matches. Repeatable
"W *" indicates that if the range of the number of repetitions is known,
A range may be substituted. For example, W {0,3}
Indicates that W is repeated 0 to 3 times. One of the element types, "text"
Element is located in the center of the column.
The expression is a regular expression represented by the following expression (6). Text = [fg] [ab] * [fg] (6) where a, b, f, and g are 3 points and 5 poi, respectively.
Points, 16 points and 20 points
You. These values come from the sample data
The element expression is the solution of the sample data of each original document.
Generated automatically from analysis. Each text is an adjacent line in the text
Between 3 and 5 points above the text
Or another adjacent document element located below
Separated by ~ 20 point line spacing. Turning to FIG. 9, document element 406-1
Contains text elements in columns. Document element
In text 406-2, the text starts at the top of the column.
This element representation is
Begins with "＾". In document element 460-3, text
The strike must descend to the bottom of the column,
The statement expression ends with the sign "@". Document element 4
In 60-4, the text is from the top to the bottom of the column.
, So the element expression starts with the sign “＾”,
End with "＄". Four of the "text" document elements of FIG.
406-1, 406-2, 406-3, 406
-4 is a regular expression shown in equations (6) to (9), respectively.
Corresponding. Text = {[ab] * [fg] (7) Text = [fg] [ab] *} (8) Text = {[ab] *} (9) Linking these four regular element expressions together
Connect or perform a logical OR operation on this element
Find the type (text) element representation. Paraphrase
Then, the four regular expressions shown in Expressions (6) to (9) are
Combined into two element expressions, as shown in the following equation (10)
Thus, all four configurations are covered. Text = [fg] [ab] * [fg] | {[ab] * [fg] | [fg] [ab] *} | 〜 [ab ] * ＄ (10) of the identified element type used in the original document
For each, an element representation is required. Each element
Input expression by the column expression comparing means 470.
Compare with the spatial string of the main white area of the document image.
You. In this example, the text element is
The unit between adjacent lines is 3 to 5 points
Another sentence that has a format and is adjacent (top and bottom)
It is separated from the writing element by 16 to 20 points. Te
The quist element is the top, bottom, and entire column
It can be located in the body, or inside the column. Document
Another example of a document element event in a ram
For example, by repeating equation (6),
I can do it. The element expression is used to store training data.
Or generated from sample data. Element representation
Is the actual image of the element type recognized in the source document
Key white space defining document elements derived from
It can be determined from the size range of the area. But the column expression
Is a source document that allows you to carefully check the column structure of the source document.
It is preferably generated by the creator or user of the document. In the preferred embodiment, tagging (solution
Analysis) Tagging process of the document element of the document 500 before
Before you begin, define a structural model
Previously stored in the structural model storage of the
Good. Therefore, the column representation and the
The element representation, together with the structure model,
It will be stored in the model storage. One or more structures stored in memory 130
Using the model, the document element of the document image 500 before analysis is
The statement 470 is identified as in FIG. Statement before parsing
The document 500 is a document element 47 that has not yet been identified.
0, which must be logically identified. analysis
The previous document 500 is stored in the scanner 210 or remote interface.
Input by face 230. Document white area extraction
In the output system 110, the main
Extract the white area. Main white area extracted
To the main white area selecting means 120. Main white
The site selection means 120 selects an appropriate primary white region.
Select and, based on this, the columns of the stored structural model
Representation and unidentified document of pre-analysis document image 500
For comparing with an element. More precisely, choose
Between the sequence of major white areas and the structural model
Perform the switching. As a comparison method, a document white area extraction
Output system 110 and main white area selecting means 120
Main white from document image, used continuously and repeatedly
Repeat the extraction and selection of the area, the document image is first column
To divide. After adjusting the threshold for the main white area,
Each column identified in the input document image is placed in the document white area.
Extraction system 110 and main white area selection means 120
Enter again. The main eaves extracted and selected in the column
The white area is used to find the spatial string for that column.
Used. This spatial string is used for column string selection.
Input to the selection means 160. Another approach is to use a document image
First, a multi-column sentence containing non-rectangular document elements
Input to the device that detects the column from the document image,
The identified column is used in the document element identification system of the present invention.
Input to the column string selection means 160 of the
Is also good. For example, the pre-analysis document image 50 shown in FIG.
0, the main white area selecting means 120
Main white that extends vertically from top to bottom of image 500
The default region 460 is identified. Then the identified vertical
From the primary white area 460, the adjacent vertical primary white
By picking out each pair of regions 460, the solution
Each column 501 of the pre-analysis document image 500 is identified. this
The image column 501 identified as above is placed in the document white area.
After entering the extraction system 110, it is
This is sent to the area selection means 120. Main white area selection means
At 120, as shown in FIG.
A pair of vertical white areas 4 specifying a column 501 of 0
Main white area 4 in the horizontal direction, which exists between 60
Select 60. The main white area selecting means 120 is
In addition, each horizontal major white area selected in column 501
At 460, each based on its vertical height
Assign a character code. Each horizontal of column 501
The character code assigned to the main white area
By concatenation, one spatial string 502 is generated.
The spatial sling 502 corresponds to the column 50 of the image 500 before analysis.
This corresponds to 1 and represents this. Spatial string
502 is also a sequence of primary white areas 460
(Pattern) and define this sequence as a structural model
With the column and element expressions of. FIG.
In the example, the spatial string 502 of the column 501 is "a
gcdcccgbfcdddd ”. The spatial string 502 has a horizontal principal
Only the white area 460 is used,
Geometrical representation of the vertical position of the light
You. Structural models represent a "physical" column structure
Generated for the purpose. Continuous columns
If so, first determine the character string before
Split columns. Or, first, multi-column images
After expressing it as a one-dimensional character,
Compare the ring with the structural model. Japanese Patent Application No. 7-24321
No. 3 is for more complex document images such as two-column document images
And how the camera is based on the horizontal
It discloses whether to determine the lacquer string. Spatial string 5 of document image 501 before analysis
01 is determined by this spatial string 502.
Document elements that are not identified in the pre-analysis document 500
Comparison with the structural model,
Doing Check the structural model for each element type.
The element representation is compared to the spatial string 502 and this
Any match that matches any part of the spatial string
Identify the position of the element type. Examples of each part identified
Is indicated by a bidirectional arrow 503 in FIG. These gather
To form an element type list. Then the document
The element identification system 100 detects the
Match the whole space string 502
The possible column strings 504 are detected. Columns
Tring 504 can be any of the structural model element types
, Resulting in a column of the original document
Will do. Each detected column string 504
Of one or more structural models stored in advance.
Compare with the system expression. One column string 504
Combinations that do not meet the criteria for the above element types
If so, erase that column string. column
When multiple matches of the string 504 are detected
Is the statistical data used in the scoring or selection process.
The best match based on the data. To perform the above processing, a spatial string
502 from the main white area selecting means 120
The output is sent to the string selector 160. Spatial story
502 is in the column 501 of the document image 500 before analysis.
Document element 470 and the document element 470
The relative position with respect to or between the major white areas 460
Is to define. Column string selecting means 160
Is the structural model element that matches the spatial string 502
Select the sequence of the statement type. More precisely,
The column string selection means 160 is defined in the structural model.
Element table for one or more defined element types
Substring of spatial string 502 that matches the current
Extract the bug. The spatial string in FIG. 11 is “agcdc
ccgbjcdddd ". Elements in the original document
Structural model element representation in type "heading"
Is as shown in the following equation (11). Heading = (＾ [ab] * [fg] | [fg] [ab] * [fg]) (11) As described above, the “heading” is determined by the structural model rule. D
Elements are never placed at the bottom of the column
And the element expression of “heading” is the character “＄”
It never ends. In addition, the element type "body"
The structural model element expression is expressed by the following equation (12).
You. Body text = ({[cd] *} | ＾ [cd] * [fh] | [fh] [cd] * [fh] | [fh ] [C-d] * ＄) (12) The “heading” strings “ag” and “gb”
f ”is an element of“ heading ”defined by equation (11).
Matches the expression. These matching "headings"
The tring, along with the corresponding start and end positions,
Stored in string 502. On the other hand, "text" in FIG.
The strings “gcdccccg” and “fcdddd”
"d" is also a "body" element expression defined by equation (12).
Match, so these "body" strings
Spatial string 5 with corresponding start and end positions
02 is stored. The column string selecting means 160
Matches the entire spatial string 502 of the previous document image 500
Column string of structural model element type
Build 504 candidate lists. Candidate column string
Group 504 is a spatial stream matching the element type.
Generated based on the position of the substring extracted from the
I do. In particular, all of the matching column strings 504
Until all possible combinations are detected,
Instead of a string, a map representing the element type
Use the ching character code. One spatial strike
Generate multiple column strings 504 from ring 502
Column string candidate list.
Collect as strike. Column string selecting means 160
Indicates each candidate column in the column string candidate list.
Compare the tring with the column representation of the structural model. Candidate
If the column string 504 does not match the column expression
The column string 504 from the candidate list.
Remove. The remaining column string 504 is
Column string that meets the criteria for
Is interpreted as In the example of FIG. 11, the spatial string 502 is
A single column string 504, "HTHT"
Is converted to a column string. When the column string selecting means 160
Multiple candidate column strings in the system string candidate list
If it is determined that 504 remains, the candidate list is
Output to the ram expression comparing means 170. Column expression comparator
Step 170 is a column of the structural model from among the remaining candidates.
Best match column string that best matches the expression
Select The document element identification system 100
Having statistical data on the size of the white area required
The primary white area size is
(Eg, document elements)
Two document elements located inside the type or
Are located in between)
You. The column expression comparing means 170 converts the statistical data
Determine best match for column string based on
I do. In the structural model, “heading”, “body”,
Suppose that three element types of "Figure" are defined
And, as shown in FIG. 12, a total of nine geometric relationships
Specified. That is, the white two-way arrow in FIG.
Within each element type shown (text, headings, figures)
Relationship between the two element types indicated by the solid arrows
Relationship. For each geometric relationship, the mean (μ) and
Calculate the standard deviation (σ) and use the document element identification system.
Stored in the system 100. The column expression comparing means 170
Each major white area of the complementary column string 504 is
The category of the geometric relationship. Soshi
To compare the size of the key white area with the statistical sample.
The main white region is
Detect categories and probabilities corresponding to them. Assume that l possible matches remain.
Then, each matching mj (j = 1,..., L) becomes n
Contains white space. Assuming a normal distribution,
The white space wi (i = 1,..., N) is a remaining candidate
The probability p corresponding to a column string is determined by statistical theory.
Is calculated. This calculation is performed as shown in the following equation (13).
Is a probability function of a normal distribution. [Equation 1] Here, x1 is the size of w1, μc and
σc is the white space category to which each Wi belongs
Mean and standard deviation. The score of each possible match is
It is calculated as the average of the probability values in the white space. Possible
The score S of the matching mj is expressed by the following equation (14).
You. [Mathematical formula-see original document] The best matching candidate column string
504 has the highest score S, and thus has the highest score.
A candidate column string 504 finally matches
Selected as the column string 504. Then sentence
The book element identification system 100 converts a column image into a document.
Divide into elements and attach corresponding logical tags to each
You. If tagging has been performed, the
Document element 470 is mapped to a matching column expression
Stored in the memory together with the logical tag of
Output to the processor 140. The following processing example
A specific document element 470 based on the logical tag.
Optical character recognition processing. Next, the stored structural model (each identified element)
Includes column and element expressions for element types
Of the document element 470 using the
Preferred embodiments will be described based on a flowchart.
You. FIG. 13 shows a document image of a column-format document image.
Float showing logical tagging of elements
It is a chart. Operation is started in step S100.
First, in step S200, the column expression of the original document and the necessary
The structural model including the element representation
It is stored in the memory 130 of the other system 100. In step S300, the main white of the document image
Extract the site area. The main image of the document image extracted here
Identify document image columns based on white space
And then the key white region sequence in each column
Is detected. If the main white area of the document image is extracted
For example, in step S400, the spatial stream of the document image column
Detecting the ringing. For each extracted column, after S400
Repeat the process. In step S500, the main white
Match with the spatial string generated by the region selection means 120
To find a set of possible column strings. Stay
In step S600, the candidate list of the detected column string is displayed.
Of the column expressions defined in the structural model
Select a column string that matches well. In step S700, the best match color
Based on the primary white area of the
The document image element of the system. Step S800
To select the split document element and select the best match
Label with a logical tag corresponding to the column string of
You. In step S900, a document element is extracted,
For example, it outputs to the printer 300. Step S1000
To end the operation. FIG. 14 shows the operation flow of FIG.
Related to extraction of document image main white area 460
Step S300 will be described in detail.
After the start of step S300, in step S310, the document
Enter an image. The input document image 400 includes a plurality of document
Element 470. In step S320, the document image
Identify connected components 410 within 400. Step S33
0, for each of the identified connected components 410,
Box 420 is generated. In step S340, the main e
The white area 460 is extracted. Main white areas extracted
Each column is detected in step S305 based on the area.
You. If all columns are detected in step S305
For example, in step S345, a column area
Extract all horizontal white regions extending from left to right of
You. The extracted horizontal white area is used to generate a spatial string
Used for In step S350, the operation is
It returns to step S400. FIG. 15 shows the extraction of the main white area in FIG.
It is a flowchart which shows step S340 in detail.
After the start of step S340, in step S342, the primary
The white area 430 is extracted. As shown in FIG.
The primary white area 430 is a rectangle between the bounding boxes 420
The white space region of the shape. Next, step S343
The width and height of each primary white area 430 in the horizontal direction
Is compared with the threshold width 440 and the threshold height 450,
Also, the width and height of each primary white area 430 in the vertical direction
With the threshold width 440 'and the threshold height 450'
Compare. The threshold width 440 of the horizontal white area is a sentence
1/3 of the horizontal length (ie, width) of the written image 400
It is preferable to set Horizontal white area threshold
Height 450 is greater than the line spacing of the text in the document image
It is preferable to set small. Meanwhile, the vertical white area
The threshold height 450 'of the document image 400 is
Preferably set to 1/3 of the length (ie height) of
In addition, the threshold width 440 'is set to
It is preferable to set the value to a value larger than the interval. Comparison results
Result, threshold height (450, 450 ') and threshold
Primary horizontal white that exceeds the width (440, 440 ')
The primary white area and the primary vertical white area
Identified as a region. In step S344, the horizontal threshold width 44
The horizontal primary white area 430 having a width smaller than 0 and the vertical
Vertical primary white area lower than threshold height 450 '
The area 430 is deleted. In step S345, the remaining primary
Merge white area 430 into group
The default region 460 is identified. In step S346, these
Of the major white areas of the horizontal and vertical
At least one is greater than the corresponding vertical and horizontal thresholds
Erase small ones. Or horizontal and vertical
Both are less than the corresponding vertical and horizontal thresholds
You may delete only the thing. In step S345
Grouping of the remaining primary white areas 430
Various methods for detecting the main white area 460
For example, the aforementioned Japanese Patent Application No. 7-243213
In the manner disclosed in US Pat. In step S347,
The operation returns to step S350. FIG. 16 shows the document image in step S400.
Preferred embodiment of the column spatial string determination process
It is a flowchart of a state. Operation in step S400
If the solution starts, in step S410, extraction
Detect the geometric relationship of the selected main white area. Stay
In step S420, the extracted main white area
That is, the main white area into which the document element 470 is divided
Select In step S430, all selected masters
For white areas that require character,
Assign a code. In step S440, the entire sentence
Main white area 460 included in the letter image column 501
Is determined. Step S450
Then, the operation returns to step S500. In the example shown in FIG. 11, the document
The horizontal major white area 460 is divided into the divisions of the element 470.
Only use. Generalize this process to
It can also be applied to column documents. Multiple columns
The method of detecting spatial strings in documents described in
No. 7,243,213. FIG. 17 shows the column list in step S500.
Float showing a preferred embodiment of the ring selection process
It is a chart. After starting in step S500,
In step S505, a structure model element type is selected.
In step S510, the element type selected in the previous process
All element representations of the
Compare. In step S515, the spatial string 502
If there is an element expression that matches any part of
To determine Element type category or
Element representation matches part of a spatial string
If so, the process proceeds to step S520. Empty in step S520
Each element expression that matches a part of the inter-string
For example, it is stored in the memory 130. Matched element
With at least the corresponding
The start part and the end part are also stored in the memory 130. Step
In step S520, the element type of each category and the related
After storing the consecutive data, the process proceeds to step S525. S
In step S515, a part of the spatial string 502 is mapped.
If there is no category for the element type
The operation jumps directly to step S525. At step S520, the structural model is checked.
The structural model element to be compared with the spatial string 502
It is determined whether or not there is any remaining comment type. ratio
If there are more element types to compare
Proceed to step S525, select the next element type, and
The operation returns to step S510. Defined by structural model
All element types defined as spatial string 5
If the operation is determined to be compared to 02,
The process proceeds from step S525 to S530. In step S530, identification in the structural model
Empty in the element representation of each element type
All element tables that matched the inter-string 502
The current is called from the memory 130. In step S535,
Various element tables that match part of the spatial string
Combine the current to match the entire spatial string
A column string 504 is generated. Step S540
The column string 504 is defined in the structural model
Check for column expression, column string 5
04 was the standard of the original document corresponding to the structural model
Determine if the column layout. The column list identified in step S540
If the ring 504 has a column layout that meets the criteria,
If it is determined, the process proceeds to step S545. Step S
At 545, this column string is
Then, the process proceeds to step S555. On the other hand,
The column string identified in the previous step in step S540
Is determined to be a column layout that does not meet the criteria
In this case, this column string is deleted in step S550.
And the process proceeds to step S555. In step S555,
The column string selecting means 160 outputs the matched element.
Additional column list from the
Determine if the rings can be collected. Further additional
If there is a column string, return to step S535.
Identifies the next column string. Step S55
If you identified all possible column strings in 5,
If not, the process proceeds to step S560, and the operation is
The process returns to step S600. FIG. 18 shows the best map of step S600.
Preferred Embodiment of the Tching Column String Selection Process
It is a flowchart which shows. Operation in step S600
After starting the configuration, in step S605
Check the string list to see if one or more
Lactor string 504 is a column string candidate list
To determine if it is. If the candidate column string
If there is only one, go to step S610, where
This column string is used as the best match for the column expression.
And select. Then, after step S615,
The process returns to step S700. In step S605, the column string
Determines that complement list contains one or more column strings
If so, the process proceeds to step S620. Step 62
0 means that the statistical data of the category
Search for data. In step S625, the first column
Identify the string candidate as the current column string
You. In step S630, the current column string
The score of the main white region of 1 is calculated according to equation (13).
decide. Equation (13) is the main white of the column string.
Expressions for scoring each of the site areas. Stay
In step S635, the current column string is checked.
There are still key white areas to score
Is determined. Major whites to score
If there is a site left, the process returns to step S630.
You. All major white regions in the current column string
After scoring the area, the process proceeds from step S635.
Proceed to step S640. In step S640, the expression (1)
Calculate the total score of the current column string based on 4)
For example, it is stored in the memory 130, for example, in step S645.
Check the column string candidate list and
Check if there are more column strings to attach
Bell. Column strings to be scored remain
If not, the process proceeds to step S650. Step S650
To replace the next candidate column string with the current column string.
And returns to step S630. Steps
In S645, all candidate column strings are scored.
If it is determined that the access has been performed, the process proceeds to step S655.
In step S655, the column string with the highest score is
Select as the column string that best matches the column expression
Select. In step S660, the process returns to step S700
I do. FIG. 19 shows a document element 47 in a column.
0, the primary white area matching unknown input documents
Demonstrates how to tag logically based on
FIG. First, step S1100
After starting, in step S1200, a plurality of types of well-known
The structural model of the original document is transferred to the document element identification system.
Remember. The structural model should have at least a cover page,
Ram representations and elements defined in the structural model
Split categories of or internal geometric relationships between types
Statistical data on the expected size of major white areas
Data. FIG. 12 shows three element types "
, "Text", and "figure"
Indicate categories of geometric relationships that require statistical data
You. At step S1300, the cover page and
Unknown containing document elements that should be logically tagged
Input document to the document element identification system.
You. Input document image divides input document into columns
It has a major white area. Extract this main white area
And store the columns of the document image for further analysis.
You. In step S1400, the unknown unknown input
Search for the cover page of the input document and store it in advance.
Compared to multiple types of original document cover pages,
Identify the cover page to load. Matched hippo
Page identifies the unknown input document type and identifies
Search the storage device for the corresponding structural model of the input document type
I do. The process of identifying cover pages for multiple documents
Is described in Japanese Patent Application No. 7-243212. Also
Means that the document element identification system
Manually identify input document types
An instruction may be given. In this case, the user
After the unknown input sentence type is identified, the corresponding structure
Search for a model. At step S1500, the document element identification
Another system uses the corresponding structural model to
Split document elements in the image split column and logically
Perform tagging. At step S1600, the input document image
Extract logically tagged document elements, for example
The data is output to the printer 200 or a storage device. At step S1700, the document element identification
Another system provides a logically tagged input document image sentence.
Search statistical data of geometric relationships between calligraphic elements,
Between the document elements in the identified corresponding structural model
Or update statistics of internal geometric relationships. Change
Corresponding structural model including new geometric relation statistical data
Is stored again. In step S1800, the process
finish. The preferred embodiment described above is to the last
It is merely exemplary, and those skilled in the art will appreciate that various
There are various data processing configurations.
Obviously, this is not a limitation. According to the present invention, a simple structure representing a document structure can be obtained.
Based on white area detection using simple structural model representation
Thus, each element in the document can be identified. This
Is used to analyze each detail in the document image
There is no need to detect the connection state. Actual document structure
Based on the model, apply to any document identification.
Can be.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a preferred embodiment of the document element identification system of the present invention. FIG. 2 shows the document element identification system of FIG.
FIG. 2 is a block diagram illustrating a preferred embodiment of a document white region extraction system. FIG. 3 is a block diagram showing a preferred embodiment of a main white area extracting means in the document white area extracting system of FIG. 2; FIG. 4 is a diagram illustrating an example of a sample document image. FIG. 5 is a diagram illustrating a horizontal primary white area extracted from a document image according to the present invention. FIG. 6 shows a vertical primary white area extracted from a document image according to the present invention. FIG. 7 is a diagram of a document image having a main white region extracted according to the present invention. FIG. 8 is a diagram of a sample document showing an example of a column layout of a predetermined column expression. FIG. 9 is a sample diagram showing an arrangement of document elements, and is a diagram showing an example including three text elements. FIG. 10 is a diagram showing an example of an input document image column before analysis. FIG. 11 is a diagram illustrating a method of comparing a spatial string of an input document image with an element expression of a structural model to identify a matching column string. FIG. 12 is a diagram showing a geometric relationship of a main white area in a structural model having three types of document elements of “heading”, “figure”, and “body”. FIG. 13 is a flowchart illustrating a good process for logically tagging document elements in a columnar document image by matching a primary white area pattern. FIG. 14 is a flowchart illustrating a preferred process for extracting primary white regions and document elements. FIG. 15 is a flowchart showing a detailed process of extracting a main white area from the flowchart of FIG. 14; FIG. 16 is a flowchart illustrating a process for determining a spatial string of columns of an input document image. FIG. 17 is a flowchart illustrating a process for detecting a candidate list of possible column strings that match a spatial string. FIG. 18 is a flowchart showing a process of selecting a column string that best matches a column expression from a column string candidate list. FIG. 19 is a flowchart illustrating a process of logically tagging document elements in a corresponding document after identifying the type of the original document according to the second embodiment of the present invention. DESCRIPTION OF SYMBOLS 100 Document element identification system 110 Document white area extraction system 120 Main white area selection means 130 Memory 140 Processor 150 Structural model definition means 160 Column string comparison means 170 Column expression comparison means 180 Logical tag assignment means 190 Document element extraction Means 400 Document image 405 Column 406 Document element 460 Primary white space 470 Document element

Continuation of the front page (56) References JP-A-5-159101 (JP, A) JP-A-6-60219 (JP, A) JP-A-8-185474 (JP, A) JP-A-8-185476 (JP) , A) A knowledge base construction method for document image structure analysis, Transactions of Information Processing Society of Japan, Japan, January 1993, Vol. 34 No. 1, pp. 75-87 Document Image Region Identification Based on Rectangular Layout Model, IEICE Technical Report, Japan, November 12, 1993, Vol. 93 No. 319, pp. Understanding of Document Image Layout Based on 45-52 Model, IEICE Transactions, Japan, October 1992, Vol. J75-DII No. 10, pp. 1673-1681 (58) Field surveyed (Int.Cl. ⁷ , DB name) G06K 9/00-9/82 G06F 17/21

Claims (1)

(57) A method for logically identifying a document element in a document image, wherein at least one document element has at least one column including at least one document element. Generating one structural model, wherein each structural model has at least one column representation and at least one
Defining a relationship between at least one document element of the original document and identifying a first main background area in the document image, wherein the first main background area identifies the document image by at least one Dividing into two columns and identifying a second main background area in each of the at least one column, forming a main background area pattern for each column of the document image from the identified second main background area Generating at least one column string that matches a main background pattern for each column of the document image; and determining a best column string from the at least one column string that matches the main background pattern. With at least one best column string Logically identifying document elements in the document image, comprising logically identifying each of the at least one document element in the document image based on a best column string that best matches a column representation of the structured model. Method.