JPH01130293A - Document image analyzing system - Google Patents

Abstract

PURPOSE:To stably and easily extract a prescribed area by generating hierarchically and automatically an element for constituting various document images and an arrangement structure between the elements, in accordance with an inclusive relation of a block and the upper and the lower or the right and the left relative position relation. CONSTITUTION:A document image which has been stored in a document image memory 1 is divided into element areas of a character line, a character, etc., by an area dividing part 2. At the time of structuring one or plural pieces of element areas as a block, a document structure generating part 20 determines hierarchically an attribute of the block and an arrangement structure between the blocks in accordance with an inclusive relation of each block and the upper and the lower or the right and the left arrangement relation, and stores them in a structured data storage part 4. An area searching part 13 searches an area to be extracted in the document image or one or plural pieces of blocks for constituting the area to be extracted, from the attribute of the block and the hierarchical arrangement structure between the blocks, and stores them in an extraction result storage part 14.

Description

Detailed Description of the Invention (Industrial Field of Application) The present invention relates to a document image analysis method that divides a document image into its constituent elements such as character lines and characters, and automatically extracts a desired area. The present invention relates to a document image analysis method suitable for decomposing a document image into elements such as character lines and characters, and automatically extracting a desired area including the flow of text.・ 1 (Prior art and its problems) In order to efficiently store, search, and transmit images of a large amount of existing documents and images, and to automatically read general books, it is necessary to create forms with fixed formats. It not only reads characters from a specific predetermined character image string, but also analyzes a wide variety of document images and separates character areas and diagram areas.
Furthermore, it is necessary to automatically extract a desired area.

Conventionally, as a structural analysis method for such document images, "Structural analysis of page images based on the split detection method" by the same inventor as the present inventor, J. Communication Society Technical Research Report Pattern Recognition and Learning PRL85-17. As described in a technical paper titled ゜pages 63 to 70), there is a method to determine the constituent elements of a document after decomposing the global area into local areas while alternately extracting vertical or horizontal projection information. Here, the above method was performed with the premise of reading characters in the main text of books, etc. Therefore, when assuming the existence of columns found in general documents and the reading of characters in specific document areas. Since the layout relationships that make up the document image are not available, it is necessary to find the elements of the document image and the layout relationships between the elements.

On the other hand, as a method for extracting a predetermined region of a document image, for example, as shown in Japanese Patent Application No. 41-281377 "Image Understanding Method", a document image f is defined as a set of a plurality of rectangular regions. A method is known in which the area to be extracted is determined according to a given grammar. However, the position of the rectangular area
Defining all sizes based on absolute or relative coordinates requires effort, and when reading text in document images such as books, defining rectangular areas using coordinates is difficult due to changes in the number of lines or graphics. It becomes difficult in practice due to the mixture of etc.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to solve the above-mentioned conventional problems by establishing an inclusive relationship and a single system between upper and lower or left and right arrangements as relationships between one or more element areas (hereinafter referred to as blocks). Accordingly, it is an object of the present invention to provide a document image analysis method that automatically generates the structure of a document image as a hierarchical structure and extracts a predetermined area based on relative inposition and positional relationships.

Another object of the present invention is to provide a document image analysis method that can easily detect the order in which text areas should be read and the structure of a document by automatically generating relationships between blocks of various document images.

Another object of the present invention is to provide a document image analysis method that can easily extract a specific area determined depending on the purpose.

Another object of the present invention is to exist at a predetermined position within a character line;
An object of the present invention is to provide a document image analysis method that extracts the shapes of character lines, including blank blocks, and generates a logical arrangement structure of text areas.

(Means for Solving the Problems) In order to solve the above-mentioned problems, the document image analysis method provided by the first invention of the present application converts document images into sentences! 1. When structuring a plurality of element areas as blocks, block attributes and interblocks are determined according to the inclusion relationship and vertical or horizontal placement relationship of each block. Sentence 8 structure generation means that hierarchically determines and stores the layout structure of the document image, and the region to be extracted in the document image or one or more of the regions constituting the region from the block attributes and the hierarchical layout structure between the blocks. and area search means for searching the blocks.

In addition, in order to solve the above-mentioned problems, the document image analysis method provided by the second invention of the present application includes means for decomposing a document image into element areas such as character lines and characters, and one or more elements. When structuring an area as blocks, a document generation means that hierarchically determines and stores the attributes of blocks and the layout structure between blocks according to the inclusion relationship of each block and the vertical or horizontal layout relationship; A means for extracting a blank space with a predetermined position and size within a block as a blank block, and a shape of each character line block based on the blank block from a block containing multiple character lines generated by a document structure generation means. a document structure updating means for checking and updating the arrangement structure in the document image; and a region to be extracted in the document image from the hierarchical arrangement structure, or a region forming the same.
and area search means for searching one or more blocks.

(Example) Examples of the present invention will be described below with reference to the drawings.

FIG. 1 and FIG. 2 are examples used to explain the structure of document images written in I-letter and horizontal writing, respectively.

In the figure, black circles indicate characters, and the diagonally shaded rectangular areas (F in the figure) are elements of figures, tables, photographs, and the like.

When conventional document image area division methods or line extraction methods are used, symbols St in FIGS. 1 and 2 (i=1...7 in FIG. 1; i=1... in FIG. 2) are used. 15) or the figure/table/character area indicated by symbol F1 in Figure 1.
Areas such as photographs (hereinafter referred to as pixel description areas) can be extracted.

Next, focusing on the flow of text information, in the case of vertical writing as shown in Figure 1, text information flows from right to left in a vertical text line, and each character within a text line flows from top to bottom. Text information flows. That is, the arrangement of each character line in FIG. 1 has a left-right relationship, and the arrangement of each character within each character line has an up-and-down relationship. Furthermore, the pixel description area F+ includes character lines Ss, S6 .
There is an arrangement relationship such as being above St. Therefore, when the block indicated by Ti (i=1.2) in Fig. 1, that is, the text area, is detected, two text areas Tr and T2 consisting of character lines with left-right relationship are located in a left-right relationship. Furthermore, the vertical relationship between the pixel description area lma and the sentence/chapter area T2 can be easily changed. Therefore, by extracting the above-mentioned inclusion relationship and vertical or horizontal positioning relationship, for example, characters in a character line are extracted sequentially from text area T1 to T2 and converted to character codes, or a pixel description area It becomes possible to easily extract only the text area under F1.

Similarly, in the case of horizontal writing as shown in Figure 2, information normally flows from left to right for each character in a horizontal character line, and the text area consisting of each horizontal character line is from top to bottom. For example, each character line S+(i=1
...15), character line S1 to character line S5, character line S6 to character line S10, and character line S++ to character line S+s are respectively text regions tJAT+ and 'rs from the character line having a vertical relationship.

It is formed from T6. Further, as a structure similar to a two-column set, there are text regions T s and T a, and it can be seen that the text region T4 is formed by the text regions T5 and T6, which have a left-right relationship in terms of information flow.

Furthermore, as shown in FIG. 1, if there are character lines with different character pitches within the text area T1, a text area T composed of character lines that maintain a vertical relationship is further defined as a text area. Even if it is decomposed into T3, a vertical relationship will be established for each.

As explained above, when expressing the arrangement relationship of the constituent elements of a text image and the flow of text information, the relationship between each element is defined as the vertical relationship and the horizontal relationship (in the case of I writing, the right-left relationship; in the case of horizontal writing, the left-right relationship). ) can be realized by detecting and generating hierarchically.

FIGS. 3(a), 3(b), and 3(c) are examples of a method for performing area division hierarchically while alternately defining vertical and horizontal relationships between blocks.

A method for realizing the above region division is described in "Structure Analysis of Page Images Based on Split Detection Method" by the same applicant. Therefore, in this explanation, details will be omitted and the third
Figures (a), (bl, and (c)) show that the above-mentioned area division is a method of hierarchically decomposing each block into constituent elements of the paper while maintaining the vertical or horizontal arrangement relationship.

In addition, in the above method, the black prime value (binary value of black or white) is horizontally
Distribution that is projected vertically and shows the pixel count value (projected distribution)
However, the present invention is not limited to this, and the present invention is not limited to this, but uses area information that overlaps by horizontally and vertically projecting the counted values of black and white changing points and rectangular information determined by circular tracing in advance. Also good.

In Figure 3(a), area P of the document image to be divided into areas includes characters indicated by black circles and a pixel description area based on a rectangle with diagonal lines. It has a structure similar to a vertical document image.

Symbols Rt (L, ,) (L, , =
1.2,3. ..., i=1.2,3. ...) indicates a block obtained through the hierarchical region segmentation process using the projection distribution (the shaded figure in the figure), and the symbol above indicates
, indicates the division level. Furthermore, the division level L, . . . represents not only the hierarchical depth but also the direction in which projection information is obtained. That is, the division level of multiple regions divided according to horizontal projection information, , has an odd value, and the division level of multiple regions divided according to vertical projection information, , has an even value. I will have it. Furthermore, it is clear that the vertical relationship (horizontal relationship) of the plurality of areas divided by the horizontal (vertical) projection distribution is preserved.

First, the horizontal projection distribution H1 is applied to the analysis target region P, and a region R1(1) is obtained.

Next, the vertical projection distribution is applied to block R, (1) to obtain blocks Rt (2), . . . R1 (2). Here, it is clear that the five regions having division level 2 satisfy the left-right relationship in sequence. Also,
Block R, (1) and one or more blocks R1
(2)...R5 It is also clear that (2) does not satisfy the inclusion relationship. In addition, how to divide the five blocks is to divide the features of each block and the feature amount between each block (blank value and correlation ratio) with the features of its parent block R1 (1) (for example, identifier). Blocks R, (2)..., R5(
It is determined by dividing the cases according to the characteristics of 2) and examining them. In the case of Fig. 3(a), the block is divided into five blocks, and block R, (2) has an undetermined identifier, and R2 (2) to R9 (2) have an identifier of character line candidate added. , will be remembered. Area division is Af
Since the &-type search tree search technique is used, block R, (2) is extracted as the next block to be divided, and the same process is repeated. When horizontal projection distribution is applied to block Rs(2), multiple character candidate areas are obtained, so the division of block R51) is stopped at this point and the multiple character candidate areas are also stored. Ru. Similarly, blocks Ra(2), Ri(2), and R2(2) are sequentially applied with the horizontal projection distribution, and the block R+(2) is processed in the same way as the block Rs<2). ), as shown in Figure 3(b), the horizontal projection distribution H
2 is applied, and the two blocks R, (3), Rt (3)
is obtained, and the same processing as described above is performed.

In the case of FIG. 3(b), blocks R+ (3) and R
An identifier "undefined" is added to t(3) and it is stored. At this time, block R1 (3) and block R2 (3) have a vertical relationship, and their parent block is R1 (2). As shown, vertical projection distribution V4 is applied, and three character line candidate blocks R1<4>, R2(4), and R3(4) are obtained.

Also, for block R1(3), area R2(3)
If a vertical projection distribution is applied in the same manner as above, since there is only one block, it cannot be divided any further, and an identifier for a pixel expression block such as a diagram, table, or photograph is given based on the block size.

When the operations described above are repeated and the vertical search is completed, the wooden structure of area information shown in FIG. 4 is generated.

Note that the region segmentation results shown in FIG. 4 were generated corresponding to the document image shown in FIG.
Since FIG. 4A has a structure similar to FIG. 1, the following description of FIG. 4 will be made in accordance with FIG. 1.

In FIG. 4, area information is represented by circles in the figure.1.
The symbol S indicates a character line, the symbol F indicates a pixel expression area (however, the subscript i was added to correspond with FIG. 1), and the black circle indicates character area information. Also,
It is assumed that a circle to which no symbol is attached represents a virtual block.

Further, in the figure, the numbers represent division levels; when the division level is an odd number, upper and lower relations are established between each area, and when the division level is an even number, a left-right relationship is established between each block. Here, in the hierarchical representation according to this structure in FIG. 4, division level i and division level i+1 (i=1.2...)
An inclusive relationship is established for the relationship, and each area within the same division level has a left-right relationship (indicated by 8 in the figure) or a vertical relationship (indicated by 8 in the figure).

Next, an example of the attribute values of the block information described in FIG. 4 will be explained using FIG. 5. The classification names shown in FIG. 5 are analysis target, character, line segment, character line, pixel representation block, and virtual block.

In addition, among virtual blocks, nodes,
There will be virtual blocks to which classification names such as sentences are added.

The position and size represent the position size of the block, the distance between blocks represents the blank size between adjacent blocks at the same division level, and the number of child blocks is the number of its own child blocks. Regarding character blocks, for example, "Adaptive Character Separation Method Based on Minimum Dispersion Criteria" by the same inventor as the present inventor (IEICE Transactions D' 85/8V
OL-J68-D, NO, 8, Hage 1497-15
Suppose that it is obtained using the method shown in 04). Furthermore, when separating each character in a character line block, if a blank character exists, it is treated as a character whose child block number is 0.

The inter-child block placement attribute has a vertical relationship or horizontal relationship (right-left relationship in vertical writing, time-to-right relationship in horizontal writing) that represents the placement relationship of multiple child blocks behind itself. Child block taxonomic name group is 1 included in that block.
A plurality of connected classification names (one line, two characters, one sentence) and their number are stored. Note that the classification name 1 inter-child block arrangement attribute, child block classification name group, etc. are updated and set in structuring, which will be described later. It is also assumed that in the child block classification name group, the classification names of character lines and sentences are stored separately for vertical writing and horizontal writing.

FIG. 6 shows an example of automatically generating a document structure from the region segmentation results shown in FIG. 4. The generation of a description of the document structure shown in Figure 6 is an example of using the classification names between blocks shown in Figure 5 as structuring conditions. (indicated by T in the figure) will be generated.

In the case of FIG. 4, the structuring condition is vertical writing.

Vertical/horizontal writing information may be given in advance, or may be determined automatically using conventional technology.Also, when multiple blocks are structured and there is only one parent, 1
．． There is no need to generate new professionals.

First, as a search for area information, a block with the largest division level (character line, 0 with a classification name of 1 pixel representation of a character that does not form a line 1 character line), for example, block R1 (4) in the diagram of FIG. R2 (4).

Rs' (4)) and their parent blocks (R in Figure 4)
2 (3) only) is extracted. Next, among the structuring conditions, vertical writing/horizontal writing information and division level L 11 I
When I is checked at the 0 division level L and 9 is an odd number, connect multiple blocks having the same parent block in an order that establishes the vertical relationship using the vertical relationship pointers shown in FIG. sorted by. On the other hand, the division level L I
When I II is an even number, from the perspective of the flow of the text, a right-left relationship (indicated by φ in FIG. 6) is established in horizontal writing, and a left-right relationship is established in vertical writing. Therefore, the above-mentioned
A plurality of blocks having a parent block R2(3) (R+(4), R2(4>, R3(4) in FIG. 4),
They are arranged in the order in which the left-right relationship is established by connecting them using the left-right relationship pointers shown in FIG.
In the figure R3(4)YR2(4)YR,(4)).

Next, a new block is generated or the attributes of the parent block are added. In the example of FIG. 4, three blocks are Rt (
4), R2(4), and Rs(4) all have the classification name of character line (S+, i=5.6°7 in the figure), so their parent block R2(3) can be used as a text as is. A classification name ('r2 in the figure) is added. Here, the division level L, 1. Block information such as the number of child blocks and placement attributes between child blocks is set and updated. Here, the fourth
In the case shown in the figure, structuring at division level 4 is completed.

Next, blocks of division level 3 (blocks R, (3), R2 (3) in Figure 6) and their parent block R + (2)
is taken out. Note that in the area of division level 3, there are character areas indicated by black circles in FIG. 6, but the classification names of their parent blocks are character lines, so they are not extracted.

Similarly, the above-mentioned structuring conditions are checked for the two blocks R+ (3) and R2 (3) of division level 3.

In this case, blocks R1(3) (with a pixel expression already added as a classification name) and R2(3) (text as a classification name) are given a vertical relationship pointer, and a virtual block (see Figure 6) is added as a classification name. 0 to which the symbol M,) in the figure is added
Next, the text (symbol T) and one-pixel representation (symbol F) are updated and set in the child block classification name group, and block information such as the vertical relationship is updated and set as the child block arrangement attribute.

Here, the structuring of division level 3 is completed.

Next, the block of division level 2 (R1 (2) in Fig. 6,
R2(2), R3(2), R4(2).

R, <2>) and its parent block R+(1) are extracted. Similarly, before the structuring conditions described above are checked, Rs
(2) to R1 (2) to which left-right relation pointers are attached in the order 0, followed by their classification names (in this case, the four character lines St, S2, Ss).

The virtual blocks M+> that include S, text, and pixel expression blocks in a vertical relationship are sequentially examined.

In this case, the four character lines can be structured as a sentence, and furthermore, their parent block R, (1) includes a virtual block M, in addition to the four character lines, so as a new area, as shown in FIG. A block is generated as a classification name text (T1) indicated by a rectangle in the figure. Next, the left-right relationship pointers of the character line block S and the virtual block M1 are separated, that is, the left-right relationship pointer of the character line block S is set to NULL.

), the address indicating the virtual block M1 is entered into the left-right relationship pointer of the text block T+.

Furthermore, four character line blocks St, 32, S,.

The address indicating the newly generated text block T is stored in the new area pointer shown in FIG. 5 in S4, and the child area pointer shown in FIG. Text block S+ as the first child area
An address indicating the address is stored. Next, the text block TI
The division level, which is an attribute of , the newly generated text block T1
Blocks having a left-right relationship (in this case, the text block T1 and the virtual block M+) are sequentially extracted from the blocks, and the above-mentioned structuring conditions are examined in the same manner. In this case, block R1 (
1) is given a classification name representing a virtual block (M2 in the figure). The child block classification name group shown in FIG. 5 of the virtual block M2 includes the virtual block Ml (in the case of a virtual block, the classification name included in the virtual block; in the case of FIG. 6, Pixel representation F1 and text T,
) and the sentence block T+, that is, the two sentences T and the graphic representation F) are stored.

Thereafter, by performing similar operations, the document structure shown in FIG. 6 is generated, and each attribute value shown in FIG. 5 is determined for each block.

Therefore, the automatic generation result of the document structure shown in Figure 6 (first
(corresponding to the document image in the figure) will be used to explain the area extraction method described in Section 1 of the present invention. The area extraction means described in the first section uses a tree 'ui as shown in FIG.
By searching the structure, one or more desired blocks are extracted in a predetermined order. As specific examples, the following two cases will be explained. First, as shown in FIG. 1, a case will be considered in which a text area indicated by a text block T1 and a text block T2 in a book or the like is sequentially read as a text.

In the above case, the area extracting means first performs a vertical search using a pointer to sequentially search the classification name and group of classification names within the block as attributes of each block shown in FIG. Character line block S, 32.33
, S4. S,,S,.

S7 can be easily extracted in the order in which the text should be read.

Here, since the taxonomic name group within the block is written in each block, if the desired character line block is not included in each block, the vertical search for this block can be stopped, which is efficient. can get. It goes without saying that each character image of the desired character line block (S, , S2, . . . S7) or character recognition using the conventional technique can be easily obtained from the results obtained by the area extracting means.

Next, as shown in FIG. 1, consider the case where only the area indicated by the three text blocks T2 below the pixel expression block Fl is read.

In this case, the area extraction means performs a vertical search for each block expressed in the tree structure shown in FIG. 6, and examines the attributes of each block when reading the text area of the document image as described above. The only difference is that the key pixel expression block F1 is searched first.

That is, when a vertical search is performed on the main structure shown in FIG. 6, the text block T is found first. Text block T1
Since there is no pixel expression block in , the search behind the text block TI is stopped and the virtual block M1 is examined next. Since a pixel expression block exists in the intra-block classification name group of the virtual block M1, a pixel expression block F+ is detected by further searching. In this way, character line blocks Ss, S6 . It is easy to extract St and read the characters. For example, it is also possible to easily extract only the adjacent character line block S4 on the right side of the pixel representation F.

In FIG. 1, there is only one pixel expression block F+, but if there are multiple, when searching for a key block group, give the approximate position and size of the area, It goes without saying that key blocks can be found by checking the position and size of each block as an attribute value.

FIG. 7 is an example showing the result of region division on the horizontally written document image shown in FIG.

As shown in FIG. 4, the area division results shown in FIG. 7 are obtained by using the conventional technique described in "Structure Analysis of Page Images Based on Split Detection Method" by the same applicant mentioned above. realizable. Further, in the figure, character areas are omitted.

FIG. 8 is an example of generating a document structure description based on the area division result shown in FIG. The automatic generation of the document structure shown in FIG. 8 is an example in which the classification names between blocks, the distance between blocks, and the estimated character pitch value ' shown in FIG. 5 are used as structural conditions. The case of FIG. 8 can also be realized by the same processing as the case of FIG. 6, and in the case of FIG. 8, the text block T1 shown in FIG. ．． The difference is that it can be decomposed into T3. Also, the 8th
The virtual block M shown in the figure stores information indicating a classification name indicating two fJ-written sentences as an intra-block classification name group and a left-right relationship as an inter-block arrangement attribute. In the case of FIG. 8, the left-right arrangement relationship of the character line blocks, text blocks, etc. is obtained from the relationship from left to right, since they are written horizontally.

Here, it is shown that a desired area can be extracted by searching for the basic structure shown in FIG. 8. For example, each character in text blocks T and T6 shown in FIG. The case of converting to a character code string using character recognition will be described. First, a two-column block (virtual block M"1 in the figure) consisting of text areas T s and T b is searched. For example, blocks X and y (blocks X and y
When searching for a block (Xφy) whose child block arrangement attributes (with a vertical relationship) have a left-right relationship Xφy, virtual block M1 in FIG. 8 is detected as the block (Xφy).

Next, a vertical search is performed sequentially from the virtual block M1, character line blocks S6...srs are extracted, and each character block is sequentially extracted in each character line block S6...315 to perform character recognition. .

In addition, when searching for the above-mentioned two-column structure, only the child block placement attributes of block X and block Y were used as conditions, but the position, size, etc. shown in Figure 5 were also used as attributes. If possible, you may use it.

In addition, two character lines St in the text block T1 in FIG.
, Ss can also be searched for block 2 that satisfies -2&(x<)y) (where x, y are the conditions described above, and 2 is a text block), the text block in Figure 8 is searched. 2 is detected.

Now, the character line blocks S-, Ss, 2 as shown in FIG.
Since we are looking for two character line blocks above the column, for example, z18z2 (where block Zl
, Z2 is a character line included in block Z, and z2 is the lowest block).
All you have to do is find z2.

As mentioned in the above explanation, when searching this structure as shown in FIG. 8 and extracting a desired block, the area to be extracted is determined in advance in language according to the attributes and relative arrangement of the block. The tree structure may be searched according to this definition, or the processing procedure may be programmed as described above and the tree structure searched according to this.

As described above, the area to be extracted by the document image analysis method of the first invention of the present application can be found by searching the block group of the input document image expressed by the present structure shown in FIG. . Although the search method has been described as a vertical search, a horizontal search may also be used.

FIG. 9, FIG. 10, and FIG. 11 are diagrams for explaining the document image analysis method of the first invention of the present application. FIG. 9 is an example of a horizontally written document image. In FIG. 9, the rectangular and circular areas indicated by diagonal lines in the figure are each a pixel expression area and one character area of a figure, table, etc. When the area division method described in FIG. 3 is applied to the analysis target area P, character line blocks and pixel blocks indicated by 81 (where i=1...6) and F in the figure are obtained. Here, Sl is a character line containing characters such as figure annotations and captions, and S2...S6
is a normal text area. Each character in the character line block is separated into individual characters by, for example, the character separation method described above, and separated as shown by dotted lines in the figure.

Here, in the figure, C1+CI+6. C241C311
The blank area indicated by C%3°C6+ is determined by comparing the starting and ending character block positions with the starting and ending positions of the character line block, and is set as a blank block. Next, as shown in FIG. 8, When the document structure shown in FIG. 9 is automatically generated, it becomes as shown in FIG. 10. Note that FIG. 10 is an example in which the classification names between blocks, the distance between blocks, and the estimated character pitch values shown in FIG. 5 are used as structuring conditions.

Here, the text usually has a logical structure such as a paragraph using sections, indentations, etc., and dividing the text area into these units is useful for converting character code strings in the document image and for creating the desired document. This is effective when extracting regions.

One of the characteristics of headings, captions, and chapter titles is that they have blank blocks at the beginning and/or end of the character line.

Therefore, each character line 32゜S31 in text block 1゛
・Blank block Cta, C31, C53 in S6
, C61, the text block T is decomposed into the first
As shown in Figure 1, the character line 32. Text line S.

Paragraph block U1 character line S consisting of S,,S,,
It consists of three blocks of 6.

Also, the text line S is above the text block T and below the graphic block F, and there are two blank blocks C11lC16 at both ends, so it is a text line block that indicates the campsite of the pixel block F. I understand.

As described above, by extracting the properties of text lines within a text block, the integration of multiple text line blocks, and furthermore the logical properties of the text line blocks themselves, This becomes easy if the area is searched using a tree structure as shown in FIG.

FIG. 12 is a logical block diagram showing an embodiment of the first invention of the present application.

In the figure, 1 is a document image memory that stores document images as quantized image information. 2 is an area dividing section. The area dividing unit 2 divides the document image in the document image memory 1 from a global area into local areas while maintaining the vertical and horizontal relationships as explained in FIG. The block information obtained in the area division process as shown in FIG. 4 or 5 is sequentially stored in the structured data storage section 4. Here, among the block information output from the area dividing unit 2, character blocks that are child areas of the character line block are converted into area information for each character in the character separating unit 15, and stored in the structured data storage unit. Stored.

Further, the area dividing unit 2 determines whether the document image is written vertically or horizontally from the area dividing result, and stores the result in the vertical/horizontal information storage unit 3.

still,! Pointer-related values (parent area pointer, child area pointer, vertical relationship pointer, horizontal relationship pointer, etc.) of each block information stored in the structured data storage unit 4 are relative values within the structured data storage unit 4 of each block. Suppose that it is expressed by position.

Furthermore, it is assumed that each block also stores its own relative position within the structured data storage unit 4 as a block attribute value in a relative position pointer.

Further, it is assumed that the relative position counter 11 stores the next relative position of the last block stored in the structured data storage unit 4 from the area dividing unit 2 as an initial value, and the relative position counter 11 stores that value as an initial value. When issued, it is assumed that the count is incremented for each block information unit.

Next, as shown in FIG. 6, the block information control unit 4 pairs a plurality of blocks to be structured with the maximum division level and their parent blocks (blocks with one division level less). At the same time as fetching from the structured data storage unit 4, using the above-mentioned division level and Ml writing/horizontal writing information of the vertical/horizontal information storage unit 3, the vertical relationship and horizontal relationship pointer of each block is used to determine the block to be connected to that block. memorize the relative position of Furthermore, the relative position of the block first searched from the parent block is stored in the child area pointer of those parent blocks, and the parent block and a plurality of structuring target blocks are stored in the target data storage unit 6.

Next, the block information control unit 5 reads the parent block and the plurality of structuring target blocks from the target data storage unit 6 and transfers them to the structuring inspection unit 8 .

As explained in FIGS. 6 and 8, the structuring inspection section 8 performs a plurality of structuring inspections according to the contents of the condition storage section 7 that stores structuring conditions for the attribute values of the plurality of structuring target blocks. Sequentially inspect the attribute values of the block to be structured. If it becomes necessary to generate a new block when the above tests are performed sequentially, the above-described parent block and a plurality of blocks to be structured are transferred to the block generation section 9. The block generation unit 9 generates new blocks for a plurality of blocks to be structured among the plurality of blocks to be structured according to the attributes of the blocks, and as explained using FIGS. 6 and 8, The relative position of the first block to be structured is stored in the child area pointer of a new block, and the relative position of the newly generated block is stored in the parent area pointer of multiple blocks to be structured. After storing the positions, a process is performed to separate the pointers in the vertical or horizontal relationship between the plurality of blocks to be structured and the uninspected blocks.

It is assumed that the value of the relative position counter 11 is read out and set in the relative position pointer of the newly generated block.

Furthermore, the relative position of the parent block among the plurality of blocks to be structured is stored in the parent area pointer of the newly generated block.

Next, the block generation unit 9 writes a plurality of blocks structured as described above among the plurality of blocks to be structured into a predetermined relative position in the structured data storage unit 4, and creates a new block. The program generated in the previous step, a plurality of uninspected blocks to be structured, and their parent blocks are sent again to the structuring inspection unit 8, and the above-mentioned process is repeated.6 Next, the structuring inspection unit 8 If there is no need to generate new blocks for the plurality of blocks to be structured that are subjected to the 141st inspection, the plurality of blocks to be structured and their parent blocks are transferred to the attribute determination unit 10.

The attribute determining unit 10 determines the attribute value of the parent block as explained in FIG. 6, and writes the parent block and a plurality of structured blocks at predetermined relative positions in the structured data storage unit 4.

Next, the block information control unit 5, as described above,
A parent block with a structured division level and multiple ′! If the bears of the y4 structuring target block remain in the target data storage unit 6, those bears are sequentially transferred to the structuring inspection unit 8.

On the other hand, if the target data storage unit 6 is empty, then the division level is reduced by 12, the block to be structured and its parent block bare are taken out from the structured data storage unit 4, and the blocks at division level 1 are Similar operations are performed until the is extracted as a structuring target. As a result, the structure of the document stored in the document image memory 1 is automatically generated in the structured data storage unit 4 as a tree structure.

The document structure generation unit 20 (indicated by dotted lines in the figure) described above reconsiders the result of dividing the document image into the main structure obtained by the area division unit 2 and the character separation unit 15 in a bottom-up manner. , the structure of the document will be stored in the structured data storage section 4.

Next, as shown in FIGS. 6 and 8, the area search unit 1
3 reads blocks in a document image from an area definition storage unit that stores conditions regarding attributes and arrangement relationships of one or more blocks to be extracted in advance, and stores them in the structured data storage unit 4 according to the conditions. Space between a! A tree structure indicating the structure is searched, and one or more blocks to be extracted are stored in the extraction result storage unit 14 in a predetermined order.

FIG. 13 is a logical block diagram showing an embodiment of the second invention of the present application. In the figure, the document image memory 1, the area dividing section 2, and the character separating section 15 have the functions explained in FIG. In the block information obtained through the area dividing unit 2 and character separating unit 15, the blank block inspection unit 21 performs
Among the multiple child block information behind it (for example, multiple character blocks for a character line block), select the position of the child block that will be the start end and the position of the child block that will be the end, and The difference between the start and end positions of the parent block information is calculated, and the blank size at the start and end of the block information is determined. Blank blocks are detected by comparing the blank size with a predetermined threshold or character size such as character pitch.

If a blank block is detected, it is also added as one piece of block information, and the position of the blank block (starting end or end) and the position of the blank block are added as attribute values of the parent block where the blank block was detected. Assume that the size is memorized.

Therefore, the structured data storage section 4 has the same function as shown in FIG. 12, except that blank block information is also stored.

The function is to read the vertical/horizontal writing information stored in the vertical/horizontal information storage section 2 and generate the document arrangement structure as shown in FIGS. 6, 8 to 10 using the document structure generating section 20. , is equivalent to the case shown in FIG.

As explained in FIG. 11, the document block inspection unit 22 sequentially retrieves document block information and a plurality of character line block information that are child blocks from the structured data storage unit 4, and stores the information as attributes of the character line block. The position and size of blank blocks in the text block are sequentially inspected for all character line blocks constituting a text block, and a plurality of character line blocks constituting a paragraph block are detected.

When a plurality of character line blocks constituting a paragraph block are detected in the sentence block inspection section, a paragraph block is generated in the block updating section 23, and its attribute values are changed to the plurality of character line blocks constituting the paragraph block. Generate according to row block information. In addition, the parent area pointer of a paragraph block stores the relative position of the document block, and the parent area pointers of multiple character line blocks of a paragraph block store the relative position of the newly generated paragraph block. The document lflA shown in FIG.
The contents of the relative position counter in the m generation section 20 are read and set.

Note that the child area pointer of a paragraph block stores the relative position of the first character line block that makes up the paragraph block, and can be used to separate the vertical or horizontal relationship pointers of the start and end blocks of the multiple character line blocks that make up the paragraph, and A connection is also made between the vertical or horizontal relationship pointer of a paragraph block and the vertical or horizontal relationship pointer of an adjacent character line block or paragraph block.

Next, when the text block inspection unit 22 and the block unit 20 detect and generate paragraph blocks for one document block and the plurality of character line blocks that constitute it, the block update unit 23 performs the following: These data are written to predetermined relative positions in the structured data storage section 4.

The above operations are performed on all document blocks stored in the structured data storage unit 4 and the plural character line programs that constitute them.
By being done about ivy, the sentence block is
It is structured into paragraph blocks that constitute it.

Note that here, blank block information is also included as an attribute value of character line blocks that do not constitute a paragraph, and it goes without saying that it can be used together with the character pitch that represents the nature of the character line.

Area definition storage unit 12. Area search unit 13. The extraction result storage unit 14 is the same as that of the present invention described in item 1 described in FIG. 12.

(Effects of the Invention) As explained above, according to the document image analysis method of the present invention, the elements constituting various document images and the arrangement between the elements can be determined according to the inclusion relationship of blocks and the relative positional relationship vertically or horizontally. The structure is automatically generated hierarchically, and at the same time, the flow of text information and logical structures such as paragraphs are also obtained. Therefore, by searching for regions determined according to various purposes from this arrangement structure, it is possible to stably and easily extract one or more regions.

[Brief explanation of the drawing]

FIG. 1 and FIG. 2 are diagrams showing the structure of a document image written vertically and horizontally, respectively. FIG. 3 is a diagram showing an example of a document area division method in which area division is performed hierarchically while alternately defining vertical and horizontal division directions. FIGS. 4 and 7 are diagrams showing examples of the region division results of the document images shown in FIGS. 1 and 2, respectively. FIG. 5 is a diagram showing an example of block information. FIGS. 6 and 8 are diagrams showing examples in which a document structure is automatically generated according to the first invention of the present application from the region segmentation results shown in FIGS. 4 and 7, respectively. FIG. 9 is a diagram showing an example of a document image. FIGS. 10 and 11 are diagrams for explaining the document image analysis method of the second invention of the present application when applied to the document image shown in FIG. 9. FIG. 12 is a logical block diagram showing an embodiment of the first invention of the present application. FIG. 13 is a logical block diagram showing an embodiment of the second invention of the present application. In the figure, 1 is a document image memory, 2 is an area dividing unit, and 3 is a document image memory.
1 is a vertical/horizontal information storage unit, 4 is a structured data storage unit, 5 is a block information control unit, 6 is a target data storage unit, 7 is a condition storage unit, 8 is a structured inspection unit, 9 is a block generation unit, 10 is an attribute determination unit, 11 is a relative position counter, 12 is an area definition storage unit, 13 is an area search unit, 14 is an extraction result storage unit,
15 is a character separation section, 20 is a document structure generation section, 21 is a blank block inspection section, 22 is a text block inspection section, and 23 is a block update section.

Claims (2)

[Claims] (1) A means for decomposing a document image into element areas such as character lines and characters, and when structuring one or more of the element areas as blocks, according to the inclusion relationship and vertical or horizontal arrangement relationship of each block. , a document structure generating means for hierarchically determining and storing the attributes of the blocks and the arrangement structure between the blocks, and an area to be extracted in the document image based on the attributes of the blocks and the hierarchical arrangement structure between the blocks. or a region search means for searching one or more blocks constituting the region to be extracted. (2) A means for decomposing a document image into element areas such as character lines and characters, and when structuring one or more of the element areas as blocks, according to the inclusion relationship and vertical or horizontal arrangement relationship of each block. , document structure generating means for hierarchically determining and storing the attributes of the blocks and the arrangement structure between blocks; means for extracting a blank having a predetermined position and size within a character line block as a blank block; document structure updating means for checking the shape of each character line block based on the blank block from blocks containing a plurality of character lines generated by the document structure generating means, and updating the arrangement structure in the document image; A document image analysis method comprising: an area search means for searching for an area to be extracted in the document image from the hierarchical arrangement structure or one or more blocks constituting the area to be extracted. .