JP2005250816A - Document image layout analyzing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To extract an appropriate line or a text block from the document image of a complicated layout. <P>SOLUTION: A blank area in a document image is extracted as a virtual separator (Step S1). While integration of a text element exceeding the virtual separator is prohibited, a plurality of text elements are integrated and extracted as an integrated text element (Step S2). By this, the integrated text element corresponding to the document image layout is extracted. Verification is made on whether or not the extracted integrated text element is appropriate as a line or a text block (Step S3). In the event that the verification result shows its inappropriateness, the size of the blank area is changed by a control parameter, a virtual separator based on the blank area changed against the integrated text element verified as inappropriate is extracted newly, and the processing of the Step S2 is carried out. By the recursive repetition of the processing such as this, an appropriate line or text block can be extracted even in a document image of a complicated layout. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a document image layout analysis program for analyzing a document image layout which is a physical arrangement of elements such as characters, lines, text blocks, diagrams, and frames in a document image.

  2. Description of the Related Art In recent years, optical character reading (OCR) that identifies character components in a document image captured by a computer using an optical device such as a scanner and outputs it as a character code has been actively used. OCR optically scans and reads document images of printed characters, handwritten characters, etc., and obtains quantized data. Then, a text block including a character component is extracted from the data. Next, character components are extracted from the text block, and character recognition is performed by a technique such as pattern matching.

  Conventionally, the following method has been proposed as a method for extracting a text block from a document image. For example, Patent Document 1 discloses that a set of basic elements has closeness (character components are arranged relatively densely) and homogeneity (the size of character components is approximately the same). Etc.) to create a row based on Similarly, a row (text block) is also generated for a set of rows based on their proximity and homogeneity. At the same time, it is disclosed that lines and columns (text blocks) are extracted again by considering the generated columns (text blocks) as constraints. Specifically, based on the connected components of black pixels in the document image as basic elements, they are integrated to generate lines, and the lines are integrated to generate a text block. Whether or not two document elements are to be integrated is determined based on the relationship between the size and position of the document element with other document elements around it.

Further, in Patent Document 2, a set of blank areas is extracted from a document image, a set satisfying a condition regarding a predetermined size is selected from them, and areas other than areas covered by them are extracted. Thus, it is disclosed that a text block region is cut out from a document image.
JP 11-219407 A JP-A-2-263272

  However, the layout of document elements is complex and diverse, and the same document element on the first floor is configured from only information on the local arrangement relationship between document elements for a set of document elements as in Patent Document 1. It is extremely difficult to link document elements to be linked. For example, when text blocks are arranged in an intricate manner, or when text blocks and diagrams are arranged in an intricate manner, character components are overintegrated, and a plurality of character strings are combined into one line. There was a problem.

21 and 22 are diagrams showing examples of over-integration. FIG. 21 shows a text block extraction result, and FIG. 22 shows a line extraction result.
FIG. 21 shows that two text blocks 310 and 320 are extracted from the document image 300. FIG. 22 shows line extraction results (lines a to n) in the text blocks 310 and 320 as shown in FIG. Here, although the rows a to h are correctly extracted, the rows j to n are overintegrated over the left and right, and the rows i, k, m, and n are overintegrated over the top and bottom. Even if these lines are recognized, correct sentences cannot be obtained.

Further, the method of Patent Document 2 in which a text block area is extracted as a closed area by a set of blank areas that satisfy certain conditions has the following problems.
The size of an appropriate blank area surrounding the text block area differs depending on each area of the document image, and it is difficult to select an appropriate blank area under fixed conditions.

FIG. 23 is an example of a layout in which a text block area is arranged in an area surrounded by a plurality of figures.
In such a case, since the text block area 330 is arranged in a form that cannot be separated from other document elements such as a figure by a simple rectangle, the text block area as in the method of Patent Document 2 is used. It is more difficult to cut out a text block area by selecting an appropriate blank area to enclose.

  The present invention has been made in view of these points, and an object thereof is to provide a document image layout analysis program capable of extracting an appropriate text block even in a document image having a complicated layout.

  In the present invention, in order to solve the above problem, in a document image layout analysis program for causing a computer to perform a process of analyzing a document image layout, as shown in FIG. Extracting as a separator (step S1), and prohibiting the integration of text elements beyond a virtual separator, a plurality of text elements are integrated and extracted as an integrated text element (step S2), and the process is executed. A document image layout analysis program is provided.

  According to the above document image layout analysis program, the computer 10 extracts a blank area in the document image as a virtual separator. Furthermore, while prohibiting the integration of text elements beyond the virtual separator, a plurality of text elements are integrated and extracted as an integrated text element. Thereby, an integrated text element corresponding to the document image layout is extracted.

  In the present invention, the computer 10 verifies whether or not the extracted integrated text element is a line or a text block as conforming or nonconforming (step S3), and in the case of nonconforming, the size of the blank area is changed by a control parameter, It is characterized in that a virtual separator is re-extracted from the integrated text element determined to be incompatible, and a process of recursively repeating the process of extracting a new integrated text element is further performed.

  As a result, the computer 10 verifies whether the extracted integrated text element is suitable or incompatible as a line or a text block. In addition, if the result of the verification is incompatible, the size of the blank area is changed by the control parameter, and a virtual separator is re-extracted based on the changed blank area for the non-conforming integrated text element. To do. Furthermore, a new integrated text element is extracted by integrating a plurality of text elements while prohibiting integration of text elements beyond the virtual separator. By repeating such processing recursively, an appropriate line or text block is extracted.

  In the present invention, a blank area in a document image is extracted as a virtual separator, and a plurality of text elements are integrated and extracted as an integrated text element while prohibiting integration of text elements exceeding the separator. Therefore, the integrated text element corresponding to the layout of the document image can be extracted.

  Also, the extracted integrated text element is verified as conforming or nonconforming as a line or text block. If the result of the verification is that the integrated text element does not conform, it is blank until the condition as a line or text block is satisfied. The process of re-extracting the virtual separator and extracting the integrated text element is recursively repeated by changing the size of the area according to the control parameter, so that an appropriate integrated text element that can be recognized as a correct sentence can be extracted. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing the principle of a document image layout analysis program of the present invention.
The document image layout analysis program of the present invention is a program that causes the computer 10 to function to analyze the document image layout. This program extracts a blank area in a document image as a virtual separator (hereinafter referred to as a virtual separator) to the computer 10 (step S1), and prohibits integration of text elements beyond the virtual separator. The text elements are integrated and extracted as an integrated text element (step S2).

  Note that a text element is a character component, a line composed of a plurality of character components, or a text block composed of a plurality of lines, and an integrated text element is a line or text block obtained by integrating a plurality of text elements.

  Further, the program verifies whether the extracted integrated text element is suitable or non-conforming as a line or a text block (step S3). If the integrated text element is not correct as a result of the verification, the program returns to step S1. The process of changing the size of the blank area according to the control parameter (details will be described later), re-extracting the virtual separator for the non-conforming integrated text element, and extracting the new integrated text element recursively. Make the process repeat.

Hereinafter, an outline of document image layout analysis processing by the computer 10 when the above-described program is executed will be described with a specific example.
2 to 5 are diagrams showing an outline of document image layout analysis processing by the document image layout analysis program of the present invention.

  When the document image layout analysis process starts, a blank area (blank rectangle) for determining the size (size in the X and Y directions) according to a certain control parameter (described later) in the document image 20 as shown in FIG. Are extracted as virtual separators 21a, 21b, 21c, 21d, and 21e (step S1).

  Next, as shown in FIG. 3, a character element or line that is a text element beyond the virtual separators 21a, 21b, 21c, 21d, and 21e extracted in step S1 is prohibited. Based on the proximity and homogeneity between the components, the character components are combined into lines, and the text elements are similarly integrated into text blocks, and extracted as integrated text elements (here, text blocks) 22a, 22b, 22c, 22d, and 22e. (Step S2).

  Further, it is verified whether or not the extracted text blocks 22a, 22b, 22c, 22d, and 22e satisfy the condition as a text block (step S3).

  The condition here is a condition for a text block that can be recognized as a correct sentence by character recognition. For example, in the text blocks 22a, 22b, 22c, 22d, and 22e extracted as shown in FIG. 3, the text blocks 22b and 22c are not recognized as correct sentences even if character recognition is performed. In the verification process in step S3, It is determined that the condition as a text block is not met and is not suitable (details about this condition will be described later).

  At this time, returning to step S1, the size of the blank area to be extracted as the virtual separator is changed by the control parameter (specifically, a thinner blank area is extracted as the virtual separator), and the text block 22b. , 22c, virtual separators 23a, 23b, 23c, 23d, and 23e are re-extracted as shown in FIG. 4, and new text blocks 24a, 24b, 24c, 24d, 24e, and 24f are extracted as shown in FIG.

The above processing is recursively repeated until the condition as a text block is satisfied.
The control parameter is set based on the number of recursions, the size of the integrated text element, and the size of characters included in the integrated text element.

According to such a document image layout analysis program, an integrated text element corresponding to the layout of the document image can be extracted.
Also, the extracted integrated text element is verified as conforming or nonconforming as a line or text block. If the result of the verification is that the integrated text element does not conform, it is blank until the condition as a line or text block is satisfied. The process of re-extracting the virtual separator and extracting the integrated text element is recursively repeated by changing the size of the area according to the control parameter, so that an appropriate integrated text element that can be recognized as a correct sentence can be extracted. .

Details of the embodiment of the present invention will be described below.
FIG. 6 is a hardware configuration example of a document image layout analysis apparatus to which the document image layout analysis program is applied.

  The document image layout analyzing apparatus 100 is, for example, a PC (personal computer), a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an HDD (Hard Disk Drive) 104, a graphic. The processing unit 105, an input I / F (Interface) 106, a communication I / F 107, and the like are connected to each other via a bus 108.

Here, the CPU 101 controls each unit according to programs stored in the ROM 102 and the HDD 104 and various data.
The ROM 102 stores basic programs and data executed by the CPU 101.

The RAM 103 stores programs being executed by the CPU 101 and data being calculated.
The HDD 104 stores an OS (Operation System) executed by the CPU 101, various application programs such as a document image layout analysis program of the present invention, and various data such as document image data read by an optical device such as a scanner (not shown).

  For example, a display 105 a is connected to the graphic processing unit 105 as a display device, and a document image or the like is displayed on the screen of the display 105 a in accordance with a drawing command from the CPU 101.

A mouse 106 a and a keyboard 106 b are connected to the input I / F 106, and information input by the user is received and transmitted to the CPU 101 via the bus 108.
The communication I / F 107 is connected to a network 120 such as the Internet, for example, and communicates with other devices connected on the network 120.

Next, details of the document image layout analysis process performed by the document image layout analysis apparatus 100 will be described.
In the following description, the integrated text element is described as a text block, but may be a line.

  In the document image layout analysis processing described below, under the control of the CPU 101, for example, the document image layout analysis program of the present invention stored in the ROM 102 or the HDD 104 and various data such as document image data are read out to the RAM 103. Realized by being deployed and executed.

FIG. 7 is a diagram showing an overview of the entire document image layout analysis process.
As shown in the figure, the document image layout analysis process includes a connected component attribute assignment process (step S10) and a recursive text block extraction process (step S20).

  In the connected component attribute assigning process (step S10), any of the attributes of a character component, a separator, a figure, a frame, and noise is assigned to all the connected components by the black pixels of the document image. Here, the figure is not a character component, a separator, a frame, or noise, but a connected component that does not include a character component in itself. A frame is a frame including a plurality of character components inside.

  The recursive text block extraction process (step S20) corresponds to the process of steps S1 to S3 shown in FIG. For the set of connected components to which attributes are assigned in the connected component attribute assigning process in step S10, the virtual separator is extracted, the text block is extracted, and whether the text block is satisfied is verified. If not correct, the size of the blank area is changed by the control parameter, the virtual separator is re-extracted from the text block, and the process of extracting a new text block is recursively repeated.

First, details of the connected component attribute assigning process will be described.
FIG. 8 is a flowchart of an example showing the flow of the connected component attribute assigning process.
For example, when a document image stored in the HDD 104 is retrieved under the control of the CPU 101, a labeling process is first performed on the document image.

FIG. 9 is a diagram illustrating a specific example of the labeling process.
For example, the character component “ta” includes three connected components 201, 202, and 203 formed of black pixels. In the labeling process, the coordinate values (XY coordinates) of the circumscribed rectangles 201a, 202a, and 203a, which are the smallest rectangles surrounding the connected components 201, 202, and 203, are obtained, thereby acquiring information about the connected components of black pixels. Such processing is performed for all connected components in the document image (step S11).

  Next, a separator determination process is performed on the set S of connected components obtained by the process of step S11. Here, when the length of the long side of the circumscribed rectangle of the connected component is not less than a certain value and the aspect ratio of the circumscribed rectangle is not less than a certain value, it is determined that the connected component is a separator (step S12). ).

  Also, noise discrimination processing is performed on the set S of connected components. Here, when the area of the circumscribed rectangle of the connected component is equal to or smaller than a certain value, it is determined that the connected component is noise (step S13).

Further, the hierarchization process is performed on the set Sa excluding the separator and noise determined in the processes of steps S12 and S13 from the set S of connected components.
FIG. 10 is a diagram illustrating a specific example of the hierarchization process.

  For example, a character component such as “ku” is composed of two connected components 211 and 212 composed of black pixels. These are labeled in the process of step S11 by the coordinate values of the circumscribed rectangles 211a and 212a of the connected components 211 and 212. In the case of this character, the connected component 212 is included in the connected component 211. If there is a relationship between such connected components, a hierarchical process is performed in which the connected component 212 is registered in the “child” of the connected component 211 and the connected component 211 is registered in the “parent” of the connected component 212 (step S14). ).

  When the hierarchization process is completed, a character recognition process is performed on the set Sa. Here, for the connected component, first, the circumscribed rectangular region is recognized as it is. When the reliability of the character recognition result is high, the character component flag “CH” is attached to the connected component. Next, for example, as shown in FIG. 10, when there is a connected component having “children”, overlap integration is performed on a set of connected components of “children”, and all the overlapping components obtained as a result thereof are integrated. One character is recognized in the circumscribed rectangular area.

FIG. 11 is a diagram showing a frame including a character component.
As shown in this figure, the connected component 220 has circumscribed rectangular regions 221, 222, 223, and 224 with high reliability of the character recognition result as a “child” as a result of overlap integration. As described above, when there are a certain number or more of “children” having high reliability of the character recognition result, the connected component 220 may be a frame surrounding a plurality of characters. A frame flag “FR” is attached (step S15).

  Next, a character component / frame / drawing discrimination process is performed on the set Sa. Specifically, if a connected component has a character component flag “CH”, and if there is a “parent” of the connected component, if the “parent” does not have a character component flag “CH”, the “parent” The frame flag “FR” is attached to “. When the character component flag “CH” is attached to “parent”, the reliability of the connected component that is “child” is compared with the reliability of the connected component of “parent”, and the connected component of “child” is compared. Is higher, the “parent” character component flag “CH” is canceled and the frame flag “FR” is added. Here, for the set Sa, all connected components with the frame flag “FR” are made frames. Among the remaining connected components, those that do not have the character component flag “CH” and the area of the circumscribed rectangle is greater than or equal to a certain value is shown in the figure. Further, the remaining connected components are set as character components (step S16).

Finally, overlap integration processing is performed on the set of connected components as character components (step S17).
With the processing as described above, any attribute of a character component, a separator, a figure, a frame, and noise can be assigned to all connected components in the document image.

Next, details of the recursive text block extraction processing in step S20 shown in FIG. 7 will be described.
FIG. 12 is a flowchart of an example showing the flow of recursive text block extraction processing.

When the connected component attribute addition process is completed and the recursive text block extraction process is started, first, a process for obtaining a maximum blank rectangle in a certain rectangular area P in the document image is performed.
The blank rectangle in the document image is a rectangular area in the document image and does not include black pixels inside. A set of blank rectangles having no blank rectangle including itself other than itself is called a maximal blank rectangle.

FIG. 13 is a diagram illustrating an example of a maximum blank rectangle.
A rectangular area P in the document image is represented. In this rectangular area P, a circumscribed rectangle set S = {R _i εP, i = 1, 2,..., N} obtained by the processing described in step 10 of FIG. Shows the case of n = 5). Here, it is assumed that a set of blank rectangles in the rectangular area P (hereinafter referred to as S blank rectangle in P) is a rectangular area in the rectangular area P and does not overlap all rectangles belonging to the set S. Further, among the set of S blank rectangles in P, one in which there is no S blank rectangle in P including itself other than itself is called an S maximum blank rectangle in P. Hereinafter, a set of S maximum blank rectangles in P is represented by M (P, S). FIG. 13 shows the S maximum blank rectangle 230 in P that is the largest within the rectangular area P of M (P, S).

M (P, S) is determined by the control parameters n and x and is defined by the following equation.
M _{n, x} (P, S) = {TεM (P, S) | min (T ^X , T ^Y ) ≧ n and max (T ^X , T ^Y ) ≧ x}
Here, T ^X represents the horizontal (X direction) length of the S maximum blank rectangle T in P included in M (P, S), and T ^Y represents the vertical length (Y direction) of T. min (T ^X , T ^Y ) ≧ n indicates that the shorter of T ^X , T ^Y is greater than or equal to the control parameter n, and max (T ^X , T ^Y ) ≧ x is longer of T ^X , T ^Y. Is greater than or equal to the control parameter x (step S21).

The control parameters n and x are set based on the number of recursions, the size of the text block, and the size of characters included in the text block.
By the way, based on the attribute given to the circumscribed rectangle by the above-described connected component attribute assigning process, the circumscribed rectangle set of the character component of the circumscribed rectangle set S is “C”, and the character component such as a frame, separator, figure, etc. Assume that a set of circumscribed rectangles other than is “H”, and “H” is classified as a link prohibited area that prohibits integration with other circumscribed rectangles.

At this time, C∪H (the sum of C and H) maximum blank rectangle set M _{n, x} (P, C∪H) in the rectangular area P obtained in step S21 is used as a virtual forbidden area “H”. Add to The added link prohibited area is set to “Ha” (step S22).

  Next, the text block is extracted by integrating the character component circumscribed rectangle set “C” beyond the link prohibition area “Ha” based on proximity or homogeneity. . A specific method is disclosed in JP-A-11-219407. As a processing result, a text block and lines constituting the text block are obtained (step S23).

Next, the number of extracted text blocks is set to l, the loop count i = 0 (step S24), and the following processing is repeated until i = 1.
That is, it is determined whether or not i <l (step S25). If i <l, whether or not the extracted text block B _i satisfies the text block compatibility condition (details will be described later). (Step S26), i is incremented if satisfied (step S27), and the process returns to step S24. When i = 1, that is, when all the text blocks B _i in the rectangular area P satisfy the text block compatibility condition, the processing for the rectangular area P in the document image is terminated, and another rectangle is obtained. The processing from step S21 is repeated for the area (return).

On the other hand, if the text block B _i does not meet the text block compatible conditions in the process of step S26, the text block B _i a rectangular region P, the circumscribed rectangle U characters components of the text block B _i, other than the character component Are newly defined as C = {U∈C | U∩P ≠ φ} and H = {V∈H | V∩P ≠ φ}. Then, the control parameters n and x are changed for these P, C, and H (step S28), and the processing from step S21 is performed again (step S29). When the recursive process ends (returns), i is incremented (step S27), and then the process returns to step S24 to verify the next text block B _{i + 1} (step S26).

In the recursive process, the control parameters n and x are set to decrease. That is, in the text block B _i that does not satisfy the text block compatibility condition, the maximum blank rectangle set as the virtual separator is made smaller.

  In this way, even when the text block and the figure are arranged in a complicated manner, it is possible to solve the problem that the character components are overintegrated and a plurality of character strings are combined into one line.

Next, details of the process of verifying the suitability as a text block in FIG. 12 (step S26) will be described.
In order to determine whether the text block extraction result in the process of step S23 satisfies the condition as a text block (text block compatibility condition), the following two processes are performed.

FIG. 14 is a diagram showing an outline of the process of verifying compatibility as a text block.
Step S30: For each line constituting the text block, it is determined whether the line includes two or more characters in the vertical direction of the line direction (vertical or horizontal).

FIG. 15 is a diagram showing an example of lines constituting a text block.
Assume that lines 241 to 245 are obtained in the text block 240 as shown in the figure. At this time, the lines 242, 244 and 245 include two or more characters in the vertical direction of the line direction. In other words, the extracted row has a plurality of rows in the region in the vertical direction of the row. Such a line 242, 244, 245 is not suitable as a line because a correct sentence cannot be obtained even if character recognition is performed, and the text block 240 including such a line 242, 244, 245 is a text block As non-conforming.

Step S40: Regarding the lines constituting the text block, it is determined whether or not a line having a predetermined number or more intersects the same blank area larger than the character spacing.
FIG. 16 is a diagram showing a blank area of the text block of FIG.

  As shown in the figure, in the text block 240, all lines intersect with the same blank area 250 larger than the character spacing. Such a text block 240 is not suitable as a text block because a correct sentence cannot be obtained even if character recognition is performed.

Hereinafter, the details of each process of FIG. 14 will be described.
FIG. 17 is a flowchart showing details of the first process of the text block compatibility verification process.

For the extracted text block B _i , the line extraction result is {L _j : j = 1, 2,..., M}. At this time, first, j = 0 is set (step S31), and the following processing is repeated until j = m.

First, it is determined whether j <m is satisfied (step S32). If a j <m, compared row L _j, obtaining the character candidate M _Lj within a row L _j from the character candidate set M. The character candidate set M is a character component having a high recognition reliability (one to which the above-mentioned flag “CH” is added) among those determined as character components in the above-described connected component attribute assignment processing (FIG. 7). (Step S33).

Next, a line is generated for the character candidate _MLj obtained in the process of step S33.
Line generation is generated in accordance with the reading order is given to the character candidate M _Lj in row L _j.

FIG. 18 is a diagram showing an outline of processing for assigning a reading order.
Character candidates M _Lj (illustrated by a circumscribed rectangle) included in the row L _j are sorted, for example, in ascending order of the Y coordinate of the upper left point of the circumscribed rectangle. When the Y coordinate is the same, the X coordinate is sorted in ascending order. Thereby, the character candidates M _Lj are numbered (1) to (16) as shown in the figure.

Here, the corner of the character string rectangle surrounding the line L _j is taken as the origin 260, sets a rectangular test region 261 including a character candidate M _Lj its origin 260 as a corner. Reading order character candidate M _Lj is the rectangular inspection area 261 and 262 is determined under the condition that do not include those of the reading order behind the character candidates M _Lj, the reading order of character candidates M _Lj .

Specifically, first, in order from the character candidate M _Lj of the number (1), a rectangular inspection area including the character candidate M _Lj is set with the origin 260 as one corner, and the character candidate M of the number after the character candidate M _Lj is set. _{When Lj} is not included, a reading number is assigned. If a number after the character candidate _MLj is included, no reading number is assigned. For example, a rectangular test area 261 that includes a character candidate M _Lj number (1) as shown in Figure 18 are numbered (4), including (5), (6), character candidate M _Lj (7). Therefore, no reading number is assigned. Similarly, reading numbers are not _{assigned to} the character candidates _MLj of the numbers (2) and (3). For the character candidate _MLj of number (4), the rectangular inspection area 262 does not include anything else, so reading number 1 is assigned. By repeating such processing, reading numbers 1 to 16 as shown in the figure are assigned to the row L _j .

FIG. 19 is a diagram showing an outline of row generation processing.
The character candidates M _Lj are integrated in order according to the reading order (assigned reading number) determined by the processing shown in FIG. 18, and a new row L _jk is generated. Here, as shown in FIG. 19, if an attempt is made to generate a line L _jk represented by a circumscribed rectangle 271 (illustrated by a dotted line) by integrating the character candidates M _Lj of the reading number 5, The character candidate M _Lj with the reading number 8 is included in the circumscribed rectangle 271. In this case, the character candidates M _{Lj with} the reading number 5 are not integrated, and the character candidates M _Lj with the reading numbers 1 to 4 are determined as one row L _j1 . Thus, under the condition that does not contain those behind the reading order from the character candidate M _Lj to integrate, as a new product line by integrating the character candidate M _Lj sequentially in accordance with the reading order. Similarly, new rows L _j2 , L _j3 , L _j4 are generated as shown in the figure (step S34).

Next, it is determined whether or not a plurality of rows L _jk newly generated by the process of step S34 are arranged in the vertical direction of the row L _j (step S35). If there is no such row L _jk , j is incremented (step S36), and the processing from step S32 is repeated. In addition, when a plurality of lines L _j2 and L _j3 arranged in the vertical direction are generated as in the line L _j as shown in FIG. 19, the line L _j is determined as a non-conforming line, and the line The extracted text block B _i including L _j is determined to be incompatible (step S37), and the process is terminated.

On the other hand, when j = m is satisfied in step S32, that is, when it is determined that all the lines L _j constituting the text block B _i do not have a plurality of lines L _jk in the vertical direction. The text block B _i is determined to be suitable as a text block (step S38), and the process ends.

Next, details of the second process (step S40) of the text block compatibility verification process shown in FIG. 14 will be described.
FIG. 20 is a flowchart showing details of the second process of the text block compatibility verification process.

Similarly to the text block compatibility verification process in step S30 described above, the line extraction result is set to {L _j : j = 1, 2,..., M} for the extracted text block B _i . First, the character component C _i included in the text block B _i is obtained (step S41).

Next, an average character spacing estimated from the character component C _i which is a connected component included in the text block B _i is obtained (step S42). Then, the average character interval, the control parameter (n, x) to set the maximum blank rectangle described above to set the (step S43), C _i maximum blank rectangle set M _n in _{B i, x (B i,} C i ) Is obtained (step S44).

Then, C _i maximum blank rectangle set M _n in _{B i, x (B i,} C i) maximum blank rectangle _{contained, {L j: j = 1,2} , ..., m} predetermined number of rows in (Th) It is determined whether or not there is anything that intersects the above line L _j (step S45). Here, if there is no such maximal blank rectangle, the extracted text block B _i is determined to be suitable (step S46), and if there is an object that intersects a certain number of lines L _j , the text block B _i is assumed to be incompatible (step S47), completing the text block compliance verification process.

For example, the predetermined number of lines (th) is set according to the number of lines constituting the extracted text block _Bi, and is set to, for example, 10% of the number of lines.
As described above, according to the present invention, when the text blocks are arranged in an intricate arrangement, which is difficult in the prior art, the text block and the figure are arranged in an intricate arrangement, Even if the document elements are arranged in a rectangle that cannot be separated from the document elements, the text block can be correctly extracted. Thereby, a text block can be extracted with high accuracy for a document having a complicated layout structure such as a magazine or an advertisement.

  In the above, text blocks and lines are extracted by integrating text elements. However, by extracting the closed region generated based on the extracted virtual separator as it is, in addition to text elements, figures and tables are extracted. Document elements such as photographs may be extracted.

  Then, it is verified as described above whether the text element of the extracted document elements conforms or does not conform as a line or a text block, and if it does not conform, the size of the blank area is changed by the control parameters n and x, A process may be performed in which a virtual separator is re-extracted from the text element and the process of extracting the document element is recursively repeated.

  The above processing functions can be realized by a computer (having a hardware configuration as shown in FIG. 6). In that case, a program describing the processing contents of the functions that the document image layout analysis apparatus 100 should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Examples of the magnetic recording device include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape. Optical discs include DVD (Digital Versatile Disc), DVD-RAM, CD-ROM, CD-R (Recordable) / RW (ReWritable), and the like. Magneto-optical recording media include MO (Magneto-Optical disk).

  When distributing the program, for example, portable recording media such as a DVD and a CD-ROM on which the program is recorded are sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

The present invention is applied when character recognition processing is performed on various document images such as advertisement articles read by an optical device such as a scanner.
(Supplementary Note 1) In a document image layout analysis program for causing a computer to function to analyze a document image layout,
On the computer,
Extract the blank area in the document image as a virtual separator,
Under the prohibition of integration of text elements beyond the virtual separator, a plurality of the text elements are integrated and extracted as an integrated text element.
A document image layout analysis program characterized by causing a process to be executed.

(Supplementary Note 2)
Verify whether the extracted integrated text element is conforming or non-conforming as a line or text block;
In the case of non-conformity, a process of changing the size of the blank area by a control parameter, re-extracting the virtual separator for the unconforming integrated text element, and extracting a new integrated text element Repeat recursively,
The document image layout analysis program according to appendix 1, wherein the processing is further executed.

(Additional remark 3) In the line in the extracted said integrated text element, the character which recognizes the thing which recognized the character which integrated the connected component or the said connected component contained in the character string rectangle surrounding the said line, and has high recognition reliability is used as a character. As a candidate,
Set a rectangular inspection area that includes the character candidate with one corner of the character string rectangle as the origin and the origin as the corner,
Determining the reading order of the character candidates under the condition that the rectangular inspection area does not include a reading order after the character candidates;
A new generation line generated by integrating the character candidates in order according to the reading order under the condition that the reading order after the character candidates to be integrated is not included, The document image layout analysis program according to supplementary note 2, wherein when there are a plurality of lines in the vertical direction of the line, the line is determined to be incompatible with the line.

(Additional remark 4) In the line in the extracted said integrated text element, the character which recognizes the thing which recognized the character which integrated the connected component or the said connected component contained in the character string rectangle surrounding the said line, and has a high recognition reliability is used as a character. As a candidate,
Set a rectangular inspection area that includes the character candidate with one corner of the character string rectangle as the origin and the origin as the corner,
Determining the reading order of the character candidates under the condition that the rectangular inspection area does not include a reading order after the character candidates;
A new generation line generated by integrating the character candidates in order according to the reading order under the condition that the reading order after the character candidates to be integrated is not included, The document image layout analysis program according to appendix 2, wherein when there are a plurality of lines in the vertical direction of the line, the text block including the line is determined to be incompatible as a text block.

  (Additional remark 5) In the line contained in the extracted said integrated text element, the said line more than predetermined number of lines is from the average character space | interval of the said integrated text element estimated from the connection component in the said integrated text element. The document image layout analysis program according to appendix 2, wherein the extracted integrated text element is determined to be incompatible as a text block when intersecting with the same large blank area.

(Additional remark 6) In the line extraction result verification program which makes a computer function the process which verifies the line extracted from the document image,
On the computer,
In the extracted line, a character candidate that has a high recognition reliability by recognizing a connected component or a combination of the connected components included in a character string rectangle surrounding the line,
Set a rectangular inspection area that includes the character candidate with one corner of the character string rectangle as the origin and the origin as the corner,
Determining the reading order of the character candidates under the condition that the rectangular inspection area does not include a reading order after the character candidates;
A new generation line generated by integrating the character candidates in order according to the reading order under the condition that the reading order after the character candidates to be integrated is not included, The extracted row is determined to be incompatible as a row when there are a plurality of rows in the vertical direction of the row,
A row extraction result verification program characterized by causing processing to be executed.

(Supplementary Note 7) In a text block extraction result verification program that causes a computer to function to verify a text block extracted from a document image,
On the computer,
In a line included in the extracted text block, a character candidate that has a high recognition reliability by recognizing a connected component or a combination of the connected components included in a character string rectangle surrounding the line,
Set a rectangular inspection area that includes the character candidate with one corner of the character string rectangle as the origin and the origin as the corner,
Determining the reading order of the character candidates under the condition that the rectangular inspection area does not include a reading order after the character candidates;
A new generation line generated by integrating the character candidates in order according to the reading order under the condition that the reading order after the character candidates to be integrated is not included, Determining that a text block including the line is incompatible as a text block when there are a plurality of lines in the vertical direction of the line in
A text block extraction result verification program characterized in that a process is executed.

(Additional remark 8) In the text block extraction result verification program which makes a computer function the process which verifies the text block extracted from the document image,
On the computer,
Among the lines included in the extracted text block, the lines of a predetermined number or more intersect with the same blank area larger than the average character spacing of the text block estimated from the connected components in the text block. When the extracted text block is determined to be incompatible as a text block,
A text block extraction result verification program characterized in that a process is executed.

(Supplementary Note 9) In a document image layout analysis program for causing a computer to function to analyze a document image layout,
On the computer,
Extract the blank area in the document image as a virtual separator,
Extracting document elements by extracting a closed region generated based on the virtual separator;
Verify whether the text elements of the extracted document elements are conforming or non-conforming as lines or text blocks,
In the case of non-conformity, the size of the blank area is changed by a control parameter, the virtual separator is re-extracted for the non-conforming text element, and the process of extracting the document element is recursively repeated.
A document image layout analysis program characterized by causing a process to be executed.

(Supplementary Note 10) In a document image layout analyzing apparatus for analyzing a document image layout,
Virtual separator extraction means for extracting a blank area in a document image as a virtual separator;
Integrated text element extraction means for integrating a plurality of the text elements and extracting them as an integrated text element, while prohibiting integration of text elements beyond the virtual separator;
A document image layout analyzing apparatus comprising:

(Additional remark 11) It further has a verification means which verifies whether the extracted said integrated text element is conformity or nonconformity as a line or a text block,
In the case of non-conformity, the virtual separator extracting means changes the size of the blank area according to a control parameter, re-extracts the virtual separator for the unconforming integrated text element, and the integrated text element The document image layout analysis apparatus according to appendix 10, wherein the extraction means recursively repeats the process of extracting the new integrated text element.

It is a figure which shows the principle of the document image layout analysis program of this invention. It is a figure which shows the outline | summary of the document image layout analysis process by the document image layout analysis program of this invention (the 1). It is a figure which shows the outline | summary of the document image layout analysis process by the document image layout analysis program of this invention (the 2). It is a figure which shows the outline | summary of the document image layout analysis process by the document image layout analysis program of this invention (the 3). It is a figure which shows the outline | summary of the document image layout analysis process by the document image layout analysis program of this invention (the 4). 2 is a hardware configuration example of a document image layout analysis apparatus to which a document image layout analysis program is applied. It is a figure which shows the outline | summary of the whole document image layout analysis process. It is a flowchart of an example which shows the flow of a connection component attribute provision process. It is a figure which shows the specific example of a labeling process. It is a figure which shows the specific example of a hierarchization process. It is the figure which showed the flame | frame containing a character component. It is a flowchart of an example which shows the flow of a recursive text block extraction process. It is a figure which shows an example of a maximum blank rectangle. It is a figure which shows the outline of the process of the compatibility verification as a text block. It is a figure which shows an example of the line which comprises a certain text block. It is the figure which showed the blank area | region of the text block of FIG. It is a flowchart which shows the detail of the 1st process of a text block compatibility verification process. It is a figure which shows the outline of the process which provides a reading order. It is a figure which shows the outline of a process of line production | generation. It is a flowchart which shows the detail of the 2nd process of a text block compatibility verification process. It is a figure which shows the example of overintegration, and is a figure which shows a text block extraction result. It is a figure which shows the example of overintegration, and is a figure which shows a row extraction result. It is an example of a layout in which a text block area is arranged in an area surrounded by a plurality of figures.

Explanation of symbols

10 Computer 20 Document image 21a, 21b, 21c, 21d, 21e, 23a, 23b, 23c, 23d, 23e Virtual separator 22a, 22b, 22c, 22d, 22e, 24a, 24b, 24c, 24d, 24e, 24f Text block

Claims (5)

In a document image layout analysis program that causes a computer to function to analyze a document image layout,
On the computer,
Extract the blank area in the document image as a virtual separator,
Under the prohibition of integration of text elements beyond the virtual separator, a plurality of the text elements are integrated and extracted as an integrated text element.
A document image layout analysis program characterized by causing a process to be executed. In the computer,
Verify whether the extracted integrated text element is conforming or non-conforming as a line or text block;
In the case of non-conformity, a process of changing the size of the blank area by a control parameter, re-extracting the virtual separator for the unconforming integrated text element, and extracting a new integrated text element Repeat recursively,
2. The document image layout analysis program according to claim 1, further comprising executing processing. In the extracted line of the integrated text element, a character component that includes a connected component or a combination of the connected component included in a character string rectangle that surrounds the line and has a high recognition reliability is used as a character candidate.
Set a rectangular inspection area that includes the character candidate with one corner of the character string rectangle as the origin and the origin as the corner,
Determining the reading order of the character candidates under the condition that the rectangular inspection area does not include a reading order after the character candidates;
A new generation line generated by integrating the character candidates in order according to the reading order under the condition that the reading order after the character candidates to be integrated is not included, 3. The document image layout analysis program according to claim 2, wherein when there are a plurality of lines in the vertical direction of the line, the line is determined to be incompatible with the line. In the extracted line of the integrated text element, a character component that includes a connected component or a combination of the connected component included in a character string rectangle that surrounds the line and has a high recognition reliability is used as a character candidate.
Set a rectangular inspection area that includes the character candidate with one corner of the character string rectangle as the origin and the origin as the corner,
Determining the reading order of the character candidates under the condition that the rectangular inspection area does not include a reading order after the character candidates;
A new generation line generated by integrating the character candidates in order according to the reading order under the condition that the reading order after the character candidates to be integrated is not included, 3. The document image layout analysis program according to claim 2, wherein when there are a plurality of lines in the vertical direction of the line, the text block including the line is determined to be incompatible as a text block. 4.   Among the lines included in the extracted integrated text element, the same number of lines equal to or greater than a predetermined number of lines is greater than the average character spacing of the integrated text element estimated from the connected components in the integrated text element. 3. The document image layout analysis program according to claim 2, wherein, when intersecting with a blank area, the extracted integrated text element is determined to be incompatible as a text block.

Images