JPH0944605A - Document image analyzer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely separate a character part or the like by performing the extraction and recognition of a linked black picture element group, providing a character code as its recognized candidate and using the shape and position of the already character-recognized black picture element group and the information on connection between words held in a grammer dictionary or the like. SOLUTION: A rectangle circumscribed to the continuous black picture element group in an image is found by a circumscribed rectangle generating means 101 and the degree of certainty is provided by a character recognizing means 102-1. The estimation and merge of separate characters are performed by a separate character candidate extracting means 104 and a separate character merging means 105, and the rectangle merged as separate characters is fixed as a character rectangle. Next, a character string is estimated as the set of character rectangles by a character string estimating means 107. Afterwards, a character block is estimated by a character block estimating means 108, and a non-character rectangle is estimated as the set of rectangles by a non-character rectangle estimating means 106. When there is any unfixed rectangle, that rectangle is estimated again as an object but when there is no rectangle, the estimated result of character structure is outputted by a character P block order estimating means 111.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document image analysis device for document images having various layouts.

[0002]

2. Description of the Related Art As a conventional method of analyzing the layout of a document image, for example, the Institute of Image Electronics Engineers, Vol. 17 N
o. As described in 5 pp. 258 to 266 "Trend of Document Image Processing Technology", a method of extracting a character line or a character block by utilizing various image characteristics has been devised.
Further, as shown in the same book, pp. 267-277 “Understanding Document Image Using Format Definition Language”, knowledge about the layout of a specific target document is stored in advance and applied to the input image. A method of working is also devised.

The former method is applicable to a wide variety of documents, but when the space between character lines or columns is narrow, the character line or character block is erroneously estimated, and the character is recognized in the character recognition processing executed thereafter. There are drawbacks such as not being able to recognize in the correct order. In addition, the characters existing in the figure cannot be subjected to the character recognition processing, or it is necessary to further execute processing such as character / graphic separation processing.

The latter method improves the extraction accuracy of character lines and character blocks, and can extract the logical structure of the title part, etc., but the type of the target document is within the range of knowledge stored in advance. Will be limited to.

FIG. 21 is a flow chart showing the operation of these conventional document image analysis apparatuses. (S210) The circumscribed rectangles of all the continuous black pixel blocks in the image are obtained. (S211) A character string is estimated as a set of one or more circumscribing rectangles. (S212) A character block is estimated as a set of one or more character strings. (S213) If the code information of each character is required, character recognition is performed for the character string or character block. (S214) The estimation result of the document structure is output, and the process ends.

As described above, in both of these methods and the other conventional document image analysis methods, the character recognition processing is performed after the extraction of the structure such as the character block is completed. It is only possible to use the information obtained from the above, and it is not possible to use the information as to whether each partial image is suitable as a character image or the information obtained from the content of the character such as the connection of the meaning of the sentence.

Also, as preprocessing of the character recognition device, for example, the operator needs to specify the range of the character blocks, the order between the character blocks, or the recognition target using the above technique. Attempts have been made to automatically extract the character blocks to be used, but they have not been put into practical use due to problems with accuracy.

[0008]

SUMMARY OF THE INVENTION The present invention accurately separates a character part and other parts from a document image having various layouts, and extracts a structure such as a character line and a character block with high accuracy. It is an object of the present invention to provide a document image analysis device capable of accurately estimating the order between character blocks.

It is another object of the present invention to provide a character recognition function for recognizing characters in a document image and outputting character codes in the correct order without an operator's instruction.

[0010]

The document image analysis apparatus of the present invention comprises the following means. (1) Pixel block extraction means for extracting a connected black pixel block in an image (2) Character recognition processing is performed on the connected black pixel block extracted by the pixel block extraction means, and at least one character code is output. 1 character recognition means (3) a grammar dictionary holding words and connection information between words (4) based on the character code determined by the character recognition means, words held by the grammar dictionary, and connection information between words, Character string estimating means for estimating the direction of a character string

(5) Separated character candidate extraction means for extracting separated character candidates. (2 ') Second character recognition means for performing character recognition processing on the separated character candidates to determine at least one character code. 6) It may be configured to further include a separated character integration unit that determines a separated character from the separated character candidates.

(7) Touch character candidate extraction means for extracting touch character candidates (8) Touch character dividing means for determining touch characters from the touch character candidates and dividing the touch characters into a plurality of images ( 2 ″) It is also possible to further include a third character recognition means for performing character recognition processing on the image obtained by the contact character division means and outputting at least one or more character codes.

In addition to the above configuration, the document image analysis apparatus of the present invention includes: (9) holding of a character string estimated by the character string estimating means, a character code output by the character recognizing means, and the grammar dictionary. It further has a character block estimation means which estimates a character block based on the word and the connection information between words.

Further, in addition to the above configuration, the document image analyzing apparatus of the present invention holds (10) the position and shape of the connected black pixel block, the character code output by the character recognizing means, and a grammar dictionary. It further has a non-character estimating means for estimating a non-character rectangle that is not a character based on the words and the connection information between the words.

Further, in addition to the configuration including (10), the document image analysis apparatus of the present invention includes (11) a character block estimated by the character block estimating means, the character string included in the character block, and Character block continuity estimating means for estimating the order of the character blocks based on the character code output by the character recognizing means, and the information held by the grammar dictionary and the connection information between the words (12) Estimating by the character block estimating means The character code included in the character block is determined from the order of the character block, the character string included in the character block, the character code output by the character recognition unit, and the character block estimated by the character block continuity estimation unit. A character code output means for outputting in the order is provided.

Further, in addition to the configuration including the above (10), the document image analyzing apparatus of the present invention includes (13) a character block estimated by the character block estimating means, each character string included in the character block, and A first logical structure analysis unit is provided that adds a logical structure to each block using the character code included in the character string and output by the character recognition unit and information in the grammar dictionary, and outputs the logical structure. be able to.

Further, in addition to the configuration including the above (10), the document image analyzing apparatus of the present invention includes (13 ') a character block estimated by the character block estimating means existing in the vicinity of the non-character rectangle. , Using each character string included in the character block, the character code included in the character string and output by the character recognition means, and information in the grammar dictionary,
Secondly, adding a logical structure to the non-character rectangle and outputting
Can be provided with the logical structure analysis means.

[0018]

The connected black pixel block is extracted from the input document image. Subsequently, each black pixel block is recognized, and at least one character code is obtained as the recognition candidate. Next, the character string is estimated using the shape and position of the black pixel block that has already undergone character recognition, the character code for the black pixel block, the words held in the grammar dictionary and the connection information between the words, and the like.

Further, the character block is estimated based on the estimated character string, the character code, and the information held by the grammar dictionary and the connection information between the words.

Further, the non-character rectangle can be accurately estimated based on the estimated character string, the character code, and the information held by the grammar dictionary and the connection information between the words.

Further, the result of character recognition can be used to determine the order of character blocks and to analyze the logical structure of a document image.

As described above, character recognition is performed prior to the estimation of the character string, the character block, the order of the character block, and the logical structure, so that the information can be obtained when estimating the order of the character string, the character block, the character block, and the logical structure. Can be used.

Further, by recognizing separated characters and contact characters to obtain more accurate character recognition, the estimation of non-character rectangles, character strings, character blocks, the order of character blocks, and the logical structure is more accurate. Becomes

[0024]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram showing the configuration of a document image analysis apparatus according to an embodiment of the present invention. A circumscribing rectangle generating means 101 for obtaining a circumscribing rectangle of a connected black pixel block in the input image,
A first character recognizing means 102-1 for performing character recognition on the circumscribing rectangle obtained by the circumscribing rectangle generating means 101, and a first character recognizing for a separated character candidate image obtained by a separated character candidate extracting means to be described later. 2 character recognition means 102-
2. Third character recognition means 102-3 for performing character recognition on an image obtained by the contact character division means described later, grammar dictionary 103 for holding words and connection information between words, separated character candidate extraction means 104, Separated character integration means 105, non-character rectangular designation means 106, character string designation means 107, character block estimation means 108, contact character candidate extraction means 10
9, a contact character division unit 110, a character block order estimation unit 111, a character code output unit 112, and a logical structure analysis unit 113. The means 104 to 113 can be appropriately combined and connected and deleted according to the application request. Further, although the character recognition means is described as being divided into three parts here, it is considered that these perform the same processing only with different inputs. Therefore, of course, these may be configured as one means and one software module. In the following, 102-1 and 10
2-2, 102-3 as "first", "second", "third"
No "will be omitted and all will be referred to as character recognition means.

First, FIG. 2 shows an example of the structure of the grammar dictionary 103.
Shown in Reference numeral 201 is an independent word table that stores the independent word together with its part of speech. Reference numeral 202 is an adjunct word table that stores adjunct words together with their pre-connection and post-connection. Reference numeral 203 is a utilization table that stores the utilization ending for each utilization type together with the subsequent connection. Reference numeral 204 is a connection table that stores the connection score between the post connection and the front connection. 0-, 1-, etc. in the table are pre-connection, -0,
-1 and the like represent the number of the headword to be connected later. For example, the connection score of 0-1 is 100. A high connection score indicates that both headwords are highly likely to be connected. In the following description, by using such a table, it is assumed that a word is detected from a sequence of character codes by referring to an independent word table, or a morphological analysis is performed to detect a phrase or the like. As the technique of morphological analysis, conventional techniques such as those shown on pages 140 to 142 of "Basics of Natural Language Analysis" of Sangyo Tosho Co., Ltd. can be used. For example, to detect a phrase,
Of the two adjacent headwords, the post-connection of the front word and the pre-connection of the rear word are obtained from the independent word table, the attached word table, and the utilization table. The obtained post-connection and pre-connection are checked with reference to the connection score to see if the two entry words can be connected.
If the connection score is other than 0, the two entry words are considered to be included in one clause.

FIG. 3 is a flow chart showing the operation of an embodiment of the document image analyzing apparatus according to the present invention.

(S30) The circumscribing rectangle generating means 101 obtains circumscribing rectangles for all continuous black pixel blocks in the image.

(S31) With respect to the generated circumscribed rectangle, the character recognition means 102-1 obtains a character code and a certainty factor indicating how closely the recognized image is recognized as a standard pattern. Using the certainty factor, it can be estimated whether the recognized image is a character image. As a result, a rectangle that is not rejected or a rectangle whose certainty factor is greater than a predetermined value is determined as a character rectangle. The obtained character code and certainty factor are stored in association with each rectangle. A specific example of the processing content will be described later with reference to FIGS. 4, 5, and 6.

(S32) The separated character candidate extraction means 104 and the separated character integration means 105 estimate and integrate the separated characters, and the rectangle integrated as the separated characters is determined as a character rectangle. Details of the processing content of this step will be described later with reference to FIG. 7.

(S33) The contact character candidate extracting means 109 and the contact character dividing means 110 estimate and divide the contact character, and determine the rectangle divided as the contact character as the character rectangle. Details of the processing content of this step will be described later with reference to FIGS. 9 and 10.

(S34) 1 by the character string estimating means 107
Estimate a string as a set of one or more character rectangles. An example of a detailed flowchart is shown in FIG. 12, and a specific process will be described later with reference to FIGS. 11 and 13.

(S35) The character block estimation means 108 estimates a character block as a set of one or more character strings. An example of a detailed flowchart is shown in FIG. 15, and a detailed description will be given later with reference to FIG.

(S36) The non-character rectangle estimating means 106 estimates a non-character rectangle as a set of one or more non-character rectangles.

(S37) It is checked whether there is an unfixed rectangle in both the character rectangle and the non-character rectangle.

(S38) If there is an undetermined rectangle, the separated character candidate extraction means 104 and the separated character integration means 105, or the contact character candidate extraction means 109 and the contact character division are targeted for the undetermined rectangle. Re-estimate by the means 110. At this time, the previously estimated character string, character block, positional relationship with a non-character rectangle, etc. are used for estimation, and the condition for confirming the character rectangle is changed, so it was not confirmed as the character rectangle in the previous estimation. A part of the rectangle may be fixed as the character rectangle. A detailed example of this re-estimation will be described later with reference to FIG. The process is repeated from the estimation of the character string in (S34), including the rectangle newly determined as the character rectangle.

(S39) If there is no undetermined rectangle, the character block order estimation means 111, the character code output means 112, the logical structure analysis means 113, etc. output the estimation results of the required character codes and document structures. Then the processing is completed.

FIG. 4 is a diagram showing an example of a circumscribed rectangle obtained by the circumscribed rectangle generation means 101 in (S30) of FIG. 401 is a part of the document image, and 411-420
Shows the generated circumscribed rectangle. Various methods have already been proposed as a method of detecting a connected black pixel block in an image and obtaining a circumscribing rectangle thereof, and any of them may be used. For example, a technique for detecting a connected black pixel block is shown on pages 115 to 116 of "Image Engineering" (published on August 25, 1989, first edition, first volume) by Corona.

FIG. 5 shows the character recognition means 102-1 and 102.
2 and 102-3 are examples of the configurations. The image input to these character recognition means is first processed by the image normalization unit 50.
The size is normalized by 1, the feature is extracted by the feature extraction unit 402, the distance between the standard feature corresponding to each character is calculated by the distance calculation unit 503, and the result output unit 504 sorts the distances in ascending order. The character code and the certainty factor corresponding to at least one character are output. For example, the reciprocal of the smallest distance can be used as the certainty factor.

FIG. 6 shows an example of the character code and the certainty factor obtained by the character recognizing means 102-1 for each of the circumscribing rectangles obtained by the circumscribing rectangle generating means 101. If the obtained certainty factor is smaller than a predetermined value, it means that the image in the rectangle is not similar to any character.
It can be inferred that it is a part of the separated characters or an image such as a figure that is not a character. Therefore, such a rectangle is rejected. Other rectangles, that is, rectangles having a certainty factor larger than a predetermined value are determined as character rectangles having the obtained character code.

FIG. 7 is a diagram showing an example in which the separated character candidate extracting means 104 extracts a candidate which may become an image corresponding to one character by integrating a plurality of adjacent circumscribing rectangles in (S32). is there. In this example, whether a rectangle obtained by integrating two or more rectangles with a certainty factor of less than 100 is within three times the width and height of a neighboring rectangle and does not overlap another circumscribed rectangle. Alternatively, it is assumed that the separated character candidates are extracted on the condition that they are completely included. In the example of FIG. 6, the rectangle obtained by integrating the circumscribing rectangles 413 and 414 satisfies the condition that the width and height of neighboring rectangles are each within three times, but since they overlap the rectangle 412, the separated characters are separated. Not a candidate. This condition is a condition for avoiding generation of unnecessary separated character candidates by using the assumption that the character rectangle does not overlap with other character rectangles or non-character rectangles. However, like the letters in the table,
Since it may be completely included in other non-character rectangles, those completely included in other circumscribed rectangles are extracted as separated character candidates. Further, when targeting a document image in which character rectangles may overlap with each other, such as italicized letters in English, the condition that they do not overlap other circumscribed rectangles need not be used. As shown in FIG. 7, the separated character candidates 711 are obtained from the circumscribing rectangles 412, 413, and 414 in FIG. Similarly, 417, 418 to 712 are 418, 4
19 to 713, 419 and 420 to 714 are obtained as the separated character candidates.

701 is a diagram showing an example of the character code and the certainty factor obtained by the character recognizing means 102-2 for each of the separated character candidates obtained by the separated character candidate extracting means 104 in this embodiment. is there.

In FIG. 8, the separated character integration means 105 estimates the actual separated characters from the separated character candidates,
It is a figure which shows the example which integrates several circumscribed rectangles.

As an example of the integration, the uniformity of the width and height of the rectangle, the uniformity of the distance between the rectangles, the degree of certainty corresponding to the rectangles, and the use of the grammar dictionary are used to correspond to the respective rectangles. The most appropriate combination of separation characters is determined based on the intra-word / inter-word connectivity between character codes.

In this example, the rectangle 412 in FIG. 4 and the rectangle 7 in FIG.
The widths of 11, 712 and 714 are almost equal, and the rectangle 4 in FIG.
The heights of 15, 416, 419, 420 and 714 of FIG. 7 are substantially equal. Also, the rectangles 412 or 711, 415,
The distances between the centers of 416, 712, 714 are approximately equal.
In addition, rectangles 412 or 711, 415, 416, 71
The character code corresponding to 2 corresponds to the order in the word of the noun “vector” stored in the grammar dictionary, and since the noun and the suffix “ka” are connected, the character code corresponding to the rectangle 714 is also Rectangle 412 or 711, 415, 41
The connectivity with the character codes corresponding to 6, 712 is satisfied.

Further, for example, rectangles 412, 413 and 41
4, the rectangles 712 and 713, and the rectangles 713 and 714 overlap with each other, and it is unlikely that they will be valid as characters at the same time. From these pieces of information, the separated character integration unit 105 estimates that the separated character candidates 711, 712, and 714 are separated characters and integrates them, and as a result, the separated character integrated rectangles 811 to 816 as illustrated in FIG. Is obtained.

If there is a portion for which sufficient information for determining an appropriate combination of separated characters cannot be obtained from a plurality of separated character candidates, the separated character integrating means 1 for that portion.
The process of 05 is suspended and the separated character candidates are stored. For example, after the process of the character string estimating unit 107 is performed, the size of the character in the character string and the direction of the character string are used,
By performing the processing by the separated character integration unit 105 again, the accuracy of integrating the separated characters can be further improved.

Next, the processing of (S33) in FIG. 3, that is, the operations of the contact character candidate extraction means 109 and the contact character division means 110 of FIG. 1 will be described. An image in which each of a plurality of rectangles obtained by dividing the circumscribed rectangle by the contact character candidate extraction means 109 from the position and shape of each circumscribed rectangle, the corresponding character code, the certainty factor, and the information in the grammar dictionary corresponds to one character. Alternatively, a contact character candidate that may be a separated character candidate is extracted.

In FIG. 9, the contact character candidate extraction means 109 is
It is a figure which shows the example which extracts a contact character candidate. 901 is a part of the document image, 911 to 917 are circumscribing rectangles obtained by the circumscribing rectangle generating means, and 902 is a character code and certainty factor obtained by the character recognizing means 102-1 for each of the circumscribing rectangles. As an example, the contact character candidate is a rectangle obtained by dividing a rectangle whose corresponding certainty factor is less than 100,
It is extracted under the condition that either the width or the height of the neighboring rectangle is 1.5 times or more. In this example, the circumscribed rectangle 917
Is obtained as a contact character candidate.

In FIG. 10, the contact character dividing means 110 estimates what is actually a contact character from among the contact character candidates, divides it into a plurality of rectangles, and for each of the obtained rectangles, It is a figure which shows the example which recognizes by the character recognition means 102-3. As a technique for estimating and dividing a position to be divided in a contact character, various techniques are known as disclosed in, for example, Japanese Patent Application Laid-Open No. 5-128308, and it is possible to use them. it can. 1
011 and 1012 are contact character candidate extraction means 109.
It is a rectangle obtained by dividing 917 obtained as a contact character candidate by. 1001 shows the character code and the certainty factor obtained by the character recognition means for each of the rectangles obtained by the division. The contact character dividing unit 110 is a rectangle 10
The widths of 11 and 1012 are almost equal to the neighboring rectangles, the certainty factor corresponding to the rectangles 1011 and 1012 is large, and the sequence of character codes corresponding to the rectangles 1011 and 1012 is the sequence of nouns "age" in the grammar dictionary. It is estimated that the contact character candidate 917 is actually a contact character, and the rectangles 1011 and 1012 are registered as character rectangles.

In addition, in the embodiment (S36), since the rectangle 411 has a small corresponding certainty factor, does not form a contact character candidate, and the width of the rectangle is larger than the average of the neighboring rectangles, the non-character The rectangle estimating unit 106 estimates that the rectangle is a non-character rectangle. These conditions are (S3
Every time the process of 6) is repeated, a strict condition is initially set, and the condition is gradually loosened, so that the estimation accuracy can be increased. For example, under the condition that the certainty factor is small, the threshold value is set to 10 in the first estimation, and the threshold value is increased by 10 in the subsequent estimation. The reason why the estimation accuracy is high is that the character rectangle is re-estimated for a rectangle that has not been determined as a character rectangle or a non-character rectangle, as will be described later with reference to FIG. This is because a rectangle that has a low certainty factor and is not determined as a character rectangle may be determined as a character rectangle. By making the initial condition strict, it is possible to reduce the error that a rectangle that is originally a character is mistakenly determined as a non-character rectangle.

Next, the processing of the character string estimating means 107 will be described. FIG. 11 is a diagram showing an example of a process in which the character string estimating unit 107 estimates a character string. 1101 is a part of the document image. Reference numeral 1102 indicates a rectangle after the separated characters are integrated. In this example, if the information on only the shape and position of the rectangle is used, it can be integrated in the vertical direction and the horizontal direction. However, the character string estimating unit 107 uses the character code corresponding to each rectangle to open the grammar dictionary. Estimate the direction of a character line by referring to it. That is, it can be presumed that the character string is written vertically by the prohibition rule, the position in the rectangular character string to which the character code of "," or "." Corresponds, the number of clauses appearing in the character code, and the like. Details of this estimation process will be described later with reference to FIGS. 12 and 13. As a result, a character string shown in 1103 is obtained.

FIG. 12 shows the character string estimating means 107 of the present invention.
3 is a flowchart showing the operation of one embodiment of FIG.
This is a detailed version of (S34). The outline of the flowchart is shown below, and the details of each step will be described later.

(S120) Character string candidates in the vertical and horizontal directions are generated for each rectangle defined as the character rectangle in the document image.

(S121) A score is calculated for each character string candidate.

(S122) It is checked whether there is a set of character string candidates that contradict each other. Here, a set of character string candidates that contradict each other is a plurality of character string candidates including the same character rectangle, a character string candidate that intersects with each other, and the like.

(S123) If there is a set of character string candidates that contradict each other, the candidate with a low score is deleted from the character string candidates. After that, the process returns to the inspection of the contradictory character string candidates, and is repeated until there is no set of the contradictory character string candidates.

(S124) If there is no set of character string candidates contradictory to each other, the character string candidates are output as the character string estimation result, and the process ends.

Conventional techniques can be used to generate the character string candidates in (S120). For example, the size and shape similarity of each rectangle, and the distance and positional relationship between the rectangles satisfying predetermined conditions are extracted as character string candidates. Although the conventional technique is used in this process, in the document image analysis device according to the embodiment of the present invention, by using the result of the character recognition means or the like, higher accuracy,
It is possible to generate character string candidates with high efficiency.

In the conventional document image analysis apparatus, character string candidates are generated for the circumscribed rectangle of a simple connected black image block. Therefore, for example, regarding a separated character such as “kawa” or a character in which a plurality of characters are in contact, a plurality of rectangles may correspond to one character, or one rectangle may correspond to a plurality of characters. . Therefore, even if an array of rectangles having similar sizes and shapes is generated as a character string candidate, there is a problem that the size and shape corresponding to one character cannot be accurately known. Therefore, if the generation conditions are made strict, the original character strings cannot be extracted as character string candidates, and if the generation conditions are made loose, a large number of extra character string candidates will be generated, resulting in poor accuracy and efficiency. .

On the other hand, in the document image analyzing apparatus which is an embodiment of the present invention, as shown in FIG. 3, the character recognizing means 102-1 in (S31) and the separated character integrating means 10 in (S32).
5, by the contact character dividing means 110 of (S33),
The character rectangle is fixed prior to the generation of the character string candidates in (S34). Then, the character string estimating means 107 generates a character string candidate for the rectangle determined as the character rectangle. Since each rectangle corresponds to one character, it is possible to accurately detect the arrangement of characters having similar sizes and shapes. Therefore, it becomes easy to set the generation condition of the character string candidates, the generation of extra character string candidates can be reduced, and the original character string can be surely included in the candidates.

FIG. 13 is a graph of (S121) of FIG.
It is a figure which shows an example of the score calculation with respect to a character string candidate.
The character string candidates in the vertical and horizontal directions for the character rectangle shown in 1102 of FIG. 11 are (1) to (22) in FIG. (1) to (12) are horizontal character string candidates, and (13) to (22) are vertical character string candidates. A character string candidate is indicated by a character code corresponding to a character rectangle included in each character string candidate.

The column S1 in the table is a score corresponding to the number of characters of each character string candidate. The greater the number of characters, the greater the score.

S2 is a score relating to prohibited characters.
If a character such as "." Or "" is at the beginning of the string, or if a character such as "" is at the end of the string,
Since it is unnatural as a character string, a negative score is given. In the case of this example, the character string candidates (8) and (11) correspond to this, and score -10 is given. The prohibited characters and prohibited rules may be stored in the character string estimating means or may be stored in the grammar dictionary.

S3 is a score relating to a character whose orientation and position are different depending on the length and width of the character string. For example, the position of the character "." Indicating a punctuation mark in the character string differs depending on the length and width of the character string. The direction of the character """indicating the bracket is different depending on the length and width of the character string.In the example of Fig. 13, horizontal character string candidates (3), (8), (9), (11).
Each contains a "." Or ",", but the corresponding character rectangles are located at the top of the horizontal character string candidates, which is unnatural for horizontal writing. , (3), (8), (9), and (11) are each given a score of -5.

S4 is a score related to the grammatical test. With respect to the arrangement of the character codes corresponding to the character rectangles included in each character string candidate, a higher score is given as the sentence becomes more natural according to the number of detected words. As an example of the criteria for being natural as a sentence, if there is a sequence of two or more characters that is a phrase in the sequence of character codes,
Give a score of 5. In the example of FIG. 13, “Moshi” of (4),
The "complex" of (17) corresponds to this. Furthermore, in the above phrase, if the length of the phrase is 5 characters or more, or if the Chinese character includes 3 characters or more, a score of 5 is added. FIG.
In the example of FIG. 3, "13" is "fighter" and "14" is "pilot's". If the beginning of the character code sequence is a sequence of two or more characters in which the front part of the phrase can be regarded as missing, a score of 5 is given. Similarly, a score of 5 is given if there is a sequence of two or more characters at the tail of the sequence of character codes that can be regarded as a lack of the rear part of the phrase.
In the example of FIG. 13, “13”, “in the ground”, and (16), “Ndy etc.” correspond to this.

T1 is the first of the scores from S1 to S4
Is the total value of.

S5 is a score relating to another character string candidate which is inconsistent with the candidate in each character string candidate. For example, the character string candidate (1) is the character string candidate (13), (1
4) and (16) to (22) include the same character rectangles and are inconsistent. The sum of the first total value T1 of the character string candidates (13), (14), (16) to (22) is multiplied by a weight of -0.1, and given as a score S5 for the character string candidate (1).

T2 is a second total value obtained by adding the first total value T1 and the score S5. This T2 is used as the final score of each character string candidate to delete the character string candidate.

In the example described with reference to FIGS. 11 and 13, as a result of repeating the process of sequentially deleting the character string candidates having a smaller score among the contradictory character string candidate sets, the character string candidates (13) are selected. (22) remains, and these are output as the character string estimation result. The distribution of the score weights and the threshold value described above may be obtained experimentally so that an appropriate estimation can be performed for the document image assumed as a target, and a value different from the numerical values described in this embodiment is used. May be.

FIG. 14 is a diagram showing an example in which the character block estimating means 108 estimates a character block. In this example, with respect to the arrangement 1103 of the character strings obtained as a result of the character string estimation means 107, the widths of the character lines and the heights of the upper sides, and the distances between the adjacent character lines in the horizontal direction are substantially constant. That the shape and size of the rectangle contained in the character line and the distance between vertically adjacent rectangles are almost constant in common for each line, and that "Handy, etc.""Russian"
It can be inferred that six lines can be integrated into one character block because there is a sequence of character codes such as “soaring up” and “stunning” that spans each line as a Japanese phrase. As a result, a character block as shown in 1401 is obtained.

FIG. 15 is a flow chart showing the operation of one embodiment of the character block estimating means in the present invention.

(S150) First, an inter-character string connection candidate, which is a set of two character strings that may be connected, is generated for a character string in a document image.

(S151) A score is calculated for each connection candidate.

(S152) The score of each connection candidate is compared with a predetermined threshold value, and the connection candidate whose score is less than the threshold value is deleted.

(S153) According to the remaining connection candidates, the character strings are integrated to generate a character block.

(S154) Finally, the generated character block is output as an estimation result, and the process is terminated.

This operation flow may be used in the conventional technique. The main feature of the character block estimation means 108 in one embodiment of the present invention is the method of calculating the score of each candidate. Next, the details of each step will be described, focusing on this score calculation method and comparing with the conventional method.

Conventional techniques can be used to generate the inter-character string connection candidates. For example, the following condition is used as a condition for generating a connection candidate between a set of two character strings.

(A) The two character strings have the same vertical or horizontal direction. (B1) In the case of a character string in the horizontal direction, there is a common portion of the respective diagonal shadings of the two character strings. (B2) In the case of the character string in the vertical direction, there is a common portion of the respective horizontal shadows of the two character strings. (C1) In the case of a horizontal character string, there is no other character string that satisfies all of the following conditions. (C1-1) It is located above one character string and below the other character string. (C1-2) The vertical shading has a common part and a common part of the respective vertical shadings of the two character strings. (C2) In the case of a vertical character string, there is no other character string that satisfies all of the following conditions. (C2-1) Position to the right of one character string and to the left of the other character string. (C2-2) The horizontal bevel has a common part and a common part of the respective bevels of the two character strings.

FIG. 16 shows an example of inter-character string connection candidates, examples of scores of connection candidates of the prior art and the embodiment of the present invention for the inter-character string connection candidates, and character block estimation according to the embodiment of the present invention. An example of the result is shown. In the figure, 1601 is a part of the document image, 1602 is a non-character rectangle, 1603, 160.
4, 1605, 1606 and 1607 are character strings (1), (2), (3), (4) and (5), respectively. In this example, the character string (1) is a non-character rectangular caption, and the character strings (2) to (5) are the texts, which are preferably different character blocks.

As the inter-character string connection candidates, the connection between the character strings (1) and (2) (abbreviated as (1)-(2) below), (2 )-(3),
(3)-(4) and (4)-(5) are obtained.

Reference numeral 1608 is an example of the score of the connection candidate according to the conventional technique.

The column S1 in the table is a score relating to the similarity in the size of the characters included in each of the two character strings.
The closer the size of the characters included in the two character strings is, the higher the score is given to the connection candidate. In this example,
Since all the characters included in the character strings (1) to (5) have substantially the same size, the score 10 is equally given to each connection candidate.

S2 is a score related to the space between character strings. In the case of a character string in the horizontal direction, the smaller the vertical interval between the character strings, the larger the score. In this example,
The respective character strings are arranged at the same intervals, and each connection candidate is given a score of 6 equally.

S3 is a score relating to the positions of both ends of the character string. In the case of a character string in the horizontal direction, the closer the positions of the two character strings in the horizontal direction to each other, the larger the score. In the case of this example, the connection candidates (1)-(2) and the connection candidates (3)-(4) are apart from each other in the positions of the beginning and end of the character string, and a score of -10 is given. In connection candidates (2)-(3), the head positions are almost equal, the tail positions are distant, and a score of -5 is given. In connection candidates (4)-(5), the positions of the tails are almost the same, the positions of the heads are distant, and a score of -2 is given.

T is the total value of the scores from S1 to S3. In an example of the conventional technique, this T is used for deleting a connection candidate. However, for example, when the threshold value is 0, there are no connection candidates to be deleted, and all the character strings (1) to (5) are integrated. Originally, the character string (1) is a caption attached to a non-character rectangle and should not be included in the same character block. On the other hand, the threshold value is set to 6 for the connection candidates (1)-(2) so that the character string (1) becomes an independent character block.
If a larger value, for example 7, is set, the connection candidates (1)-(2) and the connection candidates (3)-(4) are deleted, and the character strings (1), (2) and (3) are deleted. ), Three character blocks of character strings (4) and (5) are generated. This result is a character string that should be the same character block (2)
Since (5) becomes different, it is not preferable.

The problem with the prior art is that, as mentioned in 1608, the character string to be connected and the character string to not be connected only by the position and shape of the character string and the position and shape of the character included in the character string. Often, the columns cannot be accurately distinguished.

Reference numeral 1609 is an example of the score of the connection candidate by the character block estimating means 108 in the embodiment of the present invention.

S1 and S2 are the same as S1 and S2 described in the prior art 1508.

S3 is a score relating to the positions of both ends of the character string as in the prior art 1508. However, in the present invention, since character recognition is performed first, the score is assigned differently by using the result. For example, in the case of a character string in the horizontal direction, the closer the positions of the two character strings in the horizontal direction are, the larger the score. This is 15
Same as 08. Furthermore, when the character code corresponding to the character rectangle at the end of the character string is "." Or ".",
As an exception, a large score is given if the tail position is to the left of the other connected character string. In the case of this example, the character code of the character string (3) corresponding to the tail character rectangle is “.”, And the tail position of the character string (3) is left of the tail ends of the character strings (2) and (4). Therefore, for connection candidates (2)-(3) and connection candidates (3)-(4), 1
At 508, a score of 0 is given, which is greater than the score given in the prior art.

S4 is a score relating to grammatical continuity between character strings. The character codes corresponding to the character rectangles included in the two character strings are grammatically tested, and the higher the grammatical continuity, the greater the score. As an example, morphological analysis is performed on the character codes corresponding to the character rectangles included in the two character strings corresponding to the connection candidates, and a score of 20 is given when a word is detected across the character strings. In the case of the example in FIG. 16, connection candidates (2)-(3),
"Meaning" and "level" in (4)-(5) correspond to this. In addition, when a phrase is detected across character strings, but a word is not detected across character strings, a score of 10 is given. In the example of FIG. 16, there is no corresponding one, but for example, a connection "... meaning"-"without" ... "corresponds to this. Also, if the break of the phrase and the connection position of the character string are exactly the same, a score of 5 is given. In the case of the example in FIG. 16, the connection "(cannot) .-" There ... "in the connection candidates (3)-(4) corresponds to this. Furthermore, if it is not possible to analyze or an unknown word is detected at the connection position of the character string, score -1
0 is given. In the case of the example in FIG. 16, connection candidate (1)-
In (2), the unknown word "ru" is detected by the analysis of "... classify ...", which corresponds to this case.

T is the total value of the scores from S1 to S3. By setting the threshold value to 0, connection candidate (1)-
If (2) is deleted and the character strings are integrated according to the remaining connections, character blocks 1610 and 1611 are obtained. The distribution of score weights and threshold values described above are
It suffices to experimentally obtain it so that an appropriate estimation can be made for the document image assumed as the target, and a value different from the numerical value described in the present embodiment may be used.

FIG. 17 shows the character block order estimating means 11
FIG. 1 is a diagram showing an example of estimating the order of character blocks. 1701 is a part of the document image. 1711-1
Reference numeral 719 is an example of a non-character rectangle and a character block obtained as a result of the processing by the non-character rectangle estimating means 106 and the character block estimating means 108. In this example, as indicated by 1702, 1712, 1715, and 1719 are non-character rectangles,
1711, 1713, 1714, 1716, 1717,
1718 is estimated as a vertically written character block.

As an example of the rule used by the character block order estimating means, a character block in which the average width / height of the constituent characters is more than twice the average area of the character rectangles forming the other character blocks is Order first and then
The horizontal character block that is adjacent to the top or bottom of the non-character rectangle, or the vertical character block that is adjacent to the left or right of the non-character rectangle. Those with different averages are captions attached to figures etc. and the order is the last, and other character blocks are the text, top to bottom, right to left for vertical writing, left to right for horizontal writing, The rule is to arrange from top to bottom. According to this rule, the order of the character blocks 1713 is determined first. For the remaining character blocks, 1717,
1716, 1711, 1718, 1714 and 1717, 1716, 1718, 1711, 171
There are two possibilities in the order of four.

Next, the character code corresponding to each character rectangle included in each character block is checked. For the evaluation of grammatical continuity between character blocks, the same technique as the above-described evaluation of grammatical continuity between character strings can be used. That is, in the example of FIG. 17, the connection from 1716 to 1711 and 171
8 to 1714 for score 20, 1717 to 1716 for connection and 1711 to 1718 for score 5, 1716 to 1718 for connection, 1718 to 1711 for connection and 1711 to 1714 for connection Is given a score of -10. 1717, 1716, 1
In the arrangement of the character blocks 711, 1718, 1714, the total score is 50, 1717, 1716, 1
In the arrangement of the character blocks 718, 1711, and 1714, the total score is -25. From this,
The order 1711, 1718, 1717, 1716, 1715 is presumed to be the correct order, and finally 1
712, 1711, 1718, 1717, 1716, 1
The sequence 715, 1714 is obtained.

FIG. 18 shows an example of a character code output result by the character code output means 112.

FIG. 19 shows an example of the re-estimation of the character rectangle performed in step S38 in FIG. Reference numeral 1901 denotes a part of the document image, and 1902 shows the estimation result of the character rectangle and the character string before the first execution of step S38 in FIG. In 1902, the rectangle 1911 corresponding to the character "super" is the step S in FIG.
As a result of character recognition in 31, a certainty factor “82” is obtained as shown in 2103, but since the certainty factor is not sufficiently large, it is not decided as a character rectangle.
In the re-estimation performed in step S38, for example,
When an undetermined rectangle satisfies the following conditions,
It is assumed that 20 is added to the certainty factor corresponding to the rectangle. (1) It is between two vertical character strings or two horizontal character strings. (2) The center lines of the two character strings substantially coincide with each other. (3) The character spacing in the two character strings is almost equal. (4) The average of the sizes of the characters in the two character strings is almost equal. (5) Each of the intervals between the undetermined rectangle and the most recent character rectangle in each character string is approximately equal to the character interval in both character strings. (6) The size of the undetermined rectangle is almost equal to the average size of the characters in both character strings.

In the example of FIG. 19, since the rectangle 1911 satisfies these conditions, 20 is added to the certainty factor to give “102”, as indicated by 1904. As a result, rectangle 1
911 is determined as a character rectangle. After this, the processing from step S34 in FIG. 3 is executed again, and as shown at 1905, the character rectangle and the character string are correctly estimated.

Next, the logical structure analysis means 113 will be described. Various methods for analyzing a logical structure from a layout structure have been proposed, for example, the Institute of Image Electronics Engineers, Vol.
17 No. 5 pp.267-277 "Understanding document image using format definition language" can be used.

In the present invention, character recognition is performed first, the block is estimated by using the result of the character recognition, and when the logical structure is analyzed, the character blocks included in each character block, which is one of the layout elements, are analyzed. Since the character code of and the order within the block are clarified, the feature is that the logical structure can be determined more accurately by using the grammatical information or the information about the word.

In this example, an estimation method similar to that described in the description of the embodiment of the character block order estimating means is used.
It is estimated that the blocks 1711, 1718, and 1717 are text areas that are continuous in this order, and can be estimated to be blocks that form one part of a large paragraph.

As a result of estimation by the character block order estimating means,
A logical structure analysis as shown by 2001 in FIG. 20 is obtained.

As another example of estimating the logical structure, for example, when a logical block in which an author's name is described is determined, it is checked whether or not there is a word used for a personal name in a character block. The accuracy of judgment can be improved. Further, when determining a logical block in which a figure heading attached to a figure is described, a positional relationship such that the figure block is in the vicinity of a non-character rectangle within a certain threshold value, or “figure”, “Figure. By checking whether or not there is a character sequence such as "," the logical structure of the non-character rectangle is clarified, and the accuracy of determination can be improved.

[0104]

In the document image analyzing apparatus according to the present invention, character recognition is first performed on each rectangle and the character code obtained as the recognition result is used. Further, the grammar dictionary is used and the arrangement of character codes is used. As a result, the structure of a character line, a character block, or the like is estimated, so that the analysis accuracy can be improved and a document image having a more complicated structure can be analyzed.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an embodiment of a document image analysis apparatus according to the present invention.

FIG. 2 is a diagram showing an example of a grammar dictionary.

FIG. 3 is a flowchart showing an example of the operation of the document image analysis apparatus according to the exemplary embodiment of the present invention.

FIG. 4 is a diagram showing an example of a circumscribed rectangle obtained by a circumscribed rectangle generation means 101.

FIG. 5: Character recognition means 102-1 and 1 in this embodiment
It is a figure which shows an example of a structure of 02-2, 102-3.

FIG. 6 is a diagram showing an example of a character code and a certainty factor obtained by the character recognizing means for each circumscribing rectangle obtained by the circumscribing rectangle generating means 101.

FIG. 7 shows examples of separated character candidates obtained by the separated character candidate extraction unit 104, and examples of character codes and certainty factors obtained by the character recognition unit 102-2 for each of the obtained separated character candidates. It is a figure.

FIG. 8 is a diagram showing an example in which separated character integration means 105 determines a separated character from the separated character candidates and integrates a plurality of circumscribing rectangles.

9 is a diagram showing an example of an image including contact character candidates obtained by contact character candidate extraction means 109. FIG.

FIG. 10 is a diagram showing an example in which a contact character dividing unit 110 determines a contact character from contact character candidates and divides it into a plurality of circumscribing rectangles.

FIG. 11 is a diagram showing an example of estimation of a character string by a character string estimating means 107.

FIG. 12 is a detailed flowchart of (S34) of FIG. 3, showing an example of the operation of the character string estimating unit 107.

FIG. 13 is a diagram showing an example of a score for estimating a character string in the character string estimating means 107.

FIG. 14 is a diagram showing an example of estimation of a character block by a character block estimation means 108.

FIG. 15 is a flowchart showing an example of the operation of the character block estimating unit, which is a detailed version of (S35) in FIG.

FIG. 16 is a diagram showing an example of a score used for estimating a character block in the conventional technique and the character block estimating means 108 of the present invention.

FIG. 17 is a diagram showing an example of estimation of the order of character blocks by the character block order estimation means 111.

FIG. 18 is an example of a character code output result by the character code output unit 112.

FIG. 19 is a diagram showing an example of re-estimation of the character rectangle in (S38) of FIG.

20 is a diagram showing an example in which a logical structure is added to each character block and non-character rectangle by the logical structure analysis unit 113. FIG.

FIG. 21 is a flowchart showing an example of the operation of the document image analysis apparatus in the related art.

[Explanation of symbols]

101: circumscribed rectangle generating means; 102-1 and 102-
2, 102-3: character recognition means; 103: grammar dictionary;
104: separated character candidate extracting means; 105: separated character integrating means; 106: non-character rectangle estimating means; 107:
Character string estimating means; 108: Character block estimating means; 1
09: contact character candidate extracting means; 110: contact character dividing means; 111: character block order estimating means; 11
2: Character code output means; 113: Logical structure analysis means; 201: Independent word table; 202: Adjunct word table; 20
3: Utilization table; 204: Connection table; 401: Part of document image; 411-420: circumscribed rectangle; 501: image normalization unit; 502: feature extraction unit; 503: distance calculation unit; 504: result output unit; 601 : 701: Character code and certainty factor; 701: Character code and certainty factor for separated character candidate; 711-714: Separated character candidate; 811-8
16: Rectangle after separation character integration; 901: Part of document image; 902: Character code and certainty factor; 911 to 91
7: circumscribed rectangle; 1001: character code and certainty factor; 10
11, 1012: Rectangle obtained by dividing contact character candidates; 1101: Part of document image; 1102: Character rectangle; 1103: Character line estimation result; 1401: Character block estimation result; 1601: Part of document image; 1
602: Non-character rectangle; 1603 to 1607: Character string;
1608: example of score of prior art; 1609: example of score of the present invention; 1610, 1611: character block; 1701: part of document image; 1702: character block estimation result and non-character rectangle estimation result: 171
1, 1712, 1714 to 1718: Character block:
1713: Non-character rectangle; 1801: Character code output result; 1901: Part of character image; 1902, 190
3: Character rectangle and character string estimation result; 1903, 19
04: Character code and certainty factor; 1911: Partial enlarged view of character rectangle; 2001: Logical structure estimation result.

Claims (8)

[Claims] 1. A pixel block extracting means for extracting a connected black pixel block in an image, and character recognition processing for the connected black pixel block extracted by the pixel block extracting means to determine at least one character code. A first character recognition unit, a grammar dictionary holding a word and connection information between words; a character code determined by the character recognition unit; a character stored in the grammar dictionary; A document image analyzing apparatus, comprising: a character string estimating unit that estimates a direction of a column. 2. The document image analyzing apparatus according to claim 1, wherein the separated character candidate extracting unit extracts a separated character candidate having a possibility of integrating the plurality of connected black pixel blocks into one character. And character recognition processing is performed on the separated character candidate,
A document image analysis apparatus, comprising: a second character recognition means for determining at least one character code, and a separated character integration means for determining a separated character from the separated character candidates. 3. The document image analyzing apparatus according to claim 1, wherein one concatenated black pixel block is divided to extract contact character candidates having a possibility of forming a plurality of characters or a part of the characters. A contact character candidate extracting unit, a contact character dividing unit that determines a contact character from the contact character candidate, and divides the contact character into a plurality of images; and a character recognition process for the image obtained by the contact character dividing unit. And a document image analysis device characterized by third character recognition means for outputting at least one or more character codes. 4. The document image analysis apparatus according to claim 1, wherein at least one character code output from said first character recognition means for said connected black pixel block and held in a grammar dictionary. The document image analyzing apparatus further comprising a non-character estimating unit that estimates a non-character rectangle that is not a character based on the selected word and connection information between words. 5. The document image analyzing apparatus according to claim 1, further comprising: a character string estimated by the character string estimating means, and output by the first, second or third character recognizing means. A document image analyzing apparatus comprising: a character block estimating means for estimating a character block as a set of character strings based on a character code and a word held in the grammar dictionary and connection information between the words. 6. The document image analyzing apparatus according to claim 5, further comprising: the character block estimated by the character block estimating means, the character string included in the character block, and the first, second, or third character block. The character block order estimating means for estimating the order of the character blocks based on the character code output from the character recognizing means of No. 3, the word held in the grammar dictionary, and the connection information between the words, and the estimating of the character block estimating means From the character block, the character string included in the character block, the character code output by the first, second, or third character recognition means, and the order of the character blocks estimated by the character block order estimation means, A document image analysis apparatus, comprising: a character code output unit that outputs the character codes included in the character block in the order. 7. The document image analyzing apparatus according to claim 5, wherein the character block estimated by the character block estimating means, each character string included in the character block, and the character string included in the character string are included in the character block. Based on the character code output by the first, second or third character recognition means and the words in the grammar dictionary and the connection information between the words, a logical structure is added to the character block and the first character block is output. A document image analysis apparatus comprising a logical structure analysis means. 8. The document image analyzing apparatus according to claim 5, wherein, with respect to the non-character rectangle, a character block estimated by the character block estimating means existing in the vicinity of the non-character rectangle and each character string included in the character block. , And a logical code in the non-character rectangle based on a character code included in the character string and output by the first, second or third character recognition means, and a word in the grammar dictionary and connection information between the words. And a second logical structure analysis means for outputting a specific structure.