JPH10222612A - Document recognizing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize a document by the most suitable character recognizing method for every partial area in a document picture. SOLUTION: A character recognizing part 102 recognizes the character of the document picture 101 and outputs a character recognition result a103. A logical structure recognizing part 104 recognizes logical structure with respect to the character recognition result a103 decides that a character string in the recognition result corresponds to which element. Then, information of character areas of respective characters in the character recognition result (a) is referred to, a character string area occupied by the respective character strings made correspond to the element is derived and a logical structure recognition result 105 including at least the correspondent relation of the character string areas with the elements is outputted. In a control part 108, element-recognizing means correspondence knowledge 107 is referred to and proper recognizing means corresponding to the corresponding elements are assigned from a recognizing means managing part 109 as against the respective character string area in the logical structure recognition result 106. A recognizing means applying part 110 applys the recognizing means selected by the control part 108 as against the respective character string areas and generates a character recognition result b111.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for performing character recognition on document image information generated by means such as taking a paper document with a scanner.

[0002]

2. Description of the Related Art Research and development of character recognition technology for document image information has been actively carried out for a long time, and character recognition for printed documents can be recognized at a recognition rate of about 99% ( "Prospects of Character Recognition Technology" Kurosawa et al., Toshiba Review, 1992 Vol.47 No.2 p.
p.111-114: hereinafter referred to as "Document 1"). However, a character recognition system that recognizes important information such as legal documents and family register information has higher accuracy (for example, 9 characters).
(Approximately 9.9%), and many problems still remain to realize such highly accurate character recognition. One of these issues is that if a document image to be recognized contains various character typefaces (hereinafter referred to as “fonts”) and character types (character types such as alphanumeric characters, kanji, handwritten characters, etc.), specific fonts and character types The character string recognition rate is likely to decrease. " This is because there are advantages and disadvantages regarding fonts and character types depending on the recognition method and the character pattern dictionary used for recognition.

As a technique for solving this problem, for each partial area (hereinafter referred to as "field") composed of a character group of a specific font or character type on a document image, an appropriate character corresponding to the area is used. Methods of assigning recognition means have been studied. Three prior arts relating to this approach are shown below.

Japanese Patent Application Laid-Open No. S61-16378 describes that an appropriate character recognition means is assigned to each document, such as a form or a fixed form document, in which the position of the field is fixed. Conventional technology 1).

Japanese Patent Application Laid-Open No. Sho 61-290579 describes that field division is manually performed in advance and appropriate character recognition means is assigned to each field (prior art 2).

Japanese Patent Application Laid-Open No. 63-27990 discloses that when an image is divided in units of lines, a "line layout type and a recognition means for recognizing each layout line" are prepared in advance. It is described that an appropriate character recognition method is assigned with reference to "correspondence relation of the related art" (Prior Art 3).

[0007]

First, problems of the prior art will be described. In the prior art 1, since the method is for a document in which the position of a field is fixed, there is no effect on a type of document in which the physical position and the number of fields vary for each document.

In the technique of the prior art 2, since field division is performed manually, labor required in proportion to the number and size of documents also increases. Therefore, it is difficult to apply when the number and size of documents are large.

The technique of the prior art 3 is different from the techniques of the prior arts 1 and 2
This is a method that overcomes the problems described above. However, in practice, it is difficult to perform appropriate field division using only layout information for each row. For example, fields with different fonts may be mixed in one line. FIG. 2 is an example of a legal document image. The first three characters on the fifth line are in Gothic type, and the remaining characters are in Mincho type. At this time, it is desired to assign different recognition means for Gothic characters and Mincho characters, but this cannot be achieved by the method of the related art 3 in which the field is divided line by line. The fourth line in FIG. 2 is described in Gothic font, and the sixth line is described in Mincho font. Even if it is desired to assign different recognizing means to the two lines, if the two lines have the same layout (in this case, the indentation at the beginning of the line is the same), the same recognizing means will be assigned.

The problem to be solved by the present invention is to automatically perform field division for a document of a type in which the physical position and number of fields vary for each document, and to convert one line into a plurality of fields. The purpose of the present invention is to realize a function that can be divided so that different recognition means can be assigned depending on the contents even if the row layout is the same.

[0011]

According to the present invention, when performing field division of a document image, instead of paying attention to "line layout" as in the prior art 3, attention is paid to the following document properties. The field division is performed by paying attention to “elements of the logical structure of the document” (hereinafter, “elements”). (1) Fonts are often set for each element such as “title”, “chapter title”, and “article number”. (2) For elements such as “date” and “XX number”, the character types constituting the character string can be limited.

Dividing a line into a plurality of fields by performing field division focusing on elements;
Even if the row layout is the same, different recognition means can be assigned depending on the content.

The configuration of the present invention will be described below. The present invention provides a character recognizing means for extracting characters from image information of a document to be recognized and recognizing the extracted characters, and performing a logical structure recognition on the character recognition result to obtain an image corresponding to at least a partial character string. Relationship between the region between the partial region in the information, that is, the character string region, and the type of the logical element, that is, the logical structure element in the recognition target document, and the logical structure of the region (A region-
(a logic, B area-b logic), and a logical structure recognition means for generating a logical structure recognition result including at least a type of a logical structure element associated with each character string region in the logical structure recognition result. As one of the criteria, a character recognizing means suitable for the character string area is derived, and re-character recognizing means for performing character recognition again is provided.

The re-character recognizing means includes, for example, a recognizing means suitable for a type of a logical structure element prepared in advance and a characteristic of a character to appear in a character string area corresponding to each logical structure element. By referring to the binary relation between the character strings, an appropriate recognition means may be assigned to each character string area in the image information to be recognized, and character recognition may be performed again.

The character re-recognition means includes information indicating a type of a logical structure element created in advance and a type of a characteristic of a character to appear in a character string area corresponding to each of the logical structure elements. And a means for deriving a recognition means suitable for the characteristic from the type of the characteristic of the character, thereby allocating an appropriate recognition means to each character string region and re-character recognition. May be performed.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS This embodiment is an example in which field division is performed by focusing on elements, and a strong recognition means is assigned to a specific font for each field.

First, the handling of the recognition rate in this embodiment will be described. Generally, a recognition method that performs character recognition for all fonts (hereinafter, “multi-font method”)
It is known that a recognition method specialized in a target font (hereinafter referred to as “font specialized method”) has a higher recognition rate than a recognition method. In Reference 1, the recognition rate of the multi-font method (recognition of “omni font type kanji” in Reference 1) is 9
The recognition rate of the font-specific method (recognition of “single font type kanji” in Reference 1) is 99.7% or more, but the specific recognition rate differs depending on the individual recognition method. In the present embodiment, the degree of the recognition rate will be discussed using a qualitative property that the recognition rate of the font-specific method is higher than the recognition rate of the multi-font method. Further, in the example of character recognition in this embodiment, in order to clearly illustrate the effect of the present invention resulting from the difference between the two types of recognition rates, the recognition rate of the multi-font type is about 96%, and the recognition rate of the font specialization type is about 96%. Specific recognition result examples are shown assuming that the recognition rate is about 99%.

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of a document recognition method according to the present embodiment. Reference numeral 101 denotes data obtained by capturing a document image, that is, a paper document by a scanner or the like, and converting the document image into image information. The character recognition unit 102 receives the document image 101 as input, performs character recognition on a character area in the document image 101, and obtains a character recognition result a10
3 is output. In the character recognition unit 102, since font information for each character in the document image is not given, character recognition is performed in a uniform manner for the entire character area. This character recognition may be realized by a single recognition unit, or may be realized by combining a plurality of recognition units. The character recognition result a103 is information including at least the correspondence between the area occupied by each character on the document image (hereinafter, “character area”) and the character code information as a result of recognizing the character.

The logical structure recognizing unit 104 performs logical structure recognition on the character recognition result a103 and determines what kind of document logical structure (hereinafter, “element”) the character string in the recognition result corresponds to. I do. (For example, a certain character string is associated with an element “title” of the document, and another character string is associated with an element “chapter title.”) Then, the character of each character in the character recognition result a Referring to the area information, an area occupied by each character string associated with the element (hereinafter, “character string area”) is derived, and a logical structure recognition result 105 including at least the correspondence between the character string area and the element is obtained. Output.

The re-character recognition unit 106 refers to the logical structure recognition result 105 and the element-recognition unit correspondence knowledge 107 created in advance according to the type of the document, and recognizes a recognition unit suitable for each character string area. Is selected and character recognition is performed again, and a highly accurate character recognition result b111 is output.

The processing in the re-character recognition unit 106 will be described in more detail. The re-character recognition unit 106 includes a control unit 108, a recognition unit management unit 109, and a recognition unit application unit 110. The recognition unit management unit 109 manages a plurality of recognition units suitable for each font and character type. The control unit 108 refers to the element-recognition unit correspondence knowledge 107 and allocates an appropriate recognition unit corresponding to the corresponding element to each character string region in the logical structure recognition result 106. The recognizing means applying unit 110 applies the recognizing means selected by the control unit 108 to each character string area, and obtains a character recognition result b11
1 is generated.

Hereinafter, the processing of each component in FIG. 1 will be described in detail.

FIG. 2 shows an example of a document image 101 to be recognized. This is image information created by inputting a paper regulation document with a scanner, and is basically composed of Mincho printed characters, but some Gothic character strings are mixed. FIG. 3 shows a gothic character string and a Mincho character string in the document image of FIG. The part enclosed in a square is a Gothic character string,
Other parts are character strings in Mincho style. In other words, all the character strings of the line numbers 1, 2, 3, 4, 15, and 17, the “first rule” of the line number 5, and the “second rule” of the line number 14 are Gothic character strings. Is a character string in Mincho style. In particular, 5
In the first and fourteenth lines, a Gothic character string and a Mincho character string are mixed in one line. In addition, there is no difference in the line layout between the fourth and sixth lines (the indentation at the beginning of the line is the same)
However, the font used is different.

The character recognizing unit 102 performs character recognition on all the characters in the document image by a uniform method using the multi-font method. This is because font information for each character as shown in FIG. 3 is not given when character recognition is first performed. This character recognition may be realized by a single recognition unit, or may be realized by combining a plurality of recognition units. Such a character recognition technology for document image information is an existing technology. For example, a document “Document Recognition System Using Highly Accurate Character Extraction Processing Based on Character Structure Information” (Sun et al., IPSJ Transactions Vol. 33 N
o. 9 pp. 1083-1091 (1992)).

FIG. 4 shows an example of a character recognition result a103 obtained by recognizing the document image of FIG. 2 by the character recognition unit 102. As shown in the example of FIG. 4, the character recognition result a103 includes at least a “character area” representing an area where a character exists on the document image;
"Character code" which is the recognition result of the character in the character area
And the binary relation with FIG. 5 shows the notation of the character area in the example of FIG. Here, the coordinates of the upper left corner and the lower right corner of a rectangle circumscribing each character on the document image (a rectangle configured of four sides parallel to the x-axis or y-axis set for the document image) are shown. The set is used as information representing a character area. In the example of FIG. 5, the character area of the character "Akira"
“(X1, y1) − (x2, y2)”.

FIG. 6 shows an example in which the characters corresponding to the character codes in the character recognition result a103 are arranged so as to represent the original positions. In FIG. 6, characters surrounded by a circle are erroneously recognized characters. In this embodiment, as described at the beginning,
Assuming that the recognition rate of the multi-font method is about 96%, there are 12 misrecognized characters in 312 characters in FIG.

Next, the logical structure recognition unit 104 performs logical structure recognition on the character recognition result a103. The logical structure recognition means that a logical structure as shown in FIG. 7 is defined in advance according to the type of a document, and a partial character string of an input individual document is converted to each character. This is a process of analyzing what the column logically has in the document, that is, what element (hereinafter, “element”) in the structure shown in FIG. 7 corresponds to the character position. The analysis is performed using the information and the character code information of the recognition result.

FIG. 7 is a definition example of a logical structure set for the regulatory document of FIG. In this example, the entire document corresponds to the element "Ordinance" and the "Ordinance"
Shall have its own sub-elements of "Title", "Promulgation", "Main Rules" and "Supplementary Rules". The “promulgation” element further includes lower elements such as “promulgation date”, “regulation number”, and “promulgation sentence”. In this figure, the "?" Mark indicates that the element may or may not exist in the actual document, and the "+" mark indicates that the element can appear multiple times. . Also,"#
An element having the content “PCDATA” indicates that the content of the element is not the lower element but the character string data itself. Such an element is particularly called a “character string corresponding element”.

The logical structure recognition unit 104 analyzes which character string corresponding element in the structure of FIG. 7 corresponds to the partial character string in the character recognition result a103 as illustrated in FIG. Then, based on the arrangement of the element corresponding to the character string, a tree structure of the element having the “rule” element as a root as shown in FIG. 7 is generated. The technology of logical structure recognition for such a character recognition result is an existing technology,
For example, Japanese Patent Application No. 7-22301 filed earlier by the present applicant.
No. 7.

Incidentally, the logical structure recognition may fail due to an erroneously recognized character in the character recognition result. The character recognition performed again by the character re-recognition unit 106 is applied only to a document for which the logical structure has been successfully recognized. However, for example, in Japanese Unexamined Patent Publication No. Hei 7-257431, by repeating the procedure of “correcting the character that caused the failure of the logical structure recognition manually and performing the logical structure recognition again”, the erroneously recognized character is repeated. A method has been proposed for making the logical structure recognition successful even for the recognition result including the recognition result.

FIG. 8 shows an example of the logical structure recognition result 105 generated by the logical structure recognition unit 104. The logical structure recognition result includes a partial character string in the character recognition result a103 and an area occupied by the partial character string on the document image (character string area).
, And a character string corresponding element name assigned to the partial character string. Of the three items, the information used in the subsequent processing is information indicating a character string area,
It is two items with the corresponding element. Therefore, the logical structure recognition result 105 only needs to include at least the binary relation between the information indicating the character string area and the element name.

FIG. 9 shows a method of deriving a character string area and a notation.
Shown in As shown in FIG. 9A, the character string area is generated as a rectangle circumscribing a character area group of characters constituting the character string, and is represented by a set of upper left coordinates and lower right coordinates of the rectangle. I do. If the character string extends over a plurality of lines, a character string region is extracted for each line, and the entire character string region is expressed by a combination thereof. For example, in the example of FIG. 9B, a character string area corresponding to a certain element extends over two lines. In such a case, the rectangles circumscribing the character area group in each line are represented by a set of upper left coordinates and lower right coordinates, and they are separated by “,” and arranged to form a rectangle combination. Express.

Next, the processing of the character re-recognition unit 106 will be described in detail. FIG. 10 shows the element-recognition means correspondence knowledge 10.
7 is shown. This is information that is set in advance according to the type of the document to be recognized, and is used between an element that is an element of the logical structure and a recognition unit that is most suitable for recognizing a character string corresponding to the element. Is a set of binary relations. For example, in the example of FIG. 10, "Gothic type recognition means" is applied to the recognition of the character corresponding to the element "Title" (1001), and "Mincho type" is used to recognize the character corresponding to the element "Art. Application ”(1002)
That is set.

The control unit 108 refers to the element-recognition means correspondence knowledge 107 and allocates an appropriate recognition means corresponding to the corresponding element to each character string area in the logical structure recognition result 106. This is performed using “binary relation between character string area and element name” in the logical structure recognition result 106 and “binary relation between element name and recognition means” in the element-recognition means correspondence knowledge 107.
One of the character string regions generated as a result of the logical structure recognition is temporarily referred to as “region A”. First, an element name corresponding to the area A is obtained from the “binary relationship between the character string area and the element name” including the area A. Then, the "binary relation between the element name and the recognition means" including the element name is searched for in the element-recognition means correspondence knowledge 107. The content of the “recognition means” in the binary relation obtained as a result of the search is the area A
Is assigned as a recognition means for recognizing the.

For example, in the example of FIG. 11, the character string area "(780,0)-(1380,
50) "corresponds to the element name" title ". In order to assign the recognition means to this character string area, a binary relation including the element name "title" is searched from the element-recognition means correspondence knowledge. Then, the recognition means in the binary relation obtained as a result of the search, that is, the “recognition means for Gothic body” is used as the character string area “(780, 0) − (1
380, 50) ".

By applying this method to all character string regions, appropriate recognition means is assigned to each character string region. FIG. 12 shows an example of recognition means assigned to each character area in FIG. 8 as a result of this processing. In this example, for example, the first half ("(240,240)-(240)" of the character string area of 1202 and the character string area of 1203
0, 290) ") belong to the same fifth line in the document image of FIG. 2, but are assigned different recognition means. That is, a plurality of fields are assigned to one line.
The character area 1201 is set to the fourth line in FIG.
("(60,300)-(86
0, 350) ") correspond to the sixth row in FIG. 2, respectively, but these rows are assigned different recognition means even though they have the same layout (line indent).

In the recognizing means application unit 110, the control unit 108
Calls the recognition means selected from the recognition means management unit 109 and applies it to each character string area, and the character recognition result b11
1 is generated. For each character string region, character recognition is performed by a font specialization method according to the font of the region. Therefore, the recognition rate in each region can be expected to be higher than in the case of recognition by the multi-font method. Therefore, the recognition rate of the entire document is also the same as the character recognition result a1.
03 can be expected to be higher than the recognition rate. FIG. 13 shows an example of a character recognition result b111 obtained by performing character recognition by the character re-recognition unit 106 on the document image 101 illustrated in FIG.
Shown in In the present embodiment, as described at the beginning, it is assumed that the recognition rate of the font specialization method is about 99%. In FIG. 13, there are three misrecognized characters out of 312 characters.
Compared to the example of the character recognition result a103 in FIG. 6, the number of misrecognized characters is reduced.

An example of another document recognition method will be described below. This method differs from the above-described method only in the information referred to by the control unit 108 in the re-character recognition unit 106. FIG. 1 shows a configuration example of the character re-recognition unit 106 in this method.
It is shown in FIG.

Element-character feature type correspondence knowledge 140
1 is a set of binary relations between an element name of a character string corresponding element and a feature type in a document image of a character representing the content of the element. Character feature type-recognition means correspondence knowledge 1402 is a set of binary relations between the type of a character string feature and a recognition means suitable for recognizing a character of that type. This is an example for realizing means for deriving recognition means suitable for the feature. Here, the element-character feature type correspondence knowledge 140
1 needs to be changed according to the type of the target document, but the character feature type-recognition means correspondence knowledge 1402 is updated only when the recognition means managed by the recognition means storage unit 109 is changed. I just need.

FIG. 15 shows an example of processing in which the control unit 108 allocates appropriate recognition means to each character string area in the logical structure recognition result 105. In the logical structure recognition result 105, the element name “registration number” corresponds to the character string area “(20, 0)-(100, 20)”. At this time, a binary relation including the element name “registration number” is searched from the element-character feature type correspondence knowledge 1401 (1501). As a result, the character string area “(20,
It can be seen that the characters in (0)-(100, 20) "have the feature of" pattern B ". Furthermore, character feature type
A binary relation including the character feature type “pattern B” is searched from the recognition means correspondence knowledge 1402 (1502). As a result, the character string area "(20,0)-(100,2
Recognition means called “alphanumeric character recognition means” is assigned to “0)”.

By applying the above processing to all the character string regions, an appropriate recognition means is assigned to each character string region even in a configuration example in which the re-character recognition unit 106 is changed as shown in FIG. Then, the recognition unit application unit 110 executes the recognition unit assigned to each character string area, so that a character recognition result b110 with higher accuracy than the character recognition result a103 in FIG. 1 is obtained.

[0042]

As described above, according to the present invention, an appropriate recognition means is associated with each element, and this is applied to each character string area divided according to the logical structure recognition result. Field division for setting an area to which each recognition unit is assigned can be automatically performed.
In addition, since field division is performed focusing on elements, even if the physical position and number of fields fluctuate in each document, it is possible to cope with the change, and one line can be divided into a plurality of fields. Even if the layouts are the same, a function of assigning different recognition means depending on the contents can be realized.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a document recognition method according to an embodiment of the present invention (corresponding to claims 1 and 2).

FIG. 2 is a diagram illustrating an example of a document image.

FIG. 3 is a diagram clearly showing a distinction between a Gothic character string and a Mincho character string in the document shown in FIG. 2;

FIG. 4 is an example in which a recognition result of character recognition by a multi-font method is represented by a correspondence between a character code and a character area.

FIG. 5 is a diagram showing how to represent a character area.

FIG. 6 is a diagram illustrating erroneous character recognition in a recognition result example using the multi-font method.

FIG. 7 is a diagram illustrating an example of a definition of a logical structure set for the document illustrated in FIG. 1;

FIG. 8 is a diagram showing an example of a logical structure recognition result.

FIG. 9 is a diagram showing a method of deriving and expressing a character string area.

FIG. 10 is an example of element-recognition means correspondence knowledge.

FIG. 11 is an example of a process of assigning a recognition unit to a character string area according to an element name corresponding to the character string area.

FIG. 12 is an example of a result of assigning a recognition unit to a character string area.

FIG. 13 is a diagram illustrating an example of a recognition result of character recognition by a font specialization method.

FIG. 14 is a diagram in which the configuration of the character re-recognition unit is modified in order to make the configuration of FIG. 1 correspond to the first and third aspects.

FIG. 15 is an example of a process in which the control unit in the configuration of FIG. 14 assigns appropriate recognition means to each character string area in the logical structure recognition result.

Claims (3)

[Claims] 1. A character recognizing means for extracting a character from image information of a document to be recognized and recognizing the extracted character, and performing a logical structure recognition on the character recognition result to correspond at least to a partial character string. Logical structure recognizing means for generating a logical structure recognition result including a binary relation between a region in image information to be processed, ie, a character string region, and a type of a logical element, ie, a logical structure element, in the recognition target document; For each character string area in the recognition result, at least the type of the logical structure element associated in the logical structure recognition result is used as one of the criteria to derive a character recognition means suitable for the character string area, and A document recognizing device comprising: a character re-recognizing unit that performs character recognition. 2. The document recognition apparatus according to claim 1, wherein:
The re-character recognizing means includes at least a pre-created type of a logical structural element and a recognizing means suitable for a characteristic of a character to appear in a character string area corresponding to each of the various logical structural elements. A document recognition apparatus characterized in that an appropriate recognition means is assigned to each character string region in image information to be recognized by referring to a term relation, and character recognition is performed again. 3. A method according to claim 1, wherein said re-character recognizing means includes a pre-created type of a logical structural element;
A binary relation between information indicating the type of a character feature to appear in the character string area corresponding to each type of logical structure element,
Means for deriving recognition means suitable for the feature from the type of the character feature, thereby assigning an appropriate recognition means to each character string region and performing character recognition again. Recognition device.