JP2993533B2 - Information processing device and character recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus capable of judging an order relation between character blocks from a document image composed of a plurality of character blocks, and a character recognition apparatus having the information processing apparatus. It concerns the device.

[0002]

2. Description of the Related Art If a machine can correctly obtain information of a document from a document image on a medium, an information processing apparatus having various functions, such as a filing apparatus for storing a document image, a character recognition apparatus, and a recognition apparatus. It is possible to construct a device that outputs a recognized character by voice, a device that translates a recognized character, and the like. However, in the document image, for example, as shown in FIG. 3, ruled lines or blanks (blank in the illustrated example) and a plurality of character blocks (FIG. 3 Then, there are many document images composed of 31 to 34), and in recognition processing of such a document image, unlike processing of a document image having a known format,
Special processing of extracting character blocks and determining the order relation between the character blocks is required.

Therefore, as a conventional device capable of performing this,
For example, Document 1: IEICE Technical Report PRU8
There has been an automatic reader for unknown format documents disclosed in JP-A-6-33. In the device disclosed in this document, document components such as a heading character block and a body character block are extracted from a document of unknown format using marginal distribution features, line density features, and circumscribed rectangle features. Each character is extracted from the character block, and feature extraction is performed on each character to recognize each character.

[0004] At this time, the order relation of each character block is described in, for example, Reference 2: Transactions of the Institute of Electronics, Information and Communication Engineers (198).
6.3) VOL. J69-D, no. 3, P404, the position evaluation h of each character block in the document is calculated according to the following equation (1), and the h is generally set in ascending order.

H = Xc + ε · Yc (1)

Here, Xc is the X coordinate of the upper left corner of each character block, Yc is the Y coordinate of the upper left corner of each character block, and ε is a constant of about 0.1.

[0007]

However, in the automatic reading apparatus disclosed in the above-mentioned document, in the case of the document image shown in FIG. 3, the normal order relation of the character blocks 31 to 34 is 31 → Even in the case of 33 → 32 → 34, 31 → 32 to follow the position evaluation value h.
There is a problem that it is determined that the order relationship is → 33 → 34, and it is not possible to determine the exact order relationship between character blocks having different typefaces.

Therefore, when this method is applied to a filing apparatus for storing document images, since the data cannot be stored in a correct order, there arises a problem that the stored document image data cannot be used efficiently. Further, when this method is applied to a character recognition device, character recognition cannot be performed in the correct order, so that the character recognition device cannot be put to practical use. Further, in the character recognition device, in order to obtain an accurate recognition result, after the recognition processing for each character, some post-processing, for example, linguistic processing using linguistic information may be performed. In most cases, processing is performed on the arrangement of characters. If recognition is not performed in the correct order as described above, the post-processing will not function at all, and the recognition accuracy will be significantly reduced. Further, in order to prevent this or to correct the result, an order instructing operation or an correcting operation by an operator is necessary. However, this operation is complicated, so that the operability is deteriorated, and ultimately a practically effective one is required. There is a problem that a high-performance character recognition device cannot be realized.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an information processing apparatus capable of accurately determining the order relation between character blocks.
And a character recognition device having the same.

[0010]

In order to achieve the above object, an information processing apparatus according to the present invention extracts a character block from a document image and, based on a positional relationship between the character blocks, a provisional order relation between the character blocks. Character extraction unit, a character extraction unit that extracts a character pattern from the extracted character block, a dictionary unit that has a plurality of types of dictionaries classified by character attributes, and a character extracted by the character extraction unit A character recognition unit that compares a pattern with the dictionary and outputs a dictionary name of the candidate character together with a candidate character name for each of the character blocks, and a character block attribute indicating what attribute the character block is composed of. A block feature extraction unit for extracting a character block attribute feature for determination based on a dictionary name of the candidate character; Characterized by comprising a determination unit for determining a new sequence relationship based on the attributes of the character blocks from the order relation of the temporary between lock.

According to another aspect of the present invention, there is provided a character recognition apparatus comprising: an image input unit for inputting a document image; an information processing apparatus according to claim 1; A recognition result storage unit for storing, and a block selection unit for outputting a recognition result from the recognition result storage unit based on a determination result of an order relation of a character block in a determination unit in the information processing device. I do.

[0012]

According to the information processing apparatus of the present invention, the provisional order relation between character blocks is determined based on the positional relation between the character blocks extracted from the document image. In addition, by comparing and recognizing a character pattern cut out from a character block with a dictionary prepared in advance and classified according to the attribute of a character such as a font, a dictionary name of the candidate character is obtained along with the candidate character name. can get. A character block for determining what attribute each character block is composed of based on the dictionary name of the candidate character, for example, the dictionary name most frequently referred to among the dictionary names of the first candidate character Attribute features are extracted. Based on the character block attribute characteristics, a new order relation in which the continuity of the attribute between the character blocks is preserved is obtained from the temporary order relation between the character blocks. Therefore, even when extracting and outputting a character block from a document having no regular layout of each character block,
Each block can be output in the correct order based on the attribute of the character.

Further, when the information processing apparatus of the present invention is applied to a character recognition apparatus, character recognition can be performed on character blocks output in the correct order as described above. Character recognition of a document can be performed according to the flow of the document.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an information processing apparatus and a character recognition apparatus according to the present invention will be described below with reference to FIGS.

FIG. 1 is a block diagram showing the configuration of a document image filing apparatus 100 using an information processing apparatus 120 according to the present invention. This document image filing apparatus 100
Image input unit 110, information processing device 120 of the present invention, block selection unit 130, image storage unit 140, and output terminal 1
50 is provided. The information processing device 120 includes a character block extracting unit 121, a character segmenting unit 122, a character recognizing unit 123, a dictionary unit 126, a block feature extracting unit 124,
A determination unit 125 is provided. The dictionary unit 126 includes a first dictionary 126a and a second dictionary 126b that store dictionary matrices having different typefaces.

FIG. 2 is a block diagram showing the structure of the character recognizing unit 123. The sub pattern extracting unit 201 and the feature extracting unit 2
02, a collating unit 203 is provided.

FIG. 3 is a diagram showing an example of a document image.
The document image 30 is composed of character blocks 31 and 33 of a Roman typeface and character blocks 32 and 34 of a sans serif typeface separated by a blank portion.

FIG. 4 is a diagram for explaining the first dictionary 126a and the second dictionary 126b in the dictionary unit 126. In this embodiment, for example, the first dictionary uses Roman typeface as a standard pattern. The second dictionary is a dictionary that uses sans-serif typefaces as standard patterns.

FIG. 5 is a diagram for explaining the block feature extracting unit 124. FIG. 5A shows a character string in the first line of the character block 31, a first candidate character, and a dictionary of a dictionary matrix from which the candidate character is obtained. Hereafter, (B), (C), and (D) represent the character string of the first line of the character blocks 32, 33, and 34, the first candidate character, and the dictionary number of the dictionary matrix, respectively.

(First Embodiment) The operation of each component of the document image filing apparatus using the information processing apparatus according to the first embodiment of the present invention will be described in detail with reference to FIGS. explain.

The image input section 110 can be composed of, for example, a conventionally known image sensor or the like, and an optical signal from a document image 30 such as that shown in FIG. 3 in which characters, figures, and the like (hereinafter, referred to as characters) are described. S is converted into a binary digital signal in which the character line portion is represented by black bits and the background portion is represented by white bits (this is referred to as input image data). Output to the unit 121.

The character block extraction unit 121 of the information processing apparatus 120 extracts data (hereinafter, referred to as character block data) corresponding to a character block of a document image from input image data input from the image input unit 110, and The data is output to the extraction unit 122 and the block selection unit 130, respectively. For example, in the case of the document image 30 shown in FIG.
Areas 31 to 34 in which character strings are regularly arranged densely and separated from the surroundings by spaces or the like (each of these areas becomes a character block) are extracted.

Here, the extraction of the character block data can be performed by various conventionally known methods. In this embodiment, the extraction is performed by the following method. First, the input image data is scanned in the vertical direction (a direction corresponding to the direction indicated by y in FIG. 3) to create a peripheral distribution. Next, in this peripheral distribution, the number of black bits changes from 0 to 1 or more from the scanning line.
From the above, an area defined by the scanning line immediately before changing to 0 is detected as one block in the column direction.

Next, the input image data portion corresponding to one block in the column direction is scanned in the horizontal direction (a direction corresponding to the direction indicated by x in FIG. 3), and a black image is formed in the peripheral distribution in the horizontal direction. If the width of the area defined by the scanning line where the number of bits changes from 0 to 1 or more to the scanning line immediately before changing from 1 or more to 0 is larger than the threshold value TH, this area and the aforementioned block in the column direction are used. The data of the overlapping part with is extracted as character block data. The threshold value TH is a parameter determined by the size of the character, the font, and the like. In this embodiment, TH = 140.

The output order of the extracted character blocks (this order relationship is referred to as a tentative order relationship) is determined by the equation (2) using the position information of each character block in the document image of each character block. The position evaluation value E _i (where i is a character block number and is, for example, 31 to 34 in FIG. 3) is obtained and output in ascending order of E _i .

E _i = y _ei + f · x _ei (2)

In the equation (2), x _ei and y _ei are the x and y coordinates of the upper left corner of the rectangle circumscribing the character block, f is a predetermined constant, and is 10 in this embodiment.
It is. The x and y coordinates are based on the x and y coordinate system shown in FIG. Therefore, the output order of each character block of the document image shown in FIG. 3 is 31 → 32 → 33 → 34.

The character cutout unit 122 cuts out a character line constituting the character block from the character block data input from the character block extraction unit 121, and further cuts out a character pattern for each character from this character line. The extracted character pattern is output to the sub-pattern extraction unit 201 (see FIG. 2) in the character recognition unit 123 for each character block.

Here, the character line is cut out by sequentially scanning the character block data input from the character block extracting unit 121 with the character line direction as the main scanning direction and the character string direction as the sub scanning direction. Create a distribution of pixels,
This is performed by detecting, as character line data, a portion from a position where the distribution of black pixels changes from “0” to “1” or more to a position immediately before a position where the distribution of black pixels changes from “1” or more to “0”.
The character pattern is cut out from the character line by scanning the cut-out character line data with the main scanning direction in the character string direction and the sub-scanning direction in the character line direction to create a distribution of black pixels. This is performed by sequentially cutting out from the position where the distribution of black pixels changes from “0” to “1” or more to the position immediately before changing from “1” or more to “0” as character pattern data.

In the sub-pattern extracting unit 201 shown in FIG. 2, the character pattern input from the character extracting unit 122 is scanned in a plurality of directions, and a predetermined specific value h (h = 5 in this embodiment) on each scanning line. ) Detecting a continuous black pixel row as described above and extracting the continuous black pixel row as a black pixel component of a sub-pattern extracts a sub-pattern for each scanning direction from a character pattern, and sequentially extracts a feature of the sub-pattern. Output to the unit 202. The scanning direction is, in the present embodiment,
Direction (vertical direction) and parallel direction (horizontal direction) perpendicular to the character line direction (hereinafter, X-axis direction), 45 ° counterclockwise direction from the X axis (diagonal left direction), and 45 ° time direction (Left diagonal direction), and character patterns are scanned in each of these directions to extract four sub-patterns for each direction.

For example, in the extraction of a vertical sub-pattern, the vertical direction is set as the main scanning direction, character pattern data is scanned in the vertical direction, and continuous black pixels (black runs) are detected on the scanning line, and L ≧ h. The vertical sub-pattern is extracted by extracting the black run of length L as the black pixel portion of the vertical sub-pattern. In the same manner as the vertical sub-pattern extraction, the sub-pattern is extracted when the remaining directions are set to the main scanning direction.

In the feature extracting unit 202, square areas corresponding to the character circumscribed frame of the character pattern are set on the extracted sub-patterns in each direction, and N × M square areas (N and M are arbitrary) (A suitable natural number), and the feature amount representing the length of the character line of each sub-pattern included in each small region is extracted. The feature amount is normalized by the size of the character circumscribed frame. feature matrix F consisting of feature amounts f _i was
Is created and output to the matching unit 203. In the present embodiment, the number of divisions N and M = 8, and the normalization of the feature amount by the size of the character circumscribed frame is (dX + dY) /
It shall be normalized by two values. Here, dX is the horizontal length of the character circumscribed frame, and dY is the vertical length of the character circumscribed frame. Also, the feature amount f _i is 1 to N ×
When a number i (i = 1, 2,..., N × M) up to M is sequentially assigned to represent a small area, the feature amount of the small area with the number i is represented. is there.

The matching unit 203 compares the feature matrix F with a feature dictionary matrix G of standard patterns prepared in advance and stored in the dictionary unit 126 and classified into two or more types according to character attributes. Similarity R represented by (3)
, And the character names of the dictionary matrix whose similarity R is equal to or greater than a predetermined value P are set as candidate character names, and further ranked in order of the highest similarity with the first candidate character and the second candidate character. The above-described candidate character name and the type of dictionary of the standard pattern from which the candidate character name is obtained are output to the block feature extraction unit 124 as a recognition result.

[0034]

(Equation 1)

Here, g _i represents an element of the dictionary matrix.

The dictionary unit 126 stores a feature matrix of a standard pattern for matching with a feature matrix extracted from a character pattern in the matching unit 203 in the character recognition unit 123, that is, a dictionary matrix. Two or more dictionary matrices are prepared for one character name in accordance with character attributes such as standard / italic, line width, and aspect ratio.

In the present embodiment, the first dictionary 126a (dictionary number 1) prepared using a Roman typeface as shown in FIG. 4A as a standard pattern and a sans-serif typeface as shown in FIG. 4B. From the second dictionary 126b (dictionary number 2) created as a standard pattern
In the collating unit 203 in the character recognizing unit 123, the dictionary used to obtain the candidate character names of the collation result and each candidate character name is the first dictionary, A dictionary number indicating whether the dictionary is the second dictionary is output. FIGS. 5A to 5D show examples of recognition results when the first line of each of the character blocks 31 to 34 in FIG. 3 is recognized using the dictionary unit 126 of this embodiment.

The block feature extraction unit 124 tabulates, for each character block, the candidate character name of the recognition result input from the character recognition unit 123 and the type of dictionary from which the candidate character name was obtained. A character block attribute feature for determining the attribute of the character block as to whether the character block is constituted by the character of the attribute is extracted. That is, by summing up the dictionary numbers of the first candidate characters of the recognition results of all the characters constituting the character block, the dictionary number that has obtained the largest number of first candidates is used as the attribute feature of the character block. Output to the determination unit 125.

As shown in FIGS. 5 (A) to 5 (D), for each of the character blocks 31, 32, 33 and 34 in FIG. 2 is 33 and Roman dictionary 1 is 34
In this example, it is assumed that the sans-serif dictionary 2 has the largest number of appearances, and character block attribute feature values of F ₃₁ = 1, F ₃₂ = 2, F ₃₃ = 1, and F ₃₄ = 2 are obtained. However, F ₃₁ , F ₃₂ ,
F ₃₃ and F ₃₄ are character block attribute characteristic values of the character blocks 31, 32, ₃₃ and ₃₄ in FIG.

In the determination section 125, the block feature extraction section 12
4, a new order relation of each character block is determined based on the attribute of the character block from the tentative order relation obtained in the character block extraction unit 121 using each character block attribute feature value output from The result is output to block selection section 130. The determination of the new order relation of the character block is performed as follows.

First, the character block attribute feature values output from the block feature extraction unit 124 are compared with each other between character blocks to determine whether or not the character block is a character block having the same attribute feature value. That is, it is determined whether or not the characters are composed of characters having the same attribute. This determination is made based on whether or not the character block attribute characteristic values of the two character blocks satisfy the following expression (4).

F _i = F _j (4)

Where F _i is the character block attribute characteristic value of character block i, and F _j is the character block attribute characteristic value of character block j. In the example of the form shown in FIG. 3, as described above, the character feature value of each character block is F ₃₁ = 1, F ₃₂
= 2, F ₃₃ = 1, and F ₃₄ = 2, so that F ₃₁ = F ₃₃ ,
Only two combinations of F ₃₂ = F ₃₄ satisfy the above expression (4). Therefore, the character blocks 31 and 33 and the character blocks 32 and 3
4 are determined to be character blocks having the same attribute, that is, character blocks composed of characters of the same typeface.

Further, the determination section 125 tests the tentative order relation in the following procedure based on the result of the character block comparison, thereby sequentially selecting character blocks and obtaining a new order relation.

First, a character block having the first provisional order relation is selected as a first character block and is set as a reference character block.

Next, character blocks composed of characters of the same type as the reference character block are selected in the order of the provisional order based on the result of the character block comparison.

When the above selection is temporarily completed to the end of the order relation, the character blocks in the temporary higher order relation are sequentially tested again, and a character block which has not been selected by the above is searched.

The unselected character block searched first as described above is selected and set as a reference character block, and the subsequent processing is repeated again for the unselected character block.

If there is no unselected character block as a result of the above search, the selection of the character block is terminated.

The new order relation obtained by the above-described procedure is output to the block selector 130. According to the above-described procedure, the character block of the form as shown in FIG.
33 → 32 → 34.

The block selector 130 stores the character block data input from the character block extractor 121 in a character block memory (not shown).
The character block data is sequentially output to the image storage unit 140 according to the order relation determination result output from the determination unit 125 in the information processing device 120.

The image storage unit 140 includes the block selection unit 13
This is a storage device for sequentially storing character block data output in order from 0 as document image data, and can be easily realized by a hard disk, a magneto-optical disk, a flexible disk, or the like. Also, the image storage unit 140
Is controlled by a control device (not shown), and can arbitrarily select designated document image data and output it to the output terminal 150.

(Second Embodiment) FIG. 6 is a block diagram showing the configuration of a character recognition device using the information processing device of the present invention. The character recognition device 600 includes an image input unit 610 and an information processing device 620. , A recognition result storage unit 630, a block selection unit 640, and an output terminal 650. The information processing device 620 includes a character block extraction unit 621, a character cutout unit 622, a character recognition unit 623, a dictionary unit 626, and a block feature. An extraction unit 624 and a determination unit 625 are provided. Note that the dictionary unit 626 includes a first dictionary 626a and a second dictionary 626b that store dictionary matrices having different typefaces.

The image input device 610 of this embodiment, the character block extraction unit 621, the character cutout unit 622, the block feature extraction unit 624, the determination unit 625, and the dictionary unit 626 are respectively provided by the image input device 110 of the first embodiment. The components are the same as those of the character block extraction unit 121, the character cutout unit 122, the block feature extraction unit 124, the determination unit 125, and the dictionary unit 126, and a description thereof will be omitted.

In the character recognition unit 623, sub-patterns are extracted from the input character pattern in the same manner as in the first embodiment, and a characteristic representing the length of the character line of each sub-pattern is extracted and normalized. Table characteristic matrix F consisting of feature amounts f _i, in advance by being stored in the dictionary unit 626 character attributes against the feature dictionary matrix G of the standard character patterns which are classified into two or more kinds, equation (3) The similarity R is calculated, and the character names of the dictionary matrix whose similarity is equal to or greater than a predetermined threshold value P are set as candidate character names, and the first candidate character and the second candidate character are ranked in descending order of similarity. And the type of dictionary of the standard character pattern from which the candidate character name and the candidate character name are obtained
4 and the candidate character names are stored in the recognition result storage unit 630.
Is output for each character block.

The recognition result storage unit 630 stores the character recognition unit 62
3 is a storage unit that stores the recognition result of each character block output from 3 for each character block. In addition, the recognition result storage unit 630 is configured to be able to output the recognition results in character units of the character blocks selected by the block selection unit 640 described later in the order of selection, by a control signal (not shown).

The block selection section 640 sequentially reads the recognition results for each character block from the recognition result storage section 630 according to the order relation judgment result output from the judgment section 625 and outputs the result to the output terminal 650.

Although the embodiments of the information processing apparatus and the character recognition apparatus according to the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications as described below are possible. It is.

For example, extraction of a character block, extraction of a character line, extraction of a character pattern, creation of a feature matrix and similarity calculation in character recognition are not limited to the method described in the embodiment, but may be other methods. Obviously, any suitable method can be used.

Further, in this embodiment, the extraction of the dictionary number for obtaining the character block feature is performed for all the character patterns of the character block. However, the present invention is not limited to this. Alternatively, the feature extraction may be performed only from the character pattern of the first character line of each character block, and the character block characteristics of the character block may be obtained from the characteristics of the character pattern of the first character line.

In this embodiment, the dictionary section in the information processing apparatus includes a first dictionary that uses Roman characters as standard patterns and a second dictionary that uses sans-serif characters as standard patterns. However, the present invention is not limited to this, and two or more dictionaries are created based on differences in character attributes such as other typefaces, line widths such as medium / bold, and aspect ratios such as standard / condensed. May be.

In the above-described embodiment, the information processing apparatus of the present invention is applied to the document image filing apparatus and the character recognition apparatus. However, this information processing apparatus determines the order relation of the character blocks, and converts the character blocks in accordance with this order. The present invention can be applied to an image processing apparatus that sequentially outputs images, and an apparatus that extracts a document layout structure by extracting the order relation of character areas.

[0063]

As described above in detail, according to the information processing apparatus of the present invention, a character block is extracted from a document image in accordance with a provisional order relation, a character pattern is cut out from the character block, and the characteristics of the character pattern are extracted. The dictionary classified by the above is compared with the cut-out character pattern, recognition is performed, a dictionary name of one or more candidate characters is obtained, and a character block attribute feature is extracted from the dictionary name of the candidate character of the recognition result, Using this character block attribute characteristic, a new order relation of each character block can be determined.

Therefore, even when extracting and outputting a character block from a document in which the layout of each character block is not regular, each character block is simply output based on the position coordinates of the character block in the document image. Compared to the device,
Each block can be output in the correct order.

Therefore, when the present invention is applied to various document image processing apparatuses such as a document image filing apparatus, character block data can be used in a correct order relationship, and various high-performance document image processing apparatuses can be used. A processing device becomes possible.

When the information processing apparatus of the present invention is applied to a character recognition apparatus, character recognition can be performed on character blocks output in the correct order as described above. Character recognition of a document can be performed in accordance with the flow of the document, and a practically effective character recognition device can be realized. In addition, since the recognition result character string is obtained in the correct order, the post-processing performed after the recognition in units of characters can be performed correctly, so that the recognition accuracy is improved, the work of the operator is reduced, and the input can be performed at high speed. Therefore, a high-performance character recognition device capable of accurately and quickly inputting a document having no regular layout can be realized.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a document image filing apparatus using an information processing apparatus according to the present invention.

FIG. 2 is a block diagram illustrating a configuration of a character recognition unit.

FIG. 3 is a diagram illustrating an example of a document image.

FIG. 4 is a diagram provided for explanation of a dictionary unit.

FIG. 5 is a diagram provided for explanation of a block feature extraction unit.

FIG. 6 is a block diagram illustrating a configuration of a character recognition device using the information processing device of the present invention.

[Explanation of symbols]

 Reference Signs List 100 document image filing device 110 image input unit 120 information processing device 130 block selection unit 140 image storage unit 150 output terminal 600 character recognition device 610 image input unit 620 information processing device 630 recognition result storage unit 640 block selection unit 650 output terminal

Claims (2)

(57) [Claims] 1. A character block extracting unit for extracting a character block from a document image and determining a provisional order relation between the character blocks based on a positional relationship between the character blocks, and a character for extracting a character pattern from the extracted character block. A cutout unit, a dictionary unit having a plurality of types of dictionaries classified according to character attributes, collating the character pattern cut out by the character cutout unit with the dictionary, and combining the dictionary name of the candidate character with the candidate character name. A character recognizing unit that outputs for each character block, and a character block attribute feature for determining the attribute of the character block as to what attribute the character block is composed of based on the dictionary name of the candidate character A block feature extraction unit, and a new order based on the attribute of the character block from the temporary order relationship between the character blocks. The information processing apparatus characterized by comprising a determination unit for determining the ordinal relationship. 2. An image input unit for inputting a document image, an information processing device according to claim 1, and a recognition result storage unit for storing a character recognition result in a character recognition unit in the information processing device for each character block. And a block selection unit that outputs a recognition result from the recognition result storage unit based on a determination result of a character block order relationship in a determination unit in the information processing device.