JP2825072B2 - String recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for recognizing a handwritten character string, and more particularly to an optical character reading apparatus (OC).
The present invention relates to a method and apparatus for recognizing a handwritten character string in R).

[0002]

2. Description of the Related Art A character string recognition technique is widely used in, for example, a device for reading address information described on a mail or a form. As a conventional character string recognition technique, there is a method in which a plurality of cutout candidates are obtained from an input character string, and a character string is finally recognized by using linguistic knowledge from a result of performing character recognition on all candidates. Are known.

However, these character string recognition techniques are based on the premise that a character string is cut out one line at a time, and for example, it fails to cut out a character string line by line and two lines are input at a time. Then, there is a problem that the character string cannot be correctly recognized.

In order to solve this problem, various devices have been conventionally devised for extracting character strings with high precision by analyzing graphic information in more detail. For example, "Basic study of constraint-satisfiable character area extraction" by Gyten et al.
(IEICE Technical Report, PRU92-119, 1993) and Nakajima et al., "Introduction of Trial Verification Process in Detecting Address Line from Handwritten Mail" (IEICE Technical Report, PRU95-6, 19)
95) is known. Further, as disclosed in Japanese Patent Application Laid-Open No. 7-6202, a method of extracting a character string once and then extracting the character string again is disclosed.

However, in a free-form handwritten character string described on a mail or a form, the size and shape of each character in the character string vary greatly, and the character strings may be complicated. It is difficult to completely extract a character string line by line only from graphic information.

Therefore, in the conventional character string recognition method, if the input character string is not one line, the problem that the character string cannot be correctly recognized becomes an unavoidable problem.

[0007]

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a character string recognition method and apparatus capable of correctly recognizing a character string even when the number of lines of an input handwritten character string is plural. And

[0008]

According to the present invention, there is provided a character string recognizing apparatus according to the present invention, wherein a character string image storage unit for storing a character string image scanned optically; A character cutout unit that reads a character string image and creates a character candidate pattern and character candidate graphic information, a character recognition dictionary storage unit that stores a standard character pattern, and the standard character pattern stored in the character recognition dictionary storage unit. A character recognition unit that collates the character candidate pattern read from the character cutout unit and obtains a character code and a character recognition cost as a scale expressing reliability of the character recognition result as a character recognition result, and linguistic knowledge that stores linguistic knowledge A storage unit,
A layout knowledge storage unit that stores layout knowledge as graphic knowledge such as character description conditions, and reads the character candidate graphic information from the character cutout unit to read the character code corresponding to the character candidate graphic information from the character recognition unit. The layout knowledge is read from the layout knowledge storage unit, the linguistic knowledge stored in the linguistic knowledge storage unit is linguistically collated with the character code, and a combination of the character codes that satisfies the linguistic knowledge is read. A linguistic knowledge processing unit that obtains a linguistic cost as a measure expressing the linguistic reliability of the character string candidates created and created as column candidates; and the language knowledge processing unit that reads the character candidate graphic information from the character cutout unit And reads the layout knowledge from the layout knowledge storage unit. A layout analysis unit that obtains a layout cost as a measure for expressing the validity of the layout constraint for the character string candidate, and reads the character recognition cost from the character recognition unit, and reads the character recognition cost from the linguistic knowledge processing unit for the character string candidate. The linguistic cost is read, the layout cost of the character string candidate is read from the layout analysis unit, and the character string candidate that provides the best cost is obtained by combining the character recognition cost, the linguistic cost, and the layout cost. A character string recognition result output unit that outputs a character string recognition result is configured.

[0009]

According to the present invention, when recognizing a character string, characters are first cut out and recognized without assuming a character line.
Once you create a character combination that satisfies your linguistic knowledge. As a result, many combinations satisfying the linguistic knowledge can be obtained. From this state, a character string recognition result is obtained using the cost of character recognition, the cost of linguistic knowledge, and the cost of layout analysis. In this method, the input character string was not a single line, because it made comprehensive judgments at the same level as linguistic knowledge, including not only the ambiguity of cutting out individual characters but also the ambiguity of character line extraction. Even if it can recognize the character string correctly.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the first embodiment of the present invention. The configuration of the embodiment shown in FIG. 1 is a character string image storage unit 1 for storing a character string image optically scanned.
A character cutout unit 2 that reads a character string image and creates a character candidate pattern and character candidate graphic information corresponding to the character candidate pattern; a character recognition dictionary storage unit 3 that stores a standard character pattern; A character recognition unit 4 for collating candidate patterns to obtain a character code and a character recognition cost as a measure for expressing the reliability of the character recognition result as a character recognition result; a linguistic knowledge storage unit 5 for storing linguistic knowledge; A layout knowledge storage unit 6 that stores layout knowledge as graphic knowledge such as conditions, reads character candidate graphic information from a character cutout unit 2, reads a character code corresponding to the character candidate graphic information from a character recognition unit 4, and stores layout knowledge. The layout knowledge is read from the section 6 and the linguistic knowledge of the linguistic knowledge storage section 5 is linguistically collated with the character code, and the A plurality of combinations of character codes that satisfy the intelligibility as character string candidates, and a linguistic knowledge processing unit 7 that obtains a linguistic cost as a measure for expressing linguistic reliability for each of the plurality of generated character string candidates. The character candidate graphic information is read from the character cutout unit 2, all the character string candidates are read from the language knowledge processing unit 7, the layout knowledge is read from the layout knowledge storage unit 6, and the validity of the layout constraint is expressed for each character string candidate. A layout analysis unit 8 for obtaining a layout cost as a measure to be read, a character recognition cost from the character recognition unit 4, a linguistic cost of a character string candidate from the language knowledge processing unit 7, and a layout cost of a character string candidate from the layout analysis unit 8. Is read, the character string candidate that gives the best cost when these three costs are combined is recognized by character string recognition. Comprising a character string recognition result output unit 9 for outputting a.

Next, the operation of this embodiment will be described.

The character string image storage unit 1 is a normal storage unit for storing a character string image input by a normal image input unit such as an image scanner, and the character string image is, for example, a binarized image. .

The character cutout unit 2 extracts a partial image which may be one character from a character string image as a character candidate pattern, and simultaneously creates graphic information on the character candidate pattern as character candidate graphic information. is there.
There are several methods for extracting character candidate patterns and simultaneously creating character candidate graphic information. As an example, this can be realized by creating all combinations of the black pixel connection regions of the binarized character string image and extracting the height and width of the combination image, the circumscribed rectangle, and the position coordinates on the image. . FIG. 2 shows an example. FIG.
When a character string image such as "Noda" is given as shown in (a), the black pixel connected region is extracted as a partial image, and the partial image and its circumscribed rectangle are extracted as shown in FIG. The height, width, and position coordinates on the image are stored, and all combinations are created while maintaining the relative positional relationship between the partial images as shown in FIG. 10C, and a new partial image corresponding to the combination is created. The character candidate pattern and character candidate graphic information can be created by calculating the height and width of the circumscribed rectangle and the position coordinates on the image.

The character recognition dictionary storage section 3 is a normal storage means for storing standard character patterns.

The character recognition unit 4 reads a character candidate pattern from the character cutout unit 2, reads a standard character pattern from the character recognition dictionary storage unit 3, checks and recognizes the character candidate pattern and the standard character pattern, and obtains a character recognition result as a character recognition result. This is a means to obtain the character recognition cost as a scale expressing the reliability of the code and the character recognition result. Recognition performed here is, for example, a simple similarity method well known in the art (refer to the document “Introduction to Character Recognition,” Ohmsha, 1982, pp. 34-35) or “improvement of direction pattern matching and handwritten kanji recognition by Tsugumo”. Application ”(IEICE Technical Report, PRU90-20,
1990).

The character recognition cost may be such that the higher the possibility that the character recognition result is erroneous, the larger the value. For example, a patent specification (Japanese Patent Application No. Hei. 334347 "Character string recognition method and apparatus"), the cost can be obtained as follows using the character evaluation value.

Assuming that the character recognition cost is rc and the character evaluation value is r, rc = a1-r (1) where a1 is a predetermined constant. Rc = a2 / r (2) Here, a2 is a predetermined constant.

The linguistic knowledge storage unit 5 is a normal storage means for storing character code string information of standard words, constraints on word connection relations, and the like as linguistic knowledge. The information stored as the linguistic knowledge may be, for example, the following information.

In the case of reading a destination address written on a postal matter or a form, an upper-lower order relationship of an address represented by a tree structure in FIG. 3 is stored as a constraint condition of a word connection relationship. That is, in the address structure of Shinagawa-ku in FIG. 3, there is “Shinagawa-ku” below “Tokyo”,
It is related that there are other town names such as "EBARA", "Flag Nodai", etc. In the case of an address, in general, a basic superordinate-subordinate relationship is established in the order of a prefecture name, a municipal county name, and a town name, and the town name level is represented by a capital letter name, a character name (or a town name, a town name), or the like. It can be multistage. In the case of an address, the upper-lower relationship of the address elements directly serves as a constraint on the arrangement of the address elements. That is, the address elements are arranged in order from a higher-order address element to a lower-order address element. According to the example based on FIG. 3, the sequence of "Tokyo"-"Shinagawa-ku"-"Nakanobu" is possible, but "Tokyo"-"Nakanobu" or "Nakanobu"
-An arrangement such as "flag stand" is not allowed. Examples of such a connection constraint expression method include a method of describing whether connection is possible for all combinations of two address elements, and a method of listing address elements that can be connected immediately after or immediately before an address element. is there. In the case of reading a character string of a general document, whether or not words can be connected is described based on not a hierarchical structure such as an address but a grammatical restriction on word arrangement.

The character code string information of the standard word includes, for each character constituting the word appearing in the object to be read, a character string U of a word, a character string length L of U, and a position P in the U of the character. Stores term information. FIG. 4 shows an example of the contents of the character code string information for the address element group as shown in FIG. In FIG. 4, the key characters on the left side of “:” are the characters appearing in the address element, and the corresponding three-term information is arranged on the right side. The three items of information, for example,
In the case of [EBARA, 2, 1] corresponding to "EB", it represents that the character "EB" is the first of two characters of the address element "EBARA". Since a certain character may appear in a plurality of address elements, for example, “Nishinakanobu, 3, 3” [Nakanobu, 2, 2] “Higashinakanobu, 3,” corresponding to “Nobu”
3], the character “Nobu” is the third character of the three characters of the address element “Nishinakanobu” or the address element “Nakanobu”
, Or the third of the three characters of the address element “Higashinakanobu”. In addition,
Although the address element (word) U in the ternary information is represented by a character string in the example of FIG. 4, it may be represented by a code value associated with the address element (word) instead of the character string itself.

The layout knowledge storage section 6 is a normal storage means for storing information on a character layout such as a format as layout knowledge. The information stored as the layout knowledge may be, for example, the following information.

When attention is paid to a certain character in the described character string, a condition is given as to what position the preceding character can be described with respect to the noted character. A parameter for determining the front writing area can be stored.

For example, the front writing area for a certain character can be represented as shown in FIG. Such a region can be defined as a region satisfying the following equation.

The forward description area for a certain character block can be represented by the following area using station coordinates with the center point of the character block as the origin.

Rf> r1 0 <θ <θ1 (3) 0 θ1 <θ <θ2 r2 θ2 <θ <θ3 r1 / cosθ θ3 <θ <2π Here, the direction of θ = 0 is the basic direction of the character string. (Vertical direction from top to bottom for vertical writing, horizontal direction from left to right for horizontal writing) is a direction rotated 90 degrees counterclockwise. Note that π / 2 <θ1 <π, 3π / 2 <θ2 <θ3 =
Arccos (r1 / r2) <2π.

In FIG. 5, (a) shows an example of a postal matter written in a vertical writing mode. The right side of a character serving as a reference for determining the forward writing area may be located one line before its own line. It is possible, and has the meaning that characters in the same line may be described on the reference character.

In the case where the forward description area is defined by using equation (3), parameters r1, r2, θ1, θ2, and θ3 are used.
May be stored.

Also, only when the character serving as a reference for determining the front writing area is the first character of a word, it is determined that there is a possibility that a line feed is performed immediately before the first character, as shown in FIG. The area can be considered as the front writing area, and at other times, the front writing area can be defined as shown in FIG.

The area at this time can be defined as, for example, an area satisfying the following equation.

Rf> r3 π−θ4 <θ <π + θ4 (4) 0 else Here, the direction of θ = 0 is the basic direction for writing a character string. Note that 0 <θ4 <π / 2.

When determining the forward writing area other than the first character of the word by the equation (4), r3 and θ4 may be stored.

Further, the front writing area can be defined as a combination of rectangular areas. Some rectangular coordinate information may be stored with the center point of a certain character block serving as a reference for determining the forward writing area as the origin. In this case, the number of rectangles for determining the front writing area, the center coordinates, the height, and the width of those rectangles may be stored. In addition to this, for example, the inside of a circle having a certain radius r4 around the reference character may be used as the front writing area. In this case, r4 may be stored.

As another possible layout knowledge, if a character is skipped when recognized as a character string, an area in which the skipped character may be described is defined as a corresponding area. Can be memorized.

Now, assuming that a character block corresponding to the i-th element of the character string is block i and a character block corresponding to the i + j-th element of the character string is block i + j, the i + k-th element is located between the block i and the block i + j. An area in which a block i + k, which is a character block corresponding to an element, can be described is an applicable area. Where j> 1 and j> k
> 0.

The corresponding area can be defined, for example, as the following area. Rd is defined as a region that satisfies the following equation with the center point of block i as the origin.

[0036] Ru is defined as a region that satisfies the following equation with the center point of block i as the origin.

Ru> r5 π−θ5 <θ <π + θ5 (6) 0 else Here, r5 is the distance between the center point of block i and the center point of block i + j. Θ6 represents a deviation of an angle from a direction of a straight line connecting the center point of the block i and the center point of the block i + j, and is a value of π / 2 or less. The overlapping area of the areas Rd and Ru determined by the equations (5) and (6) may be set as the corresponding area. An example at this time is shown in FIG. Rd is a diagonally shaded area falling rightward, and Ru is a diagonally shaded area rising rightward. In this case, r5 is the block i
Since it is a value that can be calculated for the first time when the relationship between the block i and the block i + j becomes clear, there is no need to store the value in advance, and θ5 may be stored.

The corresponding area can be defined and stored as a combination of rectangular areas. In that case, for example, it can be represented by a rectangle centered on the center point of a straight line connecting the center point of block i and the center point of block i + j.
In this case, the height and width of the rectangle can be represented by the ratio of the distance between the center point of block i and the center point of block i + j, and the ratio is stored as a parameter for determining the block height and block width. Good.

As another layout knowledge, a write-inhibited area is defined that must not be written between any two consecutive characters when a character string is recognized, that is, between block i and block i + j. You can also memorize.

This can be defined in the same manner as the method used when defining the applicable area. For example, the area Rd calculated by the equation (5) and the equation (6) are calculated by setting k = 1. An area where the overlapped area Ru overlaps may be set as a writing prohibited area. Therefore, the same parameters as those of the corresponding area may be stored as the writing prohibited area.

Further, as a layout knowledge, a line end area can be defined and stored in order to check whether it is the end of a row. Usually, no character is described after the character located at the end of one line, so the line end area is defined as an area where no character can be described after the end of the line.

In order to define this area, for example, an area having the origin at the center point of the block located at the end of the line is defined as follows, assuming that the angle deviation from the basic direction of writing the character string is θ6.

[0043] Here, θ <θ6 <π / 2. In this case, the contents to be stored are r6 and θ6.

The line end area can also be defined and stored as a rectangular area. In this case, the relative coordinate information of the center point of the block located at the end of the line and the rectangle defined as the origin is stored. Good.

As other layout knowledge, depending on the word, the size of the character may vary greatly within the word. For example, if the word is "Kasumigaseki" as shown in FIG. It is possible to store the knowledge that the “ga” coming in the word is often written small as block size information in a word.

For example, as the intra-word block size information composed of registered n characters, the intra-word standard block size ratio [H1, V1, H2, V2,..., Hn, V
n] is stored. The standard block size ratio within a word can be calculated, for example, as follows. Since the size of the characters described for each document is different, first, from the data of the height and width of the character block which is a component of the word, [h1, v1, h2, v2,..., Hn, vn] This can be calculated by creating a data format and dividing the value of each element by h1 to obtain an average of the values once normalized.

In addition to the standard block size ratio within a word, a covariance matrix of [H1, V1, H2, V2,..., Hn, Vn] can be calculated and stored.

The information that can be stored as layout knowledge is not limited to this. For example, the relative relationship between ruled lines and characters, the relative relationship between patterns and characters, and the like can also be stored.

The linguistic knowledge processing unit 7 reads the character candidate graphic information from the character cutout unit 2, reads the character code corresponding to the character candidate graphic information from the character recognition unit 4, reads the layout knowledge from the layout knowledge storage unit, and stores the language knowledge. The linguistic knowledge of the part 5 is read, the linguistic collation between the linguistic knowledge and the character code is performed, and a combination of character codes satisfying the linguistic knowledge is created as a character string candidate. It is a means of obtaining linguistic costs as a measure of expressing gender.

Several methods can be considered for the linguistic collation referred to here. As an example, the following method can be considered.

Using each of the character codes corresponding to the character candidate graphic information read from the character cutout unit 2 as a key, character code string information of a standard word, which is one of language knowledge, is searched. Then, if there is a character corresponding to the key character of the character code string information of the standard word, the ternary information corresponding to the character is read and registered.

Next, as a method of linking two pieces of the three-item information among the registered three-item information and creating a candidate for the reading result, for example, the following method can be considered.

The first method for creating a three-term information chain is two pieces of three-term information [U1, L1, P1] and [U2, L
[2, P2] Condition a: The character candidate graphic corresponding to [U1, L1, P1] is in the front entry area of the character candidate graphic corresponding to [U2, L2, P2] Condition b: U1 = U2 and P1 <P2 Condition c: U1 can be present in the forward description area of U2. When Condition a and (Condition b or Condition c) are satisfied, [U2, L2, after [U1, L1, P1].
P2] to form all possible combinations, and then calculate the cost of each combination.

In addition to this, for example, a method described in a patent specification (Japanese Patent Application No. 6-317163 “character string reading device”) by the same applicant as the present application can be used.

Various linguistic costs can be considered. For example, the number of skipped characters may be used as it is as the linguistic cost, or the number of skipped characters may be divided by the total number of characters in the character string. Also, when reading addresses, etc., information such as prefectures, municipalities, etc. is often written at the beginning of the character string, and the town name is often written at the end of the character string. Since the recognition error is more likely to occur when the character cannot be read, it is possible to calculate the cost with a stronger weight toward the end of the character string. Further, when there are continuous skips, the greater the number of skips, the greater the cost.

In addition to this, for example, a method described in a patent specification (Japanese Patent Application No. 6-317163 “character string reading device”) by the same applicant as the present application can be used.

The layout analysis unit 8 includes the character cutout unit 2
, Character string candidate information is read from the language knowledge processing unit 7, character layout candidates are read from the layout knowledge storage unit 6, and layout knowledge is read from the layout knowledge storage unit 6. The layout cost as a measure for expressing the validity of the layout constraint is obtained for each character string candidate. Means.

There are several methods for calculating the layout cost. For example, this can be realized as follows.

As shown in FIG. 7, the width of the block i corresponding to the i-th character of a certain character string candidate is x _i , and the height is y _i.
Let s _i be the square root of the area of block i. Similarly, let s _j be the square root of the area of block j corresponding to the j-th character. Further, an angle between a vector v _ij connecting the center of the block i to the center of the block j and the basic direction of the character string is ω _ij, and the distance between the center points of the block i and the block j is d _ij . At this time, the following cost is calculated.

C1 _{(i, j)} = d _ij / (s _i + s _j ) (both blocks i and j are present) (8) α1 (other than that) c2 _{(i, j)} = | s _i −s _j | / (s _i + s _j ) (both blocks i and j exist)... (9) α2 (other than that) c3 _{(i, j)} = ω _ij (both blocks i and j exist) (10) α3 (other than that) At this time, α1, α2, and α3 are constants given in advance.

Further, as shown in FIG. 8, a vector connecting the center of block j to the center of block j corresponding to the i-th and j-th elements of a certain character string candidate is represented by v _ij , and the distance between block centers is represented by d _ij . _ij , v _{jk is} a vector connecting the center of block k to the center of block k corresponding to the j-th and k-th elements of a certain character string candidate, the distance between block centers is d _jk , and the angle between v _ij and v _{jk Is} θ _ijk . At this time, the following cost is calculated.

C4 _{(i, j, k)} = θ _ijk (all blocks i, j, k exist)... (11) α4 (other) c5 _{(i, j, k)} = | d _ij − d _jk | / (d _ij + d _jk ) (all blocks i, j, and k exist) (12) α5 (otherwise) At this time, α4 and α5 are predetermined constants. When calculating these costs, when i + 1 = j, c1
_{(i, j)} = 0, c2 _{(i, j)} = 0 and c3 _{(i, j)} = 0 may be set. When i + 2 = j + 1 = k, c4
_{(i, j, k)} = 0 and c5 _{(i, j, k)} = 0 may be set.

If a character code candidate is skipped because a character code that should be present in the character string candidate has been omitted, c1 _{(i, j)} , c2 _{(i, j)} , c2 _{(i, j)} ,
Instead of giving a constant value as a penalty to c3 _{(i, j)} , c4 _{(i, j, k)} , and c5 _{(i, j, k)} , a character candidate pattern that can be made to correspond to an insect-eating part is supported. If it exists in the possible area, a temporary corresponding character is set in that part, and c1 _{(i, j )} , c2 _{(i, j)} , c3 _{(i, j)} , c
4 _{(i, j, k)} and c5 _{(i, j, k)} can also be calculated.

To make provisional correspondence, for example, a block in which the block center point is located in the applicable area is detected, and
The detected blocks may be used as provisional correspondence candidates, and all possible combinations of these blocks may be used as provisional correspondence results. As a result, there are a plurality of ways of temporarily coping with the worm-eating part. Considering the possibility of multiple provisional measures, considering the case of the lowest cost value among them as the result of provisional measures, the cost at that time may be used as the layout cost, or the average cost of all provisional measures May be used as the layout cost. FIG. 9 shows the result of a plurality of provisional correspondences. The example of FIG. 9 is a case where an attempt is made to recognize “Kawagoe City Uenodamachi” as a character string. In FIG. 9A, the “field” and “field” of “Uenoda-cho” are worm-eating. FIG.
In (b), all possible combinations are shown using three blocks that have been provisional correspondence candidates.

When the layout cost is calculated using the result of the provisional correspondence, a constant value may be given to each cost as a penalty only when there is no character candidate pattern provisionally corresponding to the worm-eating part. .

It is also possible to evaluate whether or not an extra block is mixed in a certain candidate character string and use it as one of the layout costs. This is to evaluate whether there is another character candidate pattern in a writing prohibited area set between two linguistically consecutive characters in a certain candidate character string, and if there is another character candidate pattern, The cost may be set so that the larger the block size of the character candidate pattern, the higher the cost. FIG. 10 shows an example thereof. FIG. 10 shows a case where an attempt is made to recognize “Noda-cho, Kawagoe-shi” as a character string. In FIG. 10A, "field" and "town"
Since there is no other character candidate pattern in the writing prohibited area between "" and "Noda", it is fairly appropriate to read "Nodamachi". However, in FIG. 10B, a large block is mixed between "Ta" and "Town". Therefore, the validity of reading “Noda-cho” is low.
In (c), the content that the mixed block is small and thus slightly valid is reflected in the cost. The cost can be calculated by, for example, the following method.

Block i corresponding to the i-th element of a certain character string candidate and block i + corresponding to the i + 1-th element
The square roots of the area where the writing prohibited area set between the block i and the block i + 1 and the block other than the block i and the block i + 1 overlap each other, where the square roots of the area of 1 are s _i and s _{i + 1} , respectively. If the sum of S is S, the following cost can be calculated.

C6 _{(i, i + 1)} = S / (s _i + s _{i + 1} ) (both blocks i and i + 1 exist) (13) 0 (others) Also, the entire character string The following costs can be considered in order to evaluate the feasibility.

Now, assuming that there are character string candidates having the number of characters N, the block corresponding to the i-th character of the candidate character string is defined by setting the basic direction of the character string as the H axis and the direction perpendicular to the basic direction of the character string as the V axis. i the coordinate of the center point of the _{B i H m, B i V} m
And the average value of B _i V _m is AveV _m, and |
_{B N H m -B O H m} | When the the LineH, it is possible to calculate the cost as follows.

C7 _i = | B _i V _m −Ave V _m | / LineH (14) An example of this is shown in FIG. FIG. 11 shows an example where the number of characters N = 6.

Further, since the address is not always described on one line, it is possible to evaluate the case where the address is written over two lines. In this case, the line break is detected and split into two lines.
Next, it is sufficient to evaluate each of the actions.

Various methods can be considered for detecting a line feed.
Here, for example, for example, c1 _{(i, j)} and c3
_{If there is a part where (i, j)} , c5 _{(i-1, j-1, k-1)} and c5 _{(i, j, k)} have large values at the same time, Is considered to be a line break and can be divided into two lines. Here, there is a relationship of i + 2 = j + 1 = k. FIG. 12 shows an example of this. As shown in FIG.
(6,7), c3 (6,7), c5 (5,6,7) and c
When 5 (6, 7, 8) simultaneously increases, it can be detected that blocks 6 and 7 are line breaks.

When a line feed is detected, the layout cost calculated at the line break can be cleared. In addition, it is also possible to divide the line into two lines at the line feed portion and calculate the total sum of the lines.

It is also possible to evaluate whether a line break is likely. This is because a line end area is set behind a block corresponding to the end of a certain m-th row, and a constant α6 is given as a penalty c8 _m as a penalty when a block other than the end block exists in that area. Otherwise, α7 may be given. At this time, α6 and α7 are predetermined constants, and α6 >>
α7. When the address is recognized as a character string, c8 _m = 0 when the last character of the address matches the line end block, and a character candidate that can be recognized as a number in the line end area of the last character of the address. When a block exists, c8 _m = 0 can be set.

Further, when calculating the character likelihood as shown in FIG. 13, the basic direction of the character string description is newly set based on the direction of the straight line obtained by the least square method based on the center point of each block of the candidate character string. You can also ask.

Further, it is also possible to evaluate the likelihood of a word in a certain character string candidate by using graphic information using standard intra-word block size information. Now, the word-likeness can be evaluated by comparing the coordinate information of a block of a certain word in the evaluated character string candidate with the standard block size information within a word corresponding to the word.

For example, the likelihood of a word can be calculated using the Euclidean distance between the vectors in this comparison. Mahalanobis distance or the like may be used for evaluation of wordiness. The sum of the word-likeness may be c9, which is one of the layout costs of the candidate character string.

Assuming that the number of characters of a character string candidate is N and the number of character lines is M, the layout cost for the character string candidate is calculated as follows.

LC1 = (1 / N) ΣΣc1 _{(i, j)} (15) LC2 = (1 / N) ΣΣc2 _{(i, j)} (16) LC3 = (1 / N) ΣΣc3 _{( i, j)} (17) LC4 = (1 / N) ΣΣΣc4 _{(i, j, K)} (18) LC5 = (1 / N) ΣΣΣc5 _{(i, j, K)} ... ( 19) LC6 = (1 / N) Σc6 _{(i, i + 1)} (20) LC7 = (1 / N) Σc7 _i (21) LC8 = Σc8 _i (22) LC9 = c9 (23) The character string recognition result output unit 9 reads the character recognition cost from the character recognition unit 4, reads the linguistic cost of each character string candidate from the linguistic knowledge processing unit 7, and reads each linguistic cost from the layout analysis unit 8. This is a means for reading the layout cost of a character string candidate and outputting as a character string recognition result a character string candidate that provides the best (lowest) cost when these three costs are combined.

There are several methods for calculating the character string recognition cost by combining the character recognition cost, the linguistic cost, and the layout cost. For example, this can be realized as follows.

Assuming that the total number of characters of a certain character string candidate is N, the character recognition cost of the i-th character of a certain character string is rc _i
Then, the total character recognition cost RC is calculated as follows.

RC = Σrc _i (24) However, if there is no character recognition result corresponding to the i-th character, ri = β. At this time, β is a predetermined constant. At this time, the cost can be calculated by assigning a stronger weight toward the end of the character string.

The linguistic cost is KC.

Now, the character string recognition cost SC is defined by the following equation.

SC = γ × RC + δ × KC + ε1 × LC1 + ε2 × LC2 + ε3 × LC3 + ε4 × LC4 + ε5 × LC5 + ε6 × LC6 + ε7 × LC7 + ε8 × LC8 + ε9 × LC9 (25) At this time, γ, δ, ε1, ε2, ε3, ε4, ε5 , Ε
6, ε7, ε8, and ε9 are predetermined constants. This cost may be any value as long as it has a low value when the recognition result of the character string is likely to be normal, and has a large value when the recognition result of the character string is likely to be incorrect.

A character string is recognized by outputting the character string candidate whose character string recognition cost calculated as described above has the best value (low value) as a character string recognition result.

Next, a second embodiment of the present invention will be described. In this embodiment, when a character string is recognized, in addition to the method of calculating a character string recognition cost and obtaining the best cost as a recognition result as shown in the first embodiment, a recognition system is configured using learning, The character string recognition result output unit 9 has a function of recognizing a character string using the configured recognition system.

To construct a recognition system using learning,
For example, the following method can be considered.

All character string candidates for some character string images are created in advance, and if there is a correct answer in all the character string candidates, information that the character string candidate is correct is given to other character string candidates. Give information that the answer is incorrect. In this way, each cost RC, KC, LC1, LC2, LC3, LC of the character string candidate to which the correct information and the incorrect information are added.
4, learning is performed considering LC5, LC6, LC7, LC8, and LC9 as inputs.

The learning can be performed by using a statistical method such as a multiple regression analysis or a discriminant analysis, or by using a neural network.

In the case of learning using multiple regression analysis, if the character string candidate is a correct answer, for example, -1;
Should be learned so as to output. When performing recognition using a recognition system configured by learning in this way, a character string candidate that has output the lowest value from the recognition system may be used as a character string recognition result.

The results obtained by the multiple regression analysis are the parameters γ, δ, ε1, ε2, ε3, ε4, ε5, ε6, ε for calculating SC in equation (25).
7, ε8, ε9, and the parameters obtained by the multiple regression analysis are calculated using the equation (2) in the first embodiment.
It can also be the parameter of 5). The same applies when discriminant analysis is used for learning.

When a neural network is used,
RC, KC, LC1, LC2, LC3, LC4, LC
Character string recognition can also be performed using a neural network trained by inputting 5, LC6, LC7, LC8, and LC9 as input, and giving a teacher signal of 1 for a correct answer and 0 for an incorrect answer.

Next, a third embodiment of the present invention will be described. In this embodiment, when recognizing a character string, character extraction is performed to check the recognition result for an unresolved portion such as a character string candidate that gives the minimum cost and a character code of the character recognition result is partially not obtained. The character string recognition result output unit 9 is provided with a verification function of performing the recognition again, and performing character string recognition based on the result.

This verification function can be realized by using, for example, a method described in a patent specification (Japanese Patent Application No. 6-317163 “character string reading device”) by the same applicant as the present application.

[0096]

As described above, according to the present invention, a plurality of character string candidates are created by first performing linguistic knowledge processing.
The layout cost is calculated for these character string candidates, and the character string is recognized using the character recognition cost, the linguistic cost, and the layout cost at the same time. Can be correctly recognized.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an embodiment of the present invention.

FIG. 2 is a view for explaining processing of a character cutout unit in the block diagram shown in FIG. 1;

FIG. 3 is a diagram showing an example of a list of address elements.

FIG. 4 is a diagram showing an example of the contents of character code string information.

FIG. 5 is a diagram showing an example of layout knowledge.

FIG. 6 is a view for explaining graphical knowledge of words.

FIG. 7 is a view for explaining processing of a layout analysis unit in the block diagram shown in FIG. 1;

FIG. 8 is a view for explaining processing of a layout analysis unit in the block diagram shown in FIG. 1;

FIG. 9 is a view for explaining an example of provisional correspondence to worm-eating verification.

FIG. 10 is a diagram illustrating a method of calculating a cost when an extra block is mixed.

FIG. 11 is a diagram illustrating character lineness.

FIG. 12 is a view for explaining evaluation of character / character line likeness after line feed detection.

FIG. 13 is a view for explaining that a basic direction in which a character string is written is determined by a least square method.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image storage part 2 Character extraction part 3 Character recognition dictionary storage part 4 Character recognition part 5 Language knowledge storage part 6 Layout knowledge storage part 7 Language knowledge processing part 8 Layout analysis part 9 Character string recognition result output part

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-8-96075 (JP, A) JP-A-7-57054 (JP, A) JP-A-3-225579 (JP, A) JP-A-63-1988 223890 (JP, A) "Character Recognition Method Using Two-Dimensional Layout Information", IEICE Technical Report, Vol. 95, No. 278 (PRU95 103-120), page 31-36, 1995/9 (58) Fields investigated (Int. Cl. ⁶ , DB name) G06K 9/62 620 Patent file (PATOLIS) JICST file (JOIS)

Claims (7)

(57) [Claims] A character string recognizing device for recognizing a character string by inputting an image of a handwritten character string, a character string image storage unit for storing an optically scanned character string image, and reading the character string image A character cutout unit for creating a character candidate pattern and character candidate graphic information, a character recognition dictionary storage unit for storing a standard character pattern, and a standard character pattern and a character cutout unit stored in the character recognition dictionary storage unit. A character recognition unit that collates the read character candidate patterns and obtains a character code and a character recognition cost as a measure for expressing the reliability of the character recognition result as a character recognition result; a linguistic knowledge storage unit that stores linguistic knowledge; A layout knowledge storage unit for storing layout knowledge as graphical knowledge such as a description condition of the character, and reading the character candidate graphic information from the character cutout unit. The character code corresponding to the character candidate graphic information is read from the character recognition unit, the layout knowledge is read from the layout knowledge storage unit, the language of the language knowledge and the character code stored in the language knowledge storage unit. Language matching processing unit that performs linguistic matching to create a combination of the character codes satisfying the linguistic knowledge as a character string candidate and obtains a linguistic cost as a scale expressing linguistic reliability of the created character string candidate Reading the character candidate graphic information from the character cutout unit, reading the character string candidate from the linguistic knowledge processing unit, reading the layout knowledge from the layout knowledge storage unit, and setting layout restrictions on the character string candidate. A layout analysis unit for obtaining a layout cost as a measure expressing validity; The character recognition cost is read from the language knowledge processing unit, and the linguistic cost for the character string candidate is read from the layout analysis unit.The layout cost of the character string candidate is read from the layout analysis unit. A character string recognition device comprising: a character string recognition result output unit that outputs, as a character string recognition result, the character string candidate that provides the best cost when layout costs are combined. 2. The layout analysis unit according to claim 1, wherein a character code to be present in the character string candidate is omitted, and there is a character portion which is linguistically skipped, and a possibility that the skipped character is described. In a case where character candidate graphic information that is not a component of a character string exists in a certain area, a function of calculating the layout cost by temporarily associating character candidate graphic information that is not a component of a character string with a skipped character is provided. The character string recognition device according to claim 1, wherein 3. The layout analysis unit gives a penalty if character candidate graphic information corresponding to a character code that is not a component of a character string exists in a linguistically continuous part of the character string candidates, and gives the layout cost. 2. The character string recognition device according to claim 1, further comprising a function of calculating. 4. The layout analysis section has a function of detecting whether a line break has occurred in the middle of a character string candidate, and changing the method of calculating the layout cost depending on whether the line break has occurred in the middle or not. 2. The character string recognition device according to claim 1, wherein: 5. The layout analysis unit according to claim 1, further comprising a function of evaluating the likelihood of a word using graphic information for each word existing in a character string candidate and calculating the layout cost. A character string recognition device as described. 6. The character string recognition result output unit has a function of configuring a means for recognizing a character string by combining the character recognition cost, the linguistic cost, and the layout cost by learning. Item 1. The character string recognition device according to Item 1. 7. The character string recognition result output unit includes a verification function for determining whether or not the recognized character string exists when the reliability of the recognized character string is low. 1. The character string recognition device according to 1.