JPH11328318A - Probability table generating device, probability system language processor, recognizing device, and record medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a fast, high-precision probabilioty system language process with small memory capacity. SOLUTION: A unigram generation part 19 obtains unigram. A diagram generation part 20 obtains a diagram. A property trigram generation part 21 obtains a property trigram. A vector division part 23 divides the said diagram into vectors of recognition object character number dimensions. A clustering part 24 clusters the respective vectors to generate a compressed character code conversion table 13 wherein characters and cluster codes (compressed character code) are made to correspond to each other. A compressed diagram generation part obtains the transition probability (compressed diagram) of two compressed character code sets. The said compressed diagram is compressed greatly in the number of elements without losing the linguistic transition information that the diagram has. Lost linguistic information is complemented by using the unigram and property trigram in combination and the high-precision language process can be performed without spoiling low memory capacity and speediness.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a probability table creating apparatus for creating a probability table used for performing a low-capacity memory of characters and voices, and performing language processing at a high speed and a high recognition rate. The present invention relates to a stochastic language processing device and a recognition device, and a computer-readable recording medium.

[0002]

2. Description of the Related Art In recent years, with the advancement of recognition technology, high-precision recognition devices and recognition software having a sufficiently practical level of performance and their application products have been announced and widely permeated by users. As an example, a portable information terminal (PD)
A: In personal digital assistant), handwritten character recognition using a pen is an essential function.

In recent years, recognition accuracy has been improved, of course.
The biggest factor was that the recognition method for characters and voices was tuned, improved, and devised with higher accuracy, but in addition to that, progress in so-called language processing technology (also called knowledge processing) was also a major factor. It is believed that there is.

The importance and necessity of the language processing technology in the character and voice recognition apparatus are based on the following two reasons. (1) The first-order recognition rate of the recognition method alone is still low. No matter how highly accurate the current recognition method is, for example, the first place recognition rate of a single character in Japanese character OCR (optical character reader) is as low as 97% to 98%. However, the in-candidate recognition rate of about the tenth rank exceeds 99%, and it is necessary to select a correct answer in the candidate by some method in order to put a more accurate recognition device into practical use. Under such circumstances, the need for language processing technology increases for handwritten characters and voices whose recognition rate is lower than that of printed characters.

(2) Accurate segmentation (cutout) cannot be performed only with the recognition evaluation value. In current recognition apparatuses, a method of fusing cutout and recognition processing is mainstream. Even in this case, since the performance of the recognition method is unstable as described above, there is a problem that the cutout accuracy is reduced in the cutout that trusts the recognition evaluation value (distance, similarity, etc.). Further, there are also patterns that cannot be cut out by using only the recognition evaluation values. For example, in the case of the character "KA", it is not possible to determine whether the character is "KA" and "RO" in the half-width katakana or "KA" in the full-width kanji by only the pattern. Therefore, some evaluation including the characters before and after is necessary.

For the reasons (1) and (2) above, the importance and necessity of language processing has been recognized since the beginning of character recognition research, and various improvements have been made to this day. At present, language processing is an indispensable process of various recognition devices.

Attempts of linguistic processing in Japan at the beginning of the development of the recognition device are often based on word matching. In this linguistic processing, it is determined by a word dictionary search whether or not a correct recognition result is included in a character string formed by combining recognition candidates. Such word processing has been adopted by many recognition apparatuses since it was a very powerful language processing (which is also considered as recognition processing including cutout and recognition).

However, in the above-described language processing based on word collation, (1) character strings that are not registered in the word dictionary cannot be correctly recognized. If there is no break, there is a problem that segmentation (morphological analysis) of a section for performing word matching cannot be performed accurately.
No. 6464, “Character string recognition method and device thereof”,
[2] Probabilistic language processing disclosed in Japanese Patent Application Laid-Open No. 5-120494, "Character Recognition Method and Apparatus" has been proposed.

The above-mentioned published patent publications [1] and [2],
Basically, it is a stochastic language processing in which a character string is assumed to be a Markov model, and Patent Document 1 discloses a binary diagram (a term indicating the appearance frequency or probability of a combination of two characters). Publication [2] uses a multi-valued modified diagram.

By the way, the above-mentioned stochastic language processing has been used for a long time in European and American recognition technologies. These include, for example, [3] E. Riseman and A. Hans
on, “A contextual postprocessing system for error correction using binary N-grams
error correction using binary n-grams,) "IEEETran
s. Comput., pp. 480-493, May 1974. [4] A. Goshtas
by, “Word recognition using probabilistic relaxation labels (Contextual
word recognition using probabilisticrelaxation la
beling,) ”Pattern Recognition, vol.21, no.5, pp.455-4
62 1988. Reference [3] uses a binary diagram, and document [4] uses a multi-value diagram.

In Europe and the United States, a spell check (word collation) method has been studied from an early stage in language processing, as in research in Japan. However, after that, it shifted to the stochastic method to solve the unregistered problem as described above, and the stochastic method is now mainstream (however, the combination of the stochastic method and the spell check method is often used at present. Has been).

In the United States and Europe, the problem of word segmentation (morphological analysis) has been shifted to the stochastic method although it is relatively easy compared to Japan. This fact shows how many unregistered words are input to the recognizer in general use, and shows that a practically usable recognizer cannot be realized unless this problem is solved. It is. Hereinafter, advantages and problems of the above stochastic language processing will be described.

(A) Advantages of Probabilistic Language Processing The probabilistic method evaluates the certainty of a character string without cutting out morphemes. For example, for the "day" of "Today is fine weather", a recognition result of "eye" or "day" is obtained, and two character strings of "real is fine weather" and "today is fine weather" are generated. I do. The probability value for these character strings is P (real weather is clear) = 0.7
When P (the weather is fine today) = 0.76, the language processing method uses the recognition result “day” as the correct answer. The advantages of the language processing by this probability method are listed as follows.・ Unregistered words such as proper nouns that are not in the dictionary are correct.
(there is a possibility).・ Word segmentation is unnecessary. In particular, it is effective for a language in which morphological analysis is not complete, such as Japanese. -Even if the morpheme has not been determined (for example, during input), the function can be performed, so that the language processing can be performed even when the input is not completed by a recognition device of a type that sequentially inputs, such as PDA handwriting recognition.・ High speed.

Here, a method of calculating the transition probability of an actual character string will be described. Document "Speech Recognition by Stochastic Model"
According to (Seiichi Nakagawa, Institute of Electronics, Information and Communication Engineers, Corona, first edition, 1988), a character string is represented as C = (c ₁ , c ₂ , c ₃ , ...,
Assuming that c _n ) and F () are the appearance frequency of the combination of characters in (), the transition probability P (C) of the character string C is expressed by Expression (1).

(Equation 1)

From equation (1), the transition probability P (C) for an arbitrary-length character string C is, after all, the appearance frequency (or trigram) of a two-character set (digram) and a three-character set (trigram). Probability) can be calculated by preparing a table. From equation (1), it is clear that the morphological analysis is unnecessary because the transition probability value can be calculated without cutting out individual words. The transition probability value is
Since it is a probability value (evaluation value of word likelihood), even if it is an unregistered word, its value increases if it is linguistically certain. Therefore, a correct recognition candidate can be selected based on the transition probability value.

Furthermore, although multiplication / division is used in the above equation (1), generally, a digram and a trigram are represented by transition probabilities, logarithmically transformed (compressed), and added / subtracted as in equation (2). Into an expression. Then, the transition probability P (C) of the character string C is calculated by addition and subtraction according to equation (2). As a result, the transition probability P is calculated by a very simple process of referencing and adding / subtracting the transition probability table between the diagram and the trigram.
(C) can be calculated, and processing can be performed much faster than linguistic processing by dictionary search (word matching).

(Equation 2)

(B) Problems of the language processing by the stochastic method The problems of the language processing by the stochastic method are listed as follows. -Since the transition probability value is a probability value (multi-valued), it is difficult to become a definitive evaluation value. In the language processing based on word collation, the processing result is a binary value of “present” / “not present”, and the processing result is easily determined. On the other hand, since the transition probability value is multi-valued, it is difficult to become a definitive evaluation value of the processing result. In particular, in the case of a recognition device in which the vocabulary is completely determined, the word matching method is more effective in determining the processing result. -When the number of characters to be recognized is large, the memory capacity becomes enormous.

The above-mentioned first problem in the stochastic language processing can be avoided by using the word processing language processing in combination. In fact, the method is mainstream in Europe and the United States. In particular, in the case of Japanese, the second problem, that is, the problem of the memory capacity, is large.

In the case of the transition probability table of the diagram, the memory capacity of the transition probability table of the diagram and the transition probability table of the trigram increases in proportion to the number of appearances of the combination of two characters, that is, the square of the number of characters to be recognized. On the other hand, in the case of a trigram transition probability table,
It increases in proportion to the number of appearances of a combination of three characters, that is, the cube of the number of characters to be recognized. Taking Japanese OCR as an example,
As shown in FIG. 9, when 4000 characters are to be recognized and one element of the transition probability table is 1 byte, the memory capacity is 16 megabytes in the case of a diagram and 64 gigabytes in the case of a trigram. Memory capacity. In the following, the probability table itself is also called a diagram or a trigram.

The simplest solution to this second problem is to give up the use of the trigram and calculate the transition probability P (C) using only the diagram (in this case, the transition probability P ( The accuracy of C) decreases). However, it still requires considerable memory capacity. Therefore, the memory was low capacity and high price for 10 to 20 years.
Years ago, stochastic linguistic processing was feasible in Western languages with few characters, but difficult in Japanese. In recent years, as the price and capacity of memories have increased,
Finally, stochastic language processing has begun to be adopted in Japan.

In the above publications [1] and [2],
The problem of memory capacity is addressed by giving up using trigrams and using only digrams. Further, in the patent publication [1], the use of multi-valued
The use of the value diagram further reduces the memory capacity. Further, in Japanese Patent Laid-Open Publication [2], the memory capacity is further reduced by storing the probability values of only combinations having a certain probability value or more.

[0022]

However, the conventional stochastic language processing disclosed in the above publications [1] and [2] has the following problems. That is, in the patent publication [1], since the binary diagram is used, there is a problem that the accuracy of the obtained transition probability P (C) is significantly reduced as compared with the case where the multivalued diagram is used. is there. Further, although the memory capacity is reduced by using a binary diagram, the memory capacity is still large at 2 megabytes (= 4000 × 4000/8). In particular, considering the use of word-matching language processing together, a separate capacity for the language processing dictionary is required.
It is necessary to suppress the memory capacity as much as possible.

Further, in the patent publication [2], since the probability values of only combinations having a certain probability value or more are stored in the transition probability table, the transition probability P (C) of a certain two characters is calculated. Needs to constantly search whether the transition probability of the combination of the two characters is stored in the transition probability table. Therefore, there is a problem that high-speed performance is impaired.

An object of the present invention is to provide a probability table creating apparatus for creating a multi-valued probability table capable of obtaining a high transition probability and achieving a low memory capacity without impairing high-speed performance, and an object of the present invention. An object of the present invention is to provide a stochastic language processing device and a recognition device using a table.

[0025]

In order to achieve the above object, according to the first aspect of the present invention, there is provided a probability table creating apparatus comprising: a memory in which a character string of one natural language is stored; and a character stored in the memory. A clustering unit that clusters all characters in a column into clusters having similar transition characteristics and assigns a cluster code to each cluster; and obtains a cluster code of each character in the character string stored in the memory, Is characterized by including a two-cluster code transition probability table creation unit that creates a two-cluster code transition probability table by calculating the transition probabilities between the two-character cluster codes for all the two adjacent characters.

According to the above configuration, the number of characters and syllables to be recognized is compressed to the number of clusters. Therefore, the two-cluster code transition probability table that represents the transition probability between the cluster codes of two adjacent characters has a smaller number of elements than the two-character transition probability table that represents the transition probability between two adjacent characters. (Or the number of syllables) is compressed to ² , enabling low-memory capacity and high-speed stochastic language processing. Moreover, since characters and syllables belonging to the same cluster have similar transition characteristics, the linguistic transition information of each character and syllable is held in the two-cluster code transition probability table. Therefore, by using the multi-value two-cluster code transition probability table, it is possible to perform the stochastic language processing with high accuracy.

According to a second aspect of the present invention, in the probability table creating apparatus according to the first aspect of the present invention, an attribute of each character in the character string stored in the memory is obtained, and all adjacent attributes in the character string are determined. It is characterized in that a three-character attribute transition probability table creating unit for creating a three-character attribute transition probability table by calculating a transition probability between the attributes of the three characters for the three characters is provided.

According to the above configuration, a three-character attribute transition probability table representing transition probabilities between attributes of three adjacent characters in a natural language is created. Therefore, by performing the stochastic language processing using the two-character code transition probability table in combination with the three-character attribute transition probability table, each character or syllable lost when the number of all characters or all syllables to be recognized is compressed. Linguistic transition information is supplemented, and the accuracy of the stochastic language processing can be further improved.

According to a third aspect of the present invention, in the probability table creating apparatus according to the second aspect of the present invention, the occurrence probability of each one character with respect to all the characters in the character string stored in the memory is obtained to obtain one character appearance. Create a probability table 1
A character appearance probability table creation unit is provided.

According to the above configuration, a one-character appearance probability table representing the appearance probability of each character in a natural language is created. Therefore, by performing the stochastic linguistic processing using the one-character appearance probability table and the three-character attribute transition probability table in combination with the two-cluster code transition probability table, when the number of all characters or all syllables to be recognized is compressed, The lost linguistic transition information of each character or syllable is supplemented, and the accuracy of the stochastic language processing can be further improved.

According to a fourth aspect of the present invention, in the probability table creating apparatus according to the first aspect of the present invention, a logarithmic compression unit for logarithmically compressing each element value of the two-cluster code transition probability table is provided. And

According to the above configuration, each element value of the two-cluster code transition probability table is compressed so that it can be represented by one byte. In this way, the storage capacity is further reduced, and accordingly, the speed of the stochastic language processing can be further increased.

According to a fifth aspect of the present invention, in the probability table creating apparatus according to the first aspect of the present invention, the transition probability between all adjacent two characters in all the characters of the character string stored in the memory is obtained. A two-character transition probability table creating unit that creates a two-character transition probability table, and the clustering unit divides the two-character transition probability table into vectors of the number of characters to be recognized and the number of characters to be recognized. A cluster for creating a cluster code conversion table used when obtaining a cluster code of a cluster to which each of the two characters belongs based on the result of the clustering. The two-cluster code transition probability table creation unit includes a code conversion table creation unit. A cluster code conversion unit that converts each of the two characters into the cluster code by using a code conversion table, and creates the two-cluster code transition probability table by using the converted cluster code. It is characterized by:

According to the above configuration, a two-character transition probability table representing the transition probability between two adjacent characters in the character string is created. Then, based on the two-character transition probability table, characters and syllables having similar transition characteristics are clustered into the same cluster. Therefore, by adding a cluster code to each cluster and treating this cluster code as a character, the number of all characters and all syllables to be recognized is compressed to the above-mentioned number of clusters without losing the linguistic transition information of each character or syllable. Is done. Further, a cluster code conversion table used for obtaining the cluster code is created. Thus, the cluster code of the two adjacent characters is obtained by a simple process of referring to the cluster code conversion table, and the two cluster code transition probability table is created.

According to a sixth aspect of the present invention, in the probability table creating apparatus according to the second aspect of the present invention, there is provided the attribute conversion table used for obtaining the attribute of the character, and the three-character attribute transition probability table is created. The unit has an attribute conversion unit that converts the character into an attribute using the attribute conversion table, and creates the three-character attribute transition probability table using the converted attribute. And

According to the above configuration, the creation of the three-character attribute transition probability table is performed by a simple operation of referring to the attribute conversion table.

According to a seventh aspect of the present invention, in the probability table creating apparatus of the second aspect, the memory stores a Japanese sentence, the character is a Japanese character, and the attribute Is characterized by one of hiragana, katakana, symbol, kanji, numeral, uppercase alphabet, and lowercase alphabet.

According to the above configuration, the number of characters to be recognized is 40
In order to reach about 00, when the size of one element is 1 byte, the storage capacity of the two-character transition probability table for Japanese characters that requires 16 megabytes is, for example, 10 clusters.
By creating a 2-cluster code transition probability table by performing clustering to set the number of characters to 00 and compressing the number of characters, the data is compressed to 1 megabyte. Similarly, by converting the character to be recognized into any one of hiragana, katakana, symbol, kanji, numeral, uppercase alphabet and lowercase alphabet, and creating the above three-character attribute transition probability table, 3 gigabytes that require 64 gigabytes are required. The storage capacity of the character transition probability table is compressed to a maximum of 343 bytes. In this way, a probability table representing transition information between two adjacent characters and between three adjacent characters of low storage capacity that can be mounted on the Japanese character recognition device is created.

According to an eighth aspect of the present invention, in the probability table creating apparatus of the second aspect, the memory stores Chinese sentences, the characters are Chinese characters, and the attribute is: It is characterized by being any one of a supplementary character, a symbol, a kanji other than the supplementary character, a numeral, an uppercase alphabet and a lowercase alphabet.

According to the above configuration, even in the case of Chinese, where the number of characters to be recognized is extremely large, as in the case of Japanese, clustering is performed on the characters to be recognized and the number of characters is compressed to reduce the number of characters.
By creating the cluster code transition probability table, the storage capacity of the two-character transition probability table is compressed to, for example, 1 megabyte. Similarly, the target character is a supplementary character,
By converting to the attribute of any of Chinese characters other than symbols and supplementary characters, numbers, uppercase letters and lowercase letters, and creating the three-character attribute transition probability table,
The storage capacity of the three-character transition probability table is, for example, up to 216
Compressed to bytes. Thus, a probability table representing transition information between two adjacent characters and between three adjacent characters having a low storage capacity that can be mounted on the Chinese character recognition device is created.

According to a ninth aspect of the present invention, there is provided a stochastic language processing apparatus, wherein the two-cluster code transition probability table and the two-cluster code transition probability table created by the probability table creating apparatus according to the first aspect of the invention. And a character string transition probability calculation unit that calculates a character string transition probability of the input character string.

According to the above configuration, the two-cluster code transition probability table used in the stochastic language processing has a larger storage capacity than the conventional two-character transition probability table (the number of clusters / characters to be recognized (or (Syllable) Number) ² compressed. Therefore, even if Japanese characters whose number of characters to be recognized reaches about 4000 are to be processed, a storage capacity of 1 megabyte with one element as 1 byte is sufficient.

The invention according to claim 10 is the invention according to claim 9
The stochastic language processing device according to the invention according to claim 2, wherein
And at least one of the one-character appearance probability table and the three-character attribute transition probability table created by the probability table creation device according to the third aspect of the present invention. It is characterized in that the character string transition probability of the input character string is calculated using a table.

According to the above arrangement, the one-character appearance probability table and the adjacent three
A highly accurate character string transition probability is calculated by using at least one of the three character attribute transition probability tables, which are tables representing transition information between characters.

The invention according to claim 11 is based on claim 1.
0, the character string transition probability calculating unit calculates the character string transition probability by the first equation.

According to the above configuration, the character string transition probability is calculated only by referring to the two-cluster code transition probability table, the three-character attribute transition probability table, and the one-character appearance probability table, and by a very simple operation. Thus, the speed of the stochastic language processing operation is further increased.

According to a twelfth aspect of the present invention, there is provided a cut-out unit for cutting out individual language elements from an input language element sequence, and matching between a feature pattern of the cut-out language elements and a standard pattern to generate a plurality of recognition patterns. A matching unit that obtains a candidate, and a candidate character string generation unit that performs a stochastic language process on a plurality of candidate character strings obtained by combining the plurality of recognition candidates and generates a predetermined number of candidate character strings. In the recognition device, the two-cluster code transition probability table created by the probability table creation device according to the first aspect of the present invention is provided, and the candidate character string generation unit uses the two-cluster code transition probability table to create the candidate. It has a score calculation unit that calculates the score of the character string,
A probabilistic language process for evaluating the likelihood of the candidate character string based on the calculated score is performed.

According to the above configuration, the score used for evaluating the likelihood of a candidate character string in the above-described stochastic language processing is smaller in the number of elements than the conventional two-character transition probability table by (the number of clusters / It is calculated using the two-cluster chord transition probability table has been compressed in the recognition object character (or syllables) number) ^2. Therefore, the number of characters to be recognized is 4000
Even in the case of recognizing Japanese characters up to the extent, a memory capacity of 1 megabyte with one element as one byte is sufficient, and even if a dictionary for word processing language processing is installed, no adverse effect occurs.

The invention according to claim 13 is based on claim 1
The recognition device according to the second aspect of the present invention includes at least one of the one-character appearance probability table and the three-character attribute transition probability table created by the probability table creation device according to the second and third aspects of the invention. The score calculation unit is characterized in that the score of the candidate character string is calculated using the probability table as well.

According to the above configuration, at the time of the probability language processing, at least one of the one-character appearance probability table and the three-character attribute transition probability table is used together to more accurately determine the likelihood of the candidate character string. A representative score is calculated.

The invention according to claim 14 is based on claim 1
In the recognition device according to the second aspect of the invention, a predetermined number of candidate character strings generated by the candidate character string generation unit are subjected to word matching method language processing to recognize an optimal candidate character string in the input language element sequence. It is characterized by having a word collation language processing unit for outputting as a result.

According to the above arrangement, word matching method language processing by word dictionary search is performed from a plurality of candidate character strings selected by the probability method language processing, and an optimum candidate character string is obtained as a recognition result. can get. In this way, the input sentence and the input voice are more accurately recognized.

The invention according to claim 15 is the first invention.
In the recognition device according to a third aspect of the present invention, the score calculation section calculates the score by the second equation.

According to the above configuration, the score is calculated only by referring to the two-cluster code transition probability table, the three-character attribute transition probability table, and the one-character appearance probability table, and by a very simple calculation process. Thus, the speed of the recognition processing operation is increased.

The invention according to claim 16 is based on claim 1.
2. The recognition device according to claim 2, wherein the input language element string is a character string.

According to the above configuration, by using the two-cluster code transition probability table whose storage capacity is compressed by (number of clusters / number of characters to be recognized) ² compared to the conventional two-character transition probability table, a low memory Character recognition processing with high capacity and high recognition rate is performed at high speed.

A computer-readable recording medium according to a seventeenth aspect of the present invention provides a cutout unit for cutting out individual characters from an input character string, and performs matching between a feature pattern of the cutout character and a standard pattern. A matching unit that obtains a plurality of recognition candidates; and a two-cluster code transition probability created by the probability table creation device according to the first aspect of the invention, for a plurality of candidate character strings obtained by combining the plurality of recognition candidates. A character recognition program that causes a computer to function as a candidate character string generation unit that calculates a score for each of the above candidate character strings by performing stochastic language processing using a table, and generates a predetermined number of candidate character strings based on the score. Is recorded.

According to the above configuration, the character recognition processing with a low memory capacity and a high recognition rate is performed at high speed by using the two-cluster code transition probability table as in the sixteenth aspect.

The language element is a concept representing a character or a syllable that is an element of one natural language.

[0060]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the illustrated embodiments. The probability table creation device of the present embodiment is a compression diagram that compresses the number of characters or the number of syllables without impairing the linguistic transition relationship (rather than literally compressing the diagram, the number of characters to be recognized (or syllables to be recognized) Of a compressed character (or compressed syllable) code obtained by compressing the character code: a cluster code transition probability table, and one character (or one character)
A unigram, which is a table representing the probability of occurrence of (syllable),
An attribute trigram, which is a table representing the appearance probability of a combination of three character (or three syllable) attributes, is created. And
By using the above-mentioned compression diagram, low-capacity memory and high-speed processing of the stochastic language processing are realized. on the other hand,
The information lost due to the compression of the number of characters to be recognized (or syllables to be recognized) is compensated for by using the above unigram and attribute trigram together, and high-accuracy stochastic language processing is realized without impairing low-capacity memory and high-speed performance. is there.

A description will now be given of the above-described compressed diagram and its preparation method, the attribute trigram and its preparation method, and the character string probability calculation method, which form the basis of the present invention.
Note that the basic principle and basic configuration are the same for both characters and syllables, so the following description will be made using characters as a representative.

(A) Compression Diagram (Two-Cluster Code Transition Probability Table) The memory capacity of the above diagram increases in proportion to the square of the number of characters to be recognized. Conversely, if the number of characters to be recognized is reduced, the memory capacity is dramatically reduced. For example,
When the recognition target is 1000 characters, the memory capacity of the diagram is 10 if one byte is required for each element.
00 × 1000 = 1 megabyte, which is sufficiently feasible with such a memory capacity. Therefore, instead of compressing the diagram itself, the number of characters to be recognized may be compressed to about 1000 by some method. The compression diagram of the present invention has been conceived in view of this, and is created as follows.

First, the appearance probability (unigram) of one character is calculated from a large number of general documents (or the recognition target document if the recognition target document is limited), as shown in FIG.
The transition probability between characters (diagram) is obtained. Assuming that the number of character categories to be recognized is 4000, the diagram at this point has a capacity of 4000 × 4000 = 16 megabytes. Incidentally, the capacity of the unigram is 4000 = 4 kilobytes.

Next, each line (probability of transition from the character of interest to all characters) in the above diagram is assumed to be 4000-dimensional vector data, and the diagram is decomposed into 4000 vectors. Then, as shown in FIG.
The 00 vectors are clustered into a predetermined number (for example, 1000). As a clustering method in that case, a conventionally known k-means method, Ward method, or the like is used. The above clustering is
This is to represent a plurality of vectors having a short distance in one cluster.

The above-mentioned vector has a transition probability from a target character to all characters when a certain character is set as a target character as an element. Therefore, each vector (attention character) having a short distance between vectors (that is, classified into the same cluster by clustering) has a very close linguistic transition relationship. That is, even if a plurality of target characters (vectors) c1 to c4 classified into the same cluster are replaced with one compressed character code (cluster code) cc0, linguistic information loss is small. In this way, the linguistic information loss can be reduced and 4000 recognition target characters can be set to 1000.
It is compressed into compressed character codes.

After the completion of the clustering, a diagram (compressed diagram) in which the number of characters is once again compressed from a large number of documents is created using the obtained compressed character code. still,
In that case, it is necessary to convert characters from a large number of documents into a compressed character code (cluster code). Therefore, a correspondence table (compressed character code conversion table) between the compressed character codes and the character codes is created from the clustering result, and the input characters are converted into the compressed character codes using the compressed character code conversion table. is there. The compressed diagram obtained in this way has a memory capacity proportional to the square of the number of compressed character codes (the number of clusters: 1000),
When the number of characters to be recognized is 4000, it is 1 megabyte. That is, a sufficiently achievable memory capacity can be achieved. In addition, since the element value in this case is multi-valued, the transition probability between compressed character codes can be expressed with high accuracy.

(B) Attribute trigram (three-character attribute transition probability table) The attribute trigram supplements information lost during the creation of the above-mentioned compressed diagram (more precisely, the diagram obtained by compressing the number of characters to be recognized). As shown in FIG. 3, the character attributes according to the present embodiment include “genihi: hiragana”, “genka: katakana”, “genki: symbol”, “genkan: kanji”, “genus: number”, "Genus: Upper case alphabet", "Genus:
7 attributes of "lowercase alphabet". Therefore,
The memory capacity of the attribute trigram created using the above attributes is a cube of 7 bytes, that is, 343 bytes. This attribute trigram converts a character code of a character from a large number of documents into a character attribute code using a character attribute conversion table, as in the case of the compression diagram. Then, it is obtained by calculating a character attribute code transition probability between three adjacent characters.

The unigram, compressed digram and attribute trigram obtained as described above have a sufficiently practical total memory capacity of about 1 megabyte, and have high precision with little loss of linguistic information. It is a probability table. Therefore, by using these probability tables for language processing, high-accuracy, high-speed, and low-memory recognition processing can be performed.

(C) Character String Transition Probability Calculation Method In the stochastic language processing in the present invention, the following formulas are obtained from the equations (1) and (2) using the above-described probability tables of the unigram, the compressed digram and the attribute trigram. Approximate expression
The character string transition probability P (x) of the character string x is calculated by (3). Note that the character string transition probability used in the present invention is based on logarithmically compressed expression (1). Therefore, hereinafter, "log {P (x)}"
However, it is also assumed that a simple description “P (x)” also represents a log-compressed transition probability.

P (c ₁ , c ₂ ,..., C _n ) = P (c ₁ ) + P (c ₁ , c ₂ ) + P (c ₁ , c ₂ , c ₃ ) + P (c ₂ , c ₃ , c ₄ ) +... + P (c _n−2 , c _n−1 , c _n )… (3) + P (c _n−1 , c _n ) + P (c _n )

Here, the respective transition probabilities are calculated by the following approximation formula (4) using the unigram, the compressed diagram and the attribute trigram. P (x) = uni-gram (x) P (x, y ) = { uni-gram (x) + uni-gram (y)} / 2 + compression Daiguramu ^{(x c, y c) /} 2 P (x, y, z) = {unigram (x) + uni-gram (y) + uni-gram (z)} / 3 + {compression Daiguramu (x ^c, y ^c) + compressed Daiguramu ^{(y c, z c)}} / 2 + { attribute trigram (x ^a, y ^a , z ^a )} / 3 (4) where x, y, z: character code x ^c , y ^c , z ^c : compressed character code x ^a , y ^a , z of character code x, y, z ^a: character code x, y, z of the character attribute codes unigram (x): element value of a character code x in unigram compression Daiguramu (x ^c, y ^c): compression character code set of compression Daiguramu (x ^c, y ^c) the iodine value attribute trigram ^{(x a, y a, z} a): component value of the character attribute code set of attributes trigrams ^{(x a, y a, z} a)

More specifically, the character string “Today is xx
"To xx", P (to xxxx today) = P (book) + P (today) + P (today)
+ P (day) + P (ha) + ... + P (to hoop) + P (to hoop) + P (to), and the part of P (today) is as follows by equation (4). Become. P (today) = {unigram (book) + unigram (day) + unigram (wa)} / 3 + {compression diagram (book ^c , day ^c ) + compression diagram (day ^c , wa ^c )} / 2 + {attribute trigram (Book ^a , day ^a , is ^a )}
/ 3

In the expansion formula of the character string transition probability described above,
The part divided by “3” can be realized by a reduction table, and the part divided by “2” can be realized by bit shift. Therefore, the above-described arithmetic expression can be performed only by referring to and adding the table, and high-speed processing can be performed.

FIG. 1 is a functional block diagram of the probability table creating apparatus according to the first embodiment. This probability table creation device creates a probability table such as the unigram, the compressed digram, and the attribute trigram used in performing the stochastic language processing, and stores it in a memory. The present probability table creation device can be applied to any language, but for convenience of explanation, it is assumed that the language is Japanese and the number of characters to be processed is 4000 characters. It is assumed that all kanji codes are converted into internal codes (1 to 4000).

The document database 1 stores a large amount of general Japanese documents. The current character buffer 15, the previous character buffer 16, and the character buffer 17 store one character read from the document database 1, the character immediately before the character, and the character before the character. The unigram creator 19 assigns one to each character in the document database 1.
The appearance probability of the character is calculated and stored in the unigram buffer 2. The diagram creating unit 20 calculates the appearance probabilities (transition probabilities) of the two-character combinations for all the characters in the document database 1 and stores the calculated probabilities in the diagram buffer 4. Further, the attribute trigram creation unit 21 sends the document database 1
Probability of three-character attribute combination for all characters in
(Transition probability) is calculated and stored in the attribute trigram buffer 6.

When the number of characters to be recognized is clustered into k clusters and the number of characters is compressed as described later in detail, the compressed diagram generation unit 22 generates a compressed character code (cluster) to which all the characters in the document database 1 belong. With respect to (code), the appearance probability of the combination of two compressed character codes is calculated and stored in the compressed diagram buffer 8. The vector dividing unit 23 divides each row of the diagram into vector by assuming vector data as vector data, and stores each divided vector in the vector buffer 10. The clustering unit 24 clusters the vectors for all characters and stores the clustering result in the cluster buffer 11. The control unit 18 controls the unigram creating unit 19, the diagram creating unit 20, the attribute trigram creating unit 21, the compressed diagram creating unit 22, the vector dividing unit 23, the clustering unit 24, and the like to create the probability table.

The total character counter 3 counts the total number of characters in the document database 1. The total two-character combination counter 5 counts the total number of two-character combinations existing in the document database 1. The total three-character attribute combination counter 7 counts the total number of three-character attribute combinations existing in the document database 1. The total 2 compressed character code combination counter 9 counts the total number of 2 compressed character code combinations existing in the document database 1. Further, the character attribute conversion table 12 stores the document database 1
Referenced when converting the characters inside to character attributes. The compressed character code conversion table 13 is referred to when characters in the document database 1 are converted into compressed character codes (that is, when the number of characters in the document database 1 is compressed). The temporary memory 14 is used for temporarily storing data obtained in each of the above processes.

The probability table creating apparatus having the above configuration operates as follows under the control of the control unit 18 to create a probability table. FIGS. 4 and 5 are flowcharts of the probability table creation processing operation executed by the control unit 18. Hereinafter, the procedure for creating the probability table will be described with reference to FIGS.

In step S1, all buffers and all counters are cleared to "0". In step S2, one character is read from the document database 1 and the current character buffer 1
5 is stored. In step S3, it is determined whether or not a character exists in the document database 1. As a result, if there is a character, the process proceeds to step S4, and if there is no character, the process proceeds to step S14.

In step S4, the total character counter 3
Counts up the total number of characters. Step S
At 5, the unigram calculation is performed by the unigram creating unit 19. The calculation of the unigram is performed by incrementing the element value (the number of appearances) of the unigram buffer 2 corresponding to the current character stored in the current character buffer 15.

In step S6, the previous character buffer 16
It is determined whether the character code of the previous character is stored in. If the result is stored, the process proceeds to step S7,
Otherwise, go to step S9. In step S7, the total two-character combination number is counted up by the total two-character combination number counter 5. In step S8, the diagram creation unit 20 performs a diagram calculation. The calculation of the diagram is performed by incrementing the element value (number of appearances) of the diagram buffer 4 corresponding to the combination of the two characters stored in the previous character buffer 16 and the current character buffer 15.

In step S9, the character buffer 1
It is determined whether or not the character code of the character before 2 is stored in 7. If the result is stored, the process proceeds to step S10; otherwise, the process proceeds to step S13. Step S10
Thus, the character codes stored in the current character buffer 15, the previous character buffer 16, and the character buffer 17 before are converted into character attribute codes. This character attribute conversion is directly converted from a character code (all Chinese character codes are internally coded from 1 to 4000) by using the character attribute conversion table 12 described above. The character attribute conversion table 12 has a structure in which the character attribute code of the character code can be referred to using the character code as an offset address, and requires a capacity of (number of characters to be recognized × 1 = 4000) bytes. Note that this character attribute conversion table 12 is
Create it in advance.

In step S 11, the total number of three-character attribute combinations is counted by the total three-character attribute combination number counter 7. In step S12, the attribute trigram creating unit 21
Performs attribute trigram calculation. The attribute trigram is calculated by calculating the element value (the number of appearances) of the attribute trigram buffer 6 corresponding to the combination of the character attributes of the three characters stored in the character buffer 17, the previous character buffer 16 and the current character buffer 15 before and after. This is done by incrementing.

In step S13, the previous character buffer 16
And the contents of the current character buffer 15 and the
7 and the contents of the previous character buffer 16 are updated. After that, the process returns to step S2 to shift to the process for the next one character. If it is determined in step S3 that there are no more characters in the document database 1, in step S14, the unigram creation unit 19, the diagram creation unit 20 and the attribute trigram creation unit 21 use the elements of the unigram, diagram and attribute trigram. The value (number of occurrences) is converted to a probability value. The conversion into the probability value is performed by dividing each element value of the unigram, the digram or the attribute trigram by the count value of the total number of characters counter 3, the total number of 2 characters combination counter 5, or the total number of 3 characters attribute combination counter 7. It is carried out.

By the way, each element value of the unigram, the diagram and the attribute trigram calculated as described above
The (probability value) takes a value after the decimal point and has a significant number of significant digits. Therefore, if the data is tabulated for probabilistic language processing as it is, the memory capacity increases, and the calculation speed decreases because multiplication is used for calculating the character string transition probability. Then, in step S15, the unigram buffer 2, converted into the probability value in step S14,
Diagram buffer 4 or attribute trigram buffer 6
Is subjected to logarithmic compression, and the corresponding logarithmically compressed value is used to update the corresponding element value.

The logarithmic compression is performed by the following equation (5). lnP (x) = min (255, (− log (P (x)) − MIN)) × 256 / (MAX−MIN) (5) where P (x) is a probability value of an arbitrary character x. Yes, P (x)>
Take the value 0. If P (x) = 0, lnP (x) = 2
55. Note that min (a, b) is a function that takes the smaller of a and b. Further, MIN and MAX are the minimum value and the maximum value of the probability value after the logarithmic compression, and are “0” and “18” in the present embodiment.

According to the above formula (5), “1-0.0000000”
Probability value P in the range of "00000000000001"
(x) is converted to 1-byte data from 0 to 255. It should be noted that a low probability value (including -log (0) which becomes infinity) in which -log (P)> MAX (= 18) or more is suppressed by a maximum value "255" that can be expressed by 1 byte.

Normally, the larger the probability value is, the more likely it is. However, the probability value in the present embodiment is an evaluation value indicating that the smaller the value is, the more likely it is due to logarithmic transformation.

When the above-mentioned diagram is obtained, the processing shifts to the calculation process of the above-mentioned compressed diagram.

In step S16, the vector dividing unit 23
Thus, the following vector division processing is executed. That is, the diagram stored in the diagram buffer 4 is converted into a 4000 × 4000 two-dimensional array (4000 × 4000).
(4000 matrices), each row is divided into 4000 vectors as a 4000-dimensional vector, and stored in the vector buffer 10. This vector is a 4000-dimensional vector having a transition probability from the target character to all characters as an element.

In step S17, the clustering unit 2
4, the clustering is performed on the 4000 divided vectors using a clustering method such as the k-means method or the Ward method. In this case, the maximum number of clusters set in this case is “1”.
000 ". The obtained cluster is stored in the cluster buffer 11.

In step S18, a compressed character code conversion table 13 used for converting the character code into a compressed character code is created. This compressed character code conversion table 13 is created as follows. That is,
The character codes c1 to c4, c5 to c of the target characters of each vector collected for each cluster by the above clustering
The compressed character code conversion table 13 is created by creating a table in which 8,..., C3997 to c4000 are associated with the cluster codes of the cluster (that is, the compressed character codes cc0 to cc999). As in the case of the character attribute conversion table 12, the compressed character code conversion table 13 uses the character code (1 to 4000) as an offset address so that the compressed character of the character code can be directly converted from the character code to the compressed character code. The structure allows the code to be referenced. This compressed character code conversion table 13
Has a memory capacity of 4000 × 2 = 8 kilobytes.

In step S19, the above-mentioned compressed diagram buffer 8, all-two-compressed character code combination counter 9, current character buffer 15, and previous character buffer 16 are cleared to "0". In step S20, the document databases 1 to 1
Characters are read and stored in the current character buffer 15.
In step S21, it is determined whether a character exists in the document database 1. As a result, if there is a character, step S
Proceed to 22; if there are no characters, proceed to step S27.

In step S22, the previous character buffer 16
It is determined whether or not the previous character is stored in. As a result, if there is a previous character, the process proceeds to step S23, and if there is no previous character, the process proceeds to step S26. In step S23, the character codes stored in the current character buffer 15 and the previous character buffer 16 are converted into compressed character codes. In this compressed character code conversion, the character code is directly converted into the compressed character code using the compressed character code conversion table 13.

In step S24, the total number of 2-compressed character code combinations is counted by the total 2-compressed character code combination number counter 9. In step S25, a compression diagram calculation is performed by the compression diagram creation unit 22. The calculation of the compressed diagram is performed by incrementing the element value (number of appearances) of the compressed diagram buffer 8 corresponding to the combination of the two compressed character codes obtained in step S23.

In step S26, the current character buffer 15
The content of the previous character buffer 16 is updated with the content of After that, the process returns to step S20 to shift to the process for the next one character. Then, when it is determined in step S21 that there are no more characters in the document database 1, in step S27, the element value (number of appearances) of the compressed diagram is converted into a probability value by the compressed diagram creation unit 22. The conversion into the probability value is performed by dividing each element value of the above-mentioned compressed diagram by the count value of the total 2 compressed character code combination counter 9. Then, as in the case of the unigram, the digram and the attribute trigram, in step S28,
Compressed diagram buffer 8 converted to a probability value in
Are subjected to logarithmic compression, and the corresponding element values in the compressed diagram buffer 8 are updated with the obtained logarithmically compressed values. When the compressed diagram is created in this way, the probability table creation processing operation ends.

The memory size of each probability table thus created is as shown in FIG. 6, and the above unigram, compressed diagram, attribute trigram, character attribute conversion table 1
2, the total memory capacity of the compressed character code conversion table 13 is about 1 megabyte, which is very compact.

As described above, in the present embodiment, the total number of all character categories in the document database 1 is counted by the total number of characters counter 3, and the total number of two-character combinations is counted by the total two-character combination number counter 5. The three-character attribute combination number counter 7 counts the total number of three-character attribute combinations.

The unigram creation unit 19 calculates the number of appearances of all characters and updates the corresponding element value in the unigram buffer 2. When there are no more characters in the document database 1, the unigram creation unit 19 updates the total character counter 3. The element value (number of appearances) of the unigram buffer 2 is converted into a probability value using the count value. The diagram creating unit 20 calculates the number of appearances of all two-character combinations, updates the corresponding element values in the diagram buffer 4, and when there are no more characters in the document database 1, the counter of the two-character combination number counter 5 The element value (number of appearances) of the diagram buffer 4 is converted into a probability value using the count value. The attribute trigram creating unit 21 calculates the number of appearances of all three-character attribute sets based on the character attribute codes obtained by converting the character codes of the character before the previous character, the previous character, and the current character, and calculates the corresponding element value of the attribute trigram 6. When there are no more characters in the document database 1 after updating, the element value (number of appearances) of the attribute trigram buffer 6 is converted into a probability value using the count value of the all three character attribute combination number counter 7. Further, each element value of the unigram buffer 2, the digram buffer 4 and the attribute trigram buffer 6 is logarithmically compressed to form the above-mentioned unigram, digram and attribute trigram.

In this way, when the above-mentioned diagram is obtained, the vector dividing section 23 makes 4000 × 4000.
Is divided into 4000 4000-dimensional vectors. Then, the clustering unit 24
Performs clustering on each vector,
A compressed character code conversion table 13 in which character codes belonging to each cluster are associated with cluster codes (compressed character codes) is created. After that, each time one character is read from the document database 1, the current character buffer 15 and the previous character buffer 16 are updated. Further, the character codes of the current character and the previous character are converted into compressed character codes using the compressed character code conversion table 13, and the total number of 2-compressed character code combinations is counted by the total 2-compressed character code combination counter 9. Then, the compressed diagram creation unit 22 creates a compressed diagram in the same manner as in the above-described diagram creation.

The number of elements of the compressed diagram thus created is reduced to 1/16 without losing the linguistic transition information of the diagram. Therefore, by performing linguistic processing using the compressed diagram, low storage capacity and high speed can be realized without impairing linguistic information. The linguistic transition information lost due to the compression of the number of characters to be recognized is supplemented by a combination of a unigram, which is the appearance probability of one character, and an attribute trigram, which is a character attribute code transition probability between three adjacent characters. This enables high-accuracy stochastic language processing without impairing low storage capacity and high-speed performance.

That is, according to the present embodiment, even for a language having a large number of characters to be recognized, such as Japanese, a stochastic language using a high-precision multi-value probability table with a low memory capacity of about 1 megabyte. Processing can be performed.
In addition, the character string probability calculation in that case can be made a very simple calculation of addition and table reference, and high-speed processing is possible. Therefore, it is possible to realize a recognition device capable of low memory capacity, high recognition rate, and high-speed processing.

FIG. 7 is a block diagram of a recognition device according to the second embodiment. This recognition device has a stochastic language processing unit using the above-mentioned probability table created in the first embodiment, and recognizes characters from input document image data. Although the present recognition apparatus can be applied to any language, for convenience of explanation, it is assumed that the language is Japanese, the number of characters to be recognized is 4000 characters, and Japanese type OCR using similarity as a recognition evaluation value. It is assumed that all kanji codes are converted into internal codes (1 to 4000).

The image input unit 31 takes in document image data from the scanner 47 or the image file 48 and stores it in the image memory 49. The cutout unit 32 is an area cutout unit 33 that cuts out an area for character recognition based on the image data of one document stored in the image memory 49, a line cutout unit 34 that cuts out a one-line character string, and a character cutout that cuts out one character. And a contact character cutout unit 36 for forcibly separating contact characters. The obtained coordinates of the region, line and character on the image memory 49 are stored in the region coordinate memory 50, the line coordinate memory 51 and the (temporary) character coordinate memory 52, respectively. The peripheral distribution feature buffer 53 stores peripheral distribution characteristics in a row direction and a column direction, which will be described in detail later. The feature extracting unit 37 extracts a feature pattern based on the extracted one-character image data and stores the feature pattern in the feature memory 54.

The matching unit 38 calculates the extracted 1
A first similarity calculating unit 39 that calculates a similarity by performing matching between a character feature pattern and a standard pattern of a recognition target character stored in the pattern dictionary 55, and sorts the calculated similarities in descending order. First sorting section 4 for selecting a predetermined number of character candidates to obtain recognition result candidates
Has zero. Note that the obtained recognition result candidates are stored in the recognition result candidate buffer 56.

Each time a recognition result candidate for one character constituting one line is obtained, the character string generation unit 41 combines all the recognition result candidates up to the character and develops the candidate character string into a candidate character string. Unit 42, a score calculation unit 43 that calculates the score of the obtained candidate character string using the above-described table 58 such as the unigram, compressed diagram, attribute trigram, and the like, and sorts the calculated scores in descending order to obtain a predetermined number of candidates. It has a second sorting unit 44 for selecting a character string. The expanded candidate character string is stored in the candidate character string buffer 5.
7 is stored.

The word collation language processing unit 45 uses the word collation language dictionary 59 for the predetermined number of candidate character strings for one line stored in the candidate character string buffer 57 and uses the word collation language processing 59 I do. Then, the content of the recognition result buffer 60 is updated by obtaining the final recognition result based on the result of the word collation type language processing. The control unit 46 controls the image input unit 31, the extraction unit 32, the feature extraction unit 37, the matching unit 38, the character string generation unit 41, and the word matching language processing unit 45 to execute a character recognition process.

That is, in the present embodiment, the above-mentioned stochastic language processing means is constituted by the above-mentioned score calculation section 43 and the second sorting section 44.

The recognition device having the above configuration operates as follows under the control of the control section 46 to perform a character recognition process.
FIG. 8 is a flowchart of the character recognition processing operation executed by the control unit 46. Hereinafter, the character recognition procedure will be described with reference to FIG.

In step S 31, image data of one document is fetched from the scanner 47 or the image file 48 by the image input unit 31 and stored in the image memory 49. In step S32, the entire area for character recognition is extracted from the one document by the area extracting unit 33. Alternatively, the entire area is specified. Then, the coordinates of the clipped region or the designated region on the image memory 49 are stored in the region coordinate memory 50. In step S33, image data of one area is read. Step S34
It is determined whether or not there is an area. As a result, if there is a region, the process proceeds to step S35; otherwise, the process proceeds to step S49.

In step S35, the row cutout section 34 cuts out all rows from the area. In this case, the method of cutting out the rows is not particularly limited. For example, the number of pixels having a predetermined brightness or more in the row direction is counted based on the image data of the area, stored as the peripheral distribution feature in the peripheral distribution feature buffer 53, and cut out based on the peripheral distribution feature. In step S36, it is determined whether or not the line cut out in step S35 exists. As a result, if there is no line, the flow returns to step S33 to shift to the processing of the next area, and if there is a line, the flow proceeds to step S37.

In step S37, one row of image data is read. In step S38, the character cutout unit 3
5, temporary character segmentation is performed based on the read one line of image data. Note that the processing of the above “temporary character cutout” does not always perform the correct character cutout. For example, in the case of the kanji “ka”,
This is a process of extracting all portions that can be cut out as one character, such as “add” or “f” and “b”. In this case, the method of temporary character cutout is not particularly limited. For example, based on the image data of the row, the number of pixels having a predetermined brightness or more in the column direction is counted and stored in the peripheral distribution feature buffer 53 as the peripheral distribution feature, and a cutout portion is extracted based on the peripheral distribution feature. I do. Further, the contact character cutout unit 36 extracts a portion that can be forcibly separated from a character block such as a contact character by using the column-direction peripheral distribution feature stored in the peripheral distribution feature buffer 53.

In step S39, the candidate character string buffer 57 is initialized. In step S40, it is determined whether a character exists in the line. As a result, if there is a character, the process proceeds to step S41, while if there is no character, the process proceeds to step S41.
Go to 47. In step S41, one character is actually cut out from the line by the character cutout unit 35 based on the provisional character cutout result.

In step S42, the feature extraction unit 37 extracts the feature pattern of the extracted character candidate. In step S43, the first similarity calculator 39 calculates the characteristic pattern of the character candidate and the pattern dictionary 5
The matching with the standard pattern of the recognition target character stored in No. 5 is performed to calculate the similarity. Further, the first sorting unit 40 sorts the calculated similarities in descending order, and selects a predetermined number (in this embodiment, “10”) of character candidates. The character candidates thus obtained are stored in the recognition result candidate buffer 56.

Thereafter, the character candidates stored in the recognition result candidate buffer 56 are combined and developed into a character string, and stored in the candidate character string buffer 57. Here, it is impossible to store all combinations of character candidates in the candidate character string buffer 57 because memory capacity for 10 (the total number of characters in one line) candidate character strings is required. is there.
Further, the subsequent processing speed is significantly reduced. Therefore, in the present embodiment, the scores of all candidate character strings are obtained as follows, and the number of candidate character strings is reduced based on this score.

In step S44, the candidate character string developing unit 4
2, all character candidates stored in the recognition result candidate buffer 56 are combined and developed into a candidate character string. In step S45, the score calculation unit 43
The scores of all the obtained candidate character strings are calculated.
The calculation of this score is performed according to equation (6). Score _{(1) = W S S 1} + W P P (c 1) score _{(2) = W S {(} S 1 + S 2) / 2} + W P P (c 1, c 2) score (i) = [score (i-1) × (i -1) + {W S S i + W P P (c i-2, c i-2, c i)} ] / i ... (6) where score (I): character Score c ₁ , c ₂ , c ₃ ,..., C _{n of} candidate character strings with the number of candidates I: candidate character strings S ₁ , S ₂ , S ₃ ,..., _Sn : similarity P ( x): Probability of character string transition of candidate character string x (calculated by equation (4) of the first embodiment) W _P , W _S : Weights Note that the weights W _P , W _S are W _S > W _P. It is desirable to have a relationship.

The score in the present embodiment is obtained by calculating the similarity S calculated by the second similarity calculator of the score calculator 43.
_This is a weighted sum of _n and the transition probability P (x) calculated by the probability calculation unit. Therefore, even when using the recognized evaluation value and the evaluation measure other than the transition probability P (x) as described above similarity S _n, the equation (6), a value obtained by multiplying the weight on the value of the rating scale If added as the third term, it becomes possible to execute score calculation using the above-mentioned unigram, compressed diagram and attribute trigram.

In step S46, the above-mentioned calculated candidate character strings are sorted by the second sorting section 44 in descending order of the score, and the predetermined character strings are sorted in descending order of the score.
(In the present embodiment, “100”) candidate character strings are selected. The 100 candidate character strings thus selected are stored in the candidate character string buffer 57. After that, the process returns to step S40, and the process proceeds to cutting out the next one character. Thereafter, until there are no more characters in the line, character extraction from the line, character extraction of the extracted character, matching of the extracted character, generation of a candidate character string based on the already obtained character candidate and the recognition candidate of the extracted character (Score calculation and candidate reduction processing) are sequentially performed. If it is determined in step S40 that there are no more characters in the line, the process proceeds to step S47.

In step S 47, the word collation language processing unit 45 uses the word collation language dictionary 59 for the 100 candidate character strings for one line stored in the candidate character string buffer 57. The collation language processing is executed. In step S48, an optimal candidate is selected from among the 100 candidate character strings for one line stored in the candidate character string buffer 57 based on the score and the number of words existing in the word matching language dictionary 59. A character string is selected and stored in the recognition result buffer 60.

Thereafter, the flow returns to step S36, and shifts to the processing of the next line. Thereafter, until there are no more lines in the area, character extraction from each line is sequentially performed, character extraction of extracted characters, matching of the extracted characters, generation of candidate character strings based on already obtained character candidates and recognition candidates for the extracted characters. (Score calculation and candidate reduction processing), word matching type language processing, and selection of an optimal candidate character string are sequentially performed. If it is determined in step S36 that there is no line in the area, and if it is determined in step S34 that there is no area, the process proceeds to step S49. Step S49
Then, the optimal candidate character strings of all the rows stored in the recognition result buffer 60 are output as the recognition results. Thereafter, the character recognition processing operation ends.

As described above, in the present embodiment, the recognition device (Japanese type OCR) is equipped with the probability table (unigram, compressed diagram, and attribute trigram) created by the probability table creation device in the first embodiment. I do.
Then, one character is cut out by the cutout unit 32 from the image data of the document captured by the image input unit 31, a feature pattern is extracted by the feature extraction unit 37, and the matching unit 3
By using 8, 10 character candidates are obtained. Thus, when ten character candidates are obtained for all the characters in one line, the candidate character string developing unit 42 combines all character candidates and develops them into candidate character strings.

Then, the score calculation section 43 calculates scores for all the obtained candidate character strings using the probability table according to the equation (6). Then, the second sorting unit 44 sorts all the candidate character strings in descending order of the score, and selects 100 candidate character strings in descending order of the score.

As described above, by performing the above-mentioned stochastic linguistic processing, from among a large number of candidate character strings generated by expanding character candidates obtained ten by ten from all the characters in one line,
Regardless of word segmentation (morphological analysis), it is likely to be 10
Zero candidate strings are quickly selected. The probability table used in that case is composed of the unigram, the compressed diagram and the attribute trigram in the first embodiment. Therefore, the stochastic language processing executed in the present embodiment is performed at high speed and with high accuracy with a small memory capacity.

Thereafter, the word collation language processing unit 45 performs a word collation type language process using the word collation language dictionary 59 on the selected one hundred candidate character strings for one line to optimize A candidate character string is selected as a recognition result.

That is, according to the present embodiment, even for recognition of a language having a large number of characters to be recognized, such as Japanese,
A stochastic language process using a high-precision multi-value probability table with a low memory capacity of about 1 megabyte can be applied, and a recognition device capable of low-capacity memory, high recognition rate, and high-speed processing can be realized.

In the first embodiment, the case where the present invention is applied to Japanese characters is described as an example, but the present invention can be applied to any language. In particular, it is effective for languages with a large number of characters, such as Japanese and Chinese. When a probability table for Chinese recognition is created, for example, supplementary characters, symbols, kanji other than supplementary characters, numbers, uppercase letters and lowercase letters may be used as the character attributes.

Further, the character string probability calculation using the above-described probability table created in the first embodiment is effective in determining the likelihood of a kanji-kana mixed character string. Application to a processing device is also possible. Further, in the first embodiment, the probability table used for character recognition and kana-kanji conversion based on the Japanese document stored in the document database 1 is created. It is also possible to create In this case, a Japanese kana character string may be stored in the document database 1, and the compressed digram, unigram, and attribute trigram may be created using the kana character string.

The probability table according to the second embodiment may be created by mounting the probability table creation device according to the first embodiment on a recognition device and using the probability table creation device. Alternatively, one created by another probability table creation device may be installed in the present recognition device. Further, it goes without saying that the second embodiment is applicable not only to Japanese character but also to a foreign character recognition device. Further, the present invention can be applied not only to character recognition but also to voice recognition.

Further, in the first and second embodiments, all of the above-mentioned compressed diagram, unigram and attribute trigram are created or mounted, but in the present invention, at least the above-mentioned compressed diagram is created and loaded. It may be created or mounted.

In the probability table creation device according to the first embodiment, the program for the probability table creation processing is stored in the ROM (read / write) by any of the following methods.
It is stored in an only memory (not shown) or a random access memory (RAM) (not shown). (A) It is stored in the ROM in advance. (B) Part or all of the probability table creation processing program is stored in a recording medium such as a floppy disk or a hard disk device, and the program is installed in the RAM as needed. (C) Install the probability table creation processing program into the RAM from the computer network.

Similarly, in the recognition apparatus according to the second embodiment, the program for the character recognition processing is loaded into a ROM (Read Only Memory) (not shown) or a RAM (Random Access Memory) by any of the following methods. • stored in a memory (not shown). (A) It is stored in the ROM in advance. (B) A part or all of the character recognition processing program is stored in a recording medium such as a floppy disk or a hard disk device, and the program is installed in the RAM as needed. (C) Install the character recognition processing program into the RAM from a computer network.

[0132]

As is apparent from the above description, the probability table creating apparatus according to the first aspect of the present invention uses the clustering unit to perform clustering based on transition characteristics for all characters of a character string stored in a memory. Since the cluster code is assigned to each cluster, the number of characters and syllables to be recognized is compressed to the number of clusters. A cluster code diagram showing the transition probabilities between the cluster codes of all two adjacent characters in the character string stored in the memory by a two-cluster code transition probability table (hereinafter, referred to as a cluster code diagram) creation unit. So that the above cluster code diagram is adjacent 2
The number of elements can be reduced to (the number of clusters / the number of characters (or syllables) to be recognized) ² as compared with the conventional diagram showing the transition probability between characters. Therefore, a probability table having a low storage capacity can be created, and a low-memory capacity and high-speed stochastic language process can be performed.

In this case, since characters and syllables belonging to the same cluster have similar transition characteristics, the cluster code diagram stores linguistic transition information of each character and syllable. Therefore, a high-precision multi-value cluster code diagram can be created while suppressing a decrease in accuracy due to compression of the number of all characters or all syllables to be recognized. Therefore, it is possible to perform stochastic language processing with high accuracy.

The probability table creating apparatus according to the second aspect of the present invention uses a three-character attribute transition probability table (hereinafter referred to as an attribute trigram) creating unit to generate all three adjacent character strings in the character string stored in the memory. Creates attribute trigrams that represent the transition probabilities between character attributes, so three adjacent characters are used to supplement the linguistic transition information of each character or syllable lost when compressing the number of all characters or all syllables to be recognized. It is possible to obtain a probability table representing transition information between the two. Therefore, by using the attribute trigram in combination with the cluster code diagram, it is possible to perform stochastic language processing with higher accuracy.

In the probability table creating apparatus according to the third aspect of the present invention, a one-character appearance probability table (hereinafter, referred to as a unigram) creating unit represents the appearance probability of each character in the character string stored in the memory. To create a unigram, obtain a probability table showing the appearance information of each character or syllable to supplement the linguistic information of each character or syllable lost when compressing the number of all characters or syllables to be recognized Can be. Therefore, by using the unigram and the attribute trigram together with the cluster code diagram, it is possible to perform a stochastic language processing with higher accuracy.

Further, in the probability table creating apparatus according to the present invention, each element value of the cluster code diagram is logarithmically compressed by the logarithmic compression unit, so that the data length of each element value is represented by 1 byte. Can be compressed to the possible length. Therefore, it is possible to further reduce the storage capacity of the cluster code diagram, and to further increase the speed of the stochastic language processing.

Further, the probability table creation device of the invention according to claim 5 is characterized in that the two-character transition probability table creation unit uses
A diagram is created based on the character strings stored in the memory, and the clustering unit divides the diagram into vectors of the number of characters to be recognized, the number of characters to be recognized, and divides all obtained vectors into a predetermined number of clusters. Therefore, characters and syllables having similar transition characteristics can be clustered into the same cluster. Therefore, by adding a cluster code to each cluster and treating this cluster code as a character, the number of all characters or all syllables to be recognized is converted to the number of clusters without losing the linguistic transition information of each character or syllable. Can be compressed.

Further, a cluster code conversion table is created by the cluster code conversion table creation section based on the clustering result, and the character string is converted into the cluster code by the cluster code conversion table using the cluster code conversion table. Since the cluster code / diagram creating unit creates the cluster code / diagram using the converted cluster code, the cluster code / diagram is obtained by a simple process of referring to the cluster code conversion table to obtain the cluster code / diagram. Can be created.

According to a sixth aspect of the present invention, the probability table creating apparatus converts the character string into an attribute by using an attribute converting table by an attribute converting unit, and the attribute trigram creating unit converts the converted attribute into an attribute. Since the attribute trigram is created using the attribute, the attribute trigram can be created by obtaining the attribute by a simple operation of referring to the attribute conversion table.

Further, in the probability table creating apparatus according to the present invention, a Japanese sentence is stored in the memory, the characters are Japanese characters, and the attributes are hiragana, katakana, symbol, kanji, Since any one of a number, an uppercase letter and a lowercase letter is used, a diagram for Japanese characters that requires 16 megabytes when one element is 1 byte is compressed into the above-mentioned cluster number 1000 and the above cluster code diagram. By creating, the storage capacity can be compressed to 1 megabyte. Similarly,
By converting the character to be recognized into the attribute and creating the attribute trigram, the storage capacity of the trigram requiring 64 gigabytes can be compressed to a maximum of 343 bytes.
That is, according to the present invention, it is possible to create a probability table representing transition information between two adjacent characters and between three adjacent characters having a low memory capacity that can be mounted on the Japanese character recognition device.

Further, in the probability table creating apparatus according to the present invention, a Chinese sentence is stored in the memory, the character is a Chinese character, and the attribute is other than a supplementary character, a symbol, and a supplementary character. Since it is any of Chinese characters, numbers, uppercase letters and lowercase letters, even in the case of Chinese, where the number of characters to be recognized is extremely large, as in Japanese, clustering is performed on the characters to be recognized and the number of characters is compressed. By creating a cluster code diagram, the storage capacity of the diagram can be reduced to, for example, 1 megabyte. Similarly, by converting the recognition target character into the attribute and creating the attribute trigram, the storage capacity of the trigram can be compressed, for example, to a maximum of 216 bytes. That is, according to the present invention, it is possible to create a probability table representing transition information between two adjacent characters and three adjacent characters with a low memory capacity that can be mounted on the Chinese character recognition device.

A stochastic language processing apparatus according to a ninth aspect of the present invention has the cluster code diagram created by the probability table creating apparatus according to the first aspect of the present invention, and is provided with a character string transition probability calculating section. Since the storage capacity of the input string is calculated using the above-mentioned cluster code / diagram compressed by ² (the number of clusters / the number of characters (or syllables) to be recognized) compared to the conventional diagram, the recognition target Even when Japanese characters whose number of characters reaches about 4000 are to be processed, 1 element is regarded as 1 byte and 1 byte.
All you need is a megabyte of memory.

A stochastic language processing apparatus according to a tenth aspect of the present invention provides a stochastic language processing apparatus according to the second and third aspects of the present invention, which stores at least one of the unigram and attribute trigram probability tables created by the probability table creating apparatus. The character string transition probability calculation unit calculates the character string transition probability of the input character string using the probability table as well, so that at least one of the unigram and the attribute trigram is used in combination, and the character string transition probability is calculated with high accuracy. Character string transition probability can be calculated.

Further, in the stochastic language processing apparatus according to the eleventh aspect, the character string transition probability calculating section calculates the character string transition probability by the first equation.
The character string transition probability can be calculated only by referring to the cluster code diagram, the attribute trigram, and the unigram, and by very simple arithmetic processing. Thus, the speed of the stochastic language processing operation can be increased.

A recognition apparatus according to a twelfth aspect of the present invention obtains a plurality of candidate character strings by combining a plurality of recognition candidates obtained as a result of matching with a language element cut out from an input language element string. A score calculation unit calculates a score of the candidate character string using the cluster code diagram created by the probability table creation device according to the first aspect of the present invention, and the calculated score is calculated by the candidate character string generation unit. Since a predetermined number of candidate character strings are generated by performing a stochastic linguistic process that evaluates the likelihood of the candidate character strings based on the
The probabilistic linguistic processing can be performed on the candidate character string using the cluster code diagram that has been compressed by (number) ² . Therefore, the number of characters to be recognized is 4
Even when recognizing about 000 Japanese characters, 1
It is sufficient that the storage capacity of 1 megabyte is 1 byte for each element. Even if a dictionary for word processing language processing is installed, no harm is caused.

A recognition device according to a thirteenth aspect of the present invention has a probability table of at least one of the unigram and the attribute trigram created by the probability table creation device of the second and third aspects of the invention. Since the score calculation unit calculates the score of the candidate character string by using the probability table as well, the probability calculation language processing using at least one of the unigram and the attribute trigram is performed, and the probability of the candidate character string is determined. Can be calculated more accurately.

According to a fourteenth aspect of the present invention, the word matching language processing section performs word matching language processing on the predetermined number of generated candidate character strings to obtain an optimal candidate character string. Is output as a recognition result of the input language element sequence, so that an optimal candidate character string can be obtained as a recognition result by word dictionary search. Therefore, the input sentence and the input voice can be more accurately recognized.

Since the score calculation unit in the recognition device of the invention according to claim 15 calculates the score by the second equation, it is very easy to refer to the cluster code diagram, attribute trigram and unigram. The above-mentioned score can be calculated only by simple arithmetic processing. Therefore, the speed of the recognition processing operation can be increased.

In the recognition apparatus according to the sixteenth aspect, since the input language element sequence is a character string, the storage capacity is (number of clusters / number of characters to be recognized) as compared with the conventional two-character transition probability table. By using the cluster code diagram compressed by ^two , character recognition processing with a low memory capacity and a high recognition rate can be performed at high speed.

The computer readable recording medium according to the seventeenth aspect of the present invention provides a cutout unit for cutting out individual characters from an input character string, and performs matching between a feature pattern of the cutout characters and a standard pattern. A matching unit that obtains a plurality of recognition candidates; a plurality of candidate character strings obtained by combining the plurality of recognition candidates; and a predetermined unit based on a score of each of the candidate character strings calculated using the cluster code diagram. 2. A character recognition program that causes a computer to function as a candidate character string generation unit that generates a number of candidate character strings.
The same effect as that of No. 6 can be obtained.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a probability table creation device according to the present invention.

FIG. 2 is an explanatory diagram of a method of creating a compressed diagram from a diagram by a compressed diagram creating unit, a vector dividing unit, and a clustering unit in FIG. 1;

FIG. 3 is a diagram illustrating an example of character attributes of Japanese characters.

FIG. 4 is a flowchart of a probability table creation processing operation executed by a control unit in FIG. 1;

FIG. 5 is a flowchart of a probability table creation processing operation following FIG. 4;

FIG. 6 is a diagram showing a memory size of a table created by the probability table creating device shown in FIG. 1;

FIG. 7 is a functional block diagram of the recognition device of the present invention.

FIG. 8 is a flowchart of a character recognition processing operation executed by the control unit in FIG. 7;

FIG. 9 is a diagram showing memory sizes of a conventional diagram and trigram.

[Explanation of symbols]

1: Document database, 12: Character attribute conversion table, 13: Compressed character code conversion table, 18,
46: control unit, 19: unigram creation unit, 2
0: diagram creation unit, 21: attribute trigram creation unit,
22: Compression diagram creating unit, 23: Vector dividing unit, 24: Clustering unit, 31: Image input unit, 32: Cutout unit, 37 ...
Feature extraction unit 38 Matching unit 4
2 ... Candidate character string expansion unit 43: Score calculation unit 45: Word collation language processing unit 58: Table, 59: Language dictionary for word collation

Claims (17)

[Claims] 1. A memory in which a character string of one natural language is stored, and all characters of the character string stored in the memory are clustered into clusters having similar transition characteristics, and a cluster code is assigned to each cluster. A clustering unit to be assigned; a cluster code of each character in the character string stored in the memory; a transition probability between the cluster codes of the two characters for all adjacent two characters in the character string; A probability table creation device comprising a two-cluster code transition probability table creation unit for creating a transition probability table. 2. The probability table creation device according to claim 1, wherein an attribute of each character in the character string stored in the memory is obtained, and an attribute between the three adjacent characters in the character string is calculated. A probability table creation device, comprising: a three-character attribute transition probability table creation unit that creates a three-character attribute transition probability table by calculating the transition probability of. 3. The one-character appearance probability according to claim 2, wherein the one-character occurrence probability table is created by calculating the occurrence probability of each one character with respect to all the characters in the character string stored in the memory. A probability table creation device comprising a table creation unit. 4. The probability table creation device according to claim 1, further comprising a logarithmic compression unit that logarithmically compresses each element value of the two-cluster code transition probability table. 5. The probability table creation device according to claim 1, wherein a transition probability between all two adjacent characters in all characters of the character string stored in the memory is obtained, and a two-character transition probability table is created. And a clustering unit that divides the two-character transition probability table into vectors having a dimension equal to the number of characters to be recognized and a predetermined number of clusters. And a cluster code conversion table creating unit that creates a cluster code conversion table used when obtaining a cluster code of a cluster to which each of the two characters belongs based on the result of the clustering. The 2 cluster code transition probability table creation unit uses the cluster code conversion table. The two characters each have a cluster code conversion section for converting into the cluster code, the 2 by using the converted cluster code
A probability table creation device characterized by creating a cluster code transition probability table. 6. The probability table creation device according to claim 2, further comprising: an attribute conversion table used when obtaining the attribute of the character, wherein the three-character attribute transition probability table creation unit uses the attribute conversion table. A probability table creating apparatus, comprising: an attribute conversion unit for converting the character into an attribute by using the converted attribute to create the three-character attribute transition probability table. 7. The probability table creation device according to claim 2, wherein the memory stores a Japanese sentence, the character is a Japanese character, and the attribute is a hiragana, a katakana, a symbol, a kanji. A probability table creating device, which is any one of a character, a numeral, an uppercase alphabet and a lowercase alphabet. 8. The probability table creation device according to claim 2, wherein the memory stores a Chinese sentence, the character is a Chinese character, and the attribute is a supplementary character, a symbol, or a supplementary character. A probability table creation device characterized by any one of kanji, numbers, uppercase letters and lowercase letters other than. 9. A character string transition probability of an input character string is calculated using the two-cluster code transition probability table created by the probability table creation device according to claim 1 and the two-cluster code transition probability table. A stochastic language processing device comprising a character string transition probability calculation unit. 10. The stochastic language processing device according to claim 9, wherein the one-character appearance probability table and the three-character attribute transition probability table created by the probability table creation device according to claim 2 or 3. A stochastic language having at least one probability table, wherein the character string transition probability calculation unit calculates a character string transition probability of the input character string by using the probability table as well. Processing equipment. 11. The stochastic language processing device according to claim 10, wherein the character string transition probability calculating unit calculates the character string transition probability by the following first equation. Processing equipment. P (c ₁ , c ₂ ,..., C _n ) = P (c ₁ ) + P (c ₁ , c ₂ ) + P (c ₁ ,
c ₂ , c ₃ ) + P (c ₂ , c ₃ , c ₄ ) +... + P (c _n−2 , c _n−1 , c
_n ) + P (c _n−1 , c _n ) + P (c _n ) where P (x) = 1 character appearance probability table (x) P (x, y) = {1 character appearance probability table (x) +1 Character appearance probability table (y)} / 2 + 2 cluster code transition probability table (x ^c , y ^c ) / 2 P (x, y, z) = {1 character appearance probability table (x) +1 character appearance probability table (y) +1 character appearance probability table (z)} /
3+ {2 cluster code transition probability table (x ^c , y ^c ) +2 cluster code transition probability table (y ^c , z ^c )} / 2+ {3 character attribute transition probability table (x ^a , y ^a , z ^a )} / 3, however, x, y, z: the character ^{x c, y c, z c} : characters x, y, z of the cluster code ^{x a, y a, z a} : character x, y, attribute 1 appearance of z Probability table (x): Element value of language element x in 1-character appearance probability table 2 cluster code transition probability table (x ^c , y ^c ): Cluster code combination in 2 cluster code transition probability table
(x ^c, y ^c) element value 3 character attribute transition probability table of ^{(x a, y a, z} a): 3 character attributes set in the character attribute transition probability table ^{(x a, y a, z} a) element values of 12. A cutout unit for cutting out individual language elements from an input language element sequence, a matching unit for matching a feature pattern of the cutout language elements with a standard pattern to obtain a plurality of recognition candidates, A recognition apparatus comprising: a candidate character string generation unit configured to perform a stochastic language process on a plurality of candidate character strings obtained by combining a plurality of recognition candidates to generate a predetermined number of candidate character strings. The candidate character string generation unit calculates the score of the candidate character string using the two cluster code transition probability table created by the two-cluster code transition probability table created by the described probability table creation device. And performing a stochastic language process for evaluating the likelihood of the candidate character string based on the calculated score. Recognition apparatus characterized by there. 13. The recognition device according to claim 12, wherein at least one of the one-character appearance probability table and the three-character attribute transition probability table created by the probability table creation device according to claim 2 or 3. A recognition device comprising a probability table, wherein the score calculation unit calculates the score of the candidate character string by using the probability table as well. 14. The recognition apparatus according to claim 12, wherein a predetermined number of candidate character strings generated by the candidate character string generation unit are subjected to a word matching language processing to determine an optimal candidate character string. A recognition device, comprising: a word collation language processing unit that outputs a result of recognition of an input language element sequence. 15. The recognition device according to claim 13, wherein the score calculation unit calculates the score by the following second equation. Score _{(1) = W S S 1} + W P P (c 1) score _{(2) = W S {(} S 1 + S 2) / 2} + W P P (c 1, c 2) score (i) = [score (i−1) × (i−1) + {W _S S _i + W _P P
_{(c i-2, c i} -2, c i)} ] / i where score (I): Score c ₁ of the candidate character strings of characters _{I, c 2, c 3,} ..., c n: the candidate characters Sequence S ₁ , S ₂ , S ₃ ,..., _Sn : Evaluation value of each recognition candidate at the time of matching P (y): Character string transition probability of candidate character string y
(Calculated using the formulas for calculating P (x), P (x, y), P (x, y, z) according to claim 7) W _P , W _S : Weight 16. The recognition device according to claim 12, wherein the input language element sequence is a character string. 17. A cutout unit for cutting out individual characters from an input character string, a matching unit for matching a feature pattern of the cutout character with a standard pattern to obtain a plurality of recognition candidates, A plurality of candidate character strings obtained by combining the candidate character strings by performing a stochastic language process using the two-cluster code transition probability table created by the probability table creating apparatus according to claim 1. And a character recognition program for causing a computer to function as a candidate character string generation unit for generating a predetermined number of candidate character strings based on the score. .