JP4442208B2 - Character string notation analysis method and apparatus - Google Patents

Description

  The present invention relates to a character string recognition method using a grammar and a recording medium on which a character string notation analysis program is recorded.

  Character string notation analysis processing is widely used to compensate for character extraction and character recognition uncertainty in character string recognition, and to convert character string images into character string text. As the algorithm, those using morphological analysis, RTN matching (recursive transition network matching), and ascending parsing algorithm are generally used.

  For example, Japanese Patent Laying-Open No. 05-108891 (Patent Document 1) describes a method of applying morphological analysis to an OCR recognition result as means for improving OCR reading accuracy. Although it is possible to correct misreading by performing knowledge processing such as morphological analysis, the dictionary used in normal morphological analysis is intended for general sentences such as newspapers, and proofreads documents for special business use with high accuracy. To do this, it is necessary to define additional special dictionaries suitable for the field. Therefore, problems remain in terms of maintainability and computational complexity. Furthermore, since a wide range of notation knowledge called morphological analysis is targeted, it takes time to analyze notation knowledge, and there is a problem that enormous storage capacity is required for notation analysis.

  Japanese Patent Laid-Open No. 2002-117374 (Patent Document 2) proposes a character string notation analysis process using ascending syntax analysis for a handwritten digit string. In general, it is said that ascending parsing can reduce the amount of calculation compared to descending parsing, and it is often applied to what can be expressed by a simple rule such as a numeric string. However, there is a limit to the robustness against problems such as misreading of characters and noise mixing that can occur in character string recognition because the algorithm does not guarantee optimality.

On the other hand, collation algorithms using dynamic programming (DP) are widely used in the field of speech recognition and character recognition as robust algorithms that guarantee optimal solutions. Therefore, the idea of linking the advantages of dynamic programming and the notation grammar can be considered.
For example, Japanese Patent Laid-Open No. 7-219578 (Patent Document 3) proposes a method of performing recognition using dynamic programming for speech recognition. However, the method proposed here uses dynamic programming to calculate the optimal cost of word matching in the HMM model, and grammatical interpretation and selection of the optimal vocabulary are performed at different stages. Therefore, it is not an extension of dynamic programming to grammatical interpretation.

Japanese Patent Laid-Open No. 05-108891

JP 2002-117374 A JP-A-7-219578 Japanese Patent Application Laid-Open No. 09-319824 Japanese Patent Laid-Open No. 2000-251012 JP 2001-014411 A

  The present invention relates to optical character string recognition, and even in a deteriorated state due to missing characters, contact, noise mixing, misreading, etc., this is analyzed based on knowledge of character string notation, and robust character string recognition is performed. The present invention relates to a character string recognition method.

  In the conventional method, as in RTN matching, the notation grammar is traced by a horizontal search, and the interpretation result is left as an analysis tree. As a result, there is a problem that notation analysis tends to fail and the optimality of the solution is not guaranteed. A first object of the present invention is to propose a method for avoiding the adverse effects of character identification, character extraction, and character line extraction errors that may occur in OCR reading on character string recognition.

  In addition, in the notation knowledge analysis by dynamic programming in the conventional method, all cost networks are constructed in advance, and the optimal cost path search is performed for this, so a large amount of computing capacity is required for cost network construction and cost calculation. Storage capacity was needed. The second object of the present invention is to propose a method for performing character string notation analysis by optimal cost path search while suppressing calculation capacity and storage capacity.

  In order to achieve the first object, the present invention employs dynamic programming (DP) that guarantees the optimality of the solution as an algorithm for character string notation analysis, and causes unstable factors (character misreading) that may occur in character string recognition. , Unread, missing, contact, noise insertion) and incomplete notation knowledge.

  Further, in order to achieve the second object, the present invention reduces the notation knowledge by the quasi-regular expression, further analyzes the notation knowledge sequentially using the stack, and sequentially calculates the optimum cost, Achieve a reduction in computing capacity and storage capacity.

  According to the present invention, by the notation knowledge analysis algorithm using DP, character string recognition failure factors (deletion of characters, contact, noise mixing, misreading of characters caused by these, etc.) due to degradation of the document image, etc. In addition, robust notation analysis that guarantees the optimality of the solution is possible, and by allowing quasi-regular grammar as notation knowledge, notation knowledge by tri-type grammar description can be made compact and the amount of calculation can be reduced. It becomes.

  First, an outline of a processing flow for character string recognition will be described with reference to FIG. In the character string recognition device according to the embodiment of the present invention, the OCR device captures a paper document and converts it into electronic image data. This process can be omitted when the original document is electronic image data (0101). Next, based on the electronic image data, document structure analysis such as ruled line extraction, frame structure analysis, and position estimation of the reading target frame is performed (0102). For the recognition processing used at this time, a known technique (Japanese Patent Laid-Open No. 09-319824 (Patent Document 4), Japanese Patent Laid-Open No. 2000-2521012 (Patent Document 5), etc.) is used. Next, in response to the result of the document structure analysis, a character line to be read is extracted (0103). Next, extraction of character pattern candidates from the character line image and character identification of each character pattern are performed (0104). The character extraction pattern and the identification result are collectively referred to as a character string hypothesis. In a document to be read, if a character notation string that can be written is determined in advance, a notation analysis is performed on the character string hypothesis (0105). As a result, the temporary character string including character extraction and character identification ambiguity is converted into character string text and output from the OCR as a read result text (0106), however, analysis with notation knowledge is sufficient. When the reliability of the read result text is low, such as when it could not be performed, the character string hypothesis is output. Both the reading result text and the reading hypothesis data hold position information on the document image in which the character string is written, if necessary. Through the above processing, the reading result text and the reading hypothesis data are output, and the document processing is generally performed based on these data.

  The character string notation analysis process and the character string hypothesis are outlined in FIGS. FIG. 2 is a diagram for explaining the flow of character string recognition using the character string hypothesis and notation knowledge. FIG. 3 is a diagram showing the concept of the character string hypothesis and details of the data. FIG. 2 will be described. A character string hypothesis (b) is a character pattern hypothesis (b) in which a portion estimated to be a character pattern is cut out from a character line (a) to be read to create a character pattern candidate and each character pattern candidate is identified. It is assumed that the character string hypothesis has at least character pattern candidates, ranked identification character code groups obtained as a result of character identification, and information on connection relations between character pattern candidates in the character string hypothesis. Such expression of the character string hypothesis is called expression in a graph format.

  Next, the character string path (d) is calculated from the character string hypothesis using the character string notation knowledge (c). The character string path means a character code string (text) uniquely determined and a character pattern corresponding to each character code. In this example, knowledge of character string notation is expressed by arranging words with an OR symbol (|). That is, it means that a word group sandwiched between symbols | is designated as notation knowledge. In addition to this expression, there is a method using a try, a context free grammar, etc. (described in Japanese Patent Application Laid-Open No. 2001-014411 (Patent Document 6)). Details of the string hypothesis are detailed in FIG. The character string hypothesis is expressed as a directed graph in which a character pattern candidate is an arc (0301) and a character pattern boundary is a node (0302). Each character pattern includes boundary ID numbers representing left and right (upper and lower if vertical writing) nodes (pattern boundaries), information on character identification candidates (0303), and identification similarity (0304). Knowledge processing is processing for finding words and pattern strings that can be included in the character string hypothesis by using the character string hypothesis and knowledge of character string notation as input. For example, the word “blood chemistry test” in the character string notation knowledge can be found by following the character code and character pattern (0305) indicated by a circle in the character string hypothesis of FIG. When the notation of the character string written in the field is determined in advance, the character string code is determined by performing this process.

The above is the description regarding the character string recognition and the positioning of the character string notation analysis process therein. In such processing, the object of the present invention is related to the following two points.
1) In order to realize robust character string notation analysis against the deterioration of the image to be read (letter missing, contact, noise, and misreading of characters that may occur as a result), the optimality of the solution is guaranteed. Match using dynamic programming (DP).
2) The notation knowledge dictionary used by the DP collation allows a quasi-regular dictionary description to allow easy definition of keyword notation. In addition, by allowing quasi-regular expressions, a compact description of the notation dictionary is possible.

  Although string matching using dynamic programming (DP) is somewhat slow, it is generally possible to collate with N characters unreadable / inserted / defective, and edit distance (Edit Distance) There is an advantage not found in conventional notation analysis processing, such as being able to find the optimal character string. In general, in notation analysis processing, RTN grammar (Recursive Transition Network, recursive transition structure network type grammar) is used to describe a character string to be collated, but there is no engine that performs DP collation in conformity with the structure of this RTN grammar. This is because the RTN grammar allows recursive description. In DP matching, the problem is caught as an optimal control problem on a time series (one-dimensional series), and a function recurrence formula is solved with the options arranged in the time series as arguments. If this is replaced with a character string matching problem, it becomes a problem of selecting an optimal grammar on a one-dimensional grammar string so as to explain the input character pattern group. However, since the recursive structure can be defined in the RTN grammar, the grammar string Does not become a one-dimensional series. For this reason, DP cannot be applied in principle.

  Therefore, the grammar structure used in DP is not a RTN grammar but a simple regular expression. However, there are several levels of regular expression notation in relation to the DP matching algorithm. This will be described next.

  In regular expressions, notations are defined using special characters such as parallel |, set (), abbreviation [] (* and? Are not considered), and normal characters. In addition, there are some special symbols that can be used in the DP matching engine in consideration of adaptation to address dictionaries and first name surname dictionaries. The keywords to be extracted are described using quasi-regular expressions together with special symbols. Here, the description format is classified into the following three. This classification is related to the DP matching algorithm described in the next section.

1) Word enumeration notation An example of word enumeration notation is given below. In this definition, the four words “psychotherapy”, “psychoanalysis”, “psychoanalysis therapy” and “psychoanalysis” are defined.
S (psychotherapy | psychoanalysis | psychoanalysis therapy | psychoanalysis) E
In this example, the word definition sequence starts with S and ends with E. In addition, four words are written in parentheses in parallel.

2) Tri-type notation Rewriting the definition of the above four words into a tri-type is as follows. There is a merit that the dictionary definition can be made compact by using the tri notation.
S (Mental (Therapy | Analysis [Therapy | Study])) E
3) Halfway omission / branch allowance type notation omission / branch allowance can be used to absorb notation fluctuations in the following words in the grammar definition.
S (Computer [-] (Management | Diagnosis) Fee) E
This describes the notation of “computer” and the shaking of the notation of “computer”. It also indicates that “management fee” and “diagnosis fee” exist as vocabularies that follow.

  Here, the DP collation algorithm is formulated as follows. Assume that the network to be verified (candidate character network) is N, and the verification grammar is C. The network to be verified is configured as a set of start point nodes, end point nodes, and edges.

. Each symbol means a start point node ns, an end point node ne, and an edge n. This edge corresponds to a character pattern on the candidate character network. Similarly, the collation grammar C has a graph structure. this

It expresses. Here, each symbol means a start point node cs, an end point node ce, and an edge c. This edge corresponds to a grammar child on the grammar C.

  DP matching performs matching between two graphs. Here, it is considered that a certain grammar cj is collated with a certain character pattern nq. The matching cost at this time

Represented by The first character of the grammar child cj is ci.

It will be expressed as However,

Promises that the graph G represents a set of edges located before the certain edge e. In addition, the character pattern located in the preceding stage of the character pattern nq is represented by np,

Write. It is also possible that a plurality of these can exist.

  The DP verification cost at this time can be expressed by the following recurrence formula.

Here, Match () is a cost indicating whether or not the grammar child c is included in the character candidate group of the character pattern n. Path () is a cost indicating whether or not the character patterns np and nq are connected, and Next is a cost indicating whether or not the grammars ci and cj are connected. The optimal grammar sequence is found by obtaining the above recursive expression for all grammar sequence cjεC. This is the DP verification.

It should be noted that the structures of graphs C and N must be partially ordered in order to obtain an optimal solution by DP. That is, the structures of the candidate character network N and the grammar C are directed graphs, and must not include a circulation path inside. Furthermore, as described above, it is necessary to consider the following three levels as the structure of grammar C.
Grammar Level 1 Enumeration S (word | word | ... | word) E
Grammar level 2 Tri-type S (word ((word | word) | word | ...)) E
Grammar Level 3 Branch Allowed Type S ((word (word | word) word) ...) E
Simple DP matching corresponds to a grammar level 1 word enumeration type. On the other hand, in order to reduce the dictionary capacity and the collation calculation amount using the trie notation and the branch allowable notation, it is necessary to cope with grammar level 2 or higher. The RTN grammar is considered to correspond to the above grammar level 3 unless a non-terminal symbol is used recursively. From the standpoint of DP verification, the difference between grammar level 2 and grammar level 3 corresponds to whether the grammar group PreE (C, c _j ) preceding the grammar class cj is singular or plural. In the case of grammar level 2, since the preceding character is singular, a collation algorithm can be easily realized by using a stack type DP calculation table as described below. In the case of grammar level 3, since there are a plurality of preceding characters, an algorithm using a plurality of stacks is required. Here, a grammar level 2 collation algorithm is described.

  The DP verification formulated above can also be viewed as an optimal cost path problem for the network. There are two types of optimal cost calculation methods: a cost network pre-construction type and a stack table type. Using FIG. 4 as an example, the merits and demerits are compared below.

  FIG. 4A is a conceptual diagram when matching of character strings is regarded as a minimum cost path problem based on an edit distance. Here, the collation relationship between two character strings is expressed as a branch stretched on a lattice. Vertical and horizontal branches on the grid take a cost of 1, while diagonal branches take a cost of 0 if the upper and left characters match, and cost 1 if they do not match. At this time, the minimum cost from the upper left point to the lower right point represents the edit distance (Edit Distance), and the minimum cost path represents the correspondence between the character strings. In this example, three characters in the correct character string are correct and one character does not correspond.

1) Cost network pre-construction type (Fig. 4 (b))
A method in which a DP cost network is established in advance between a word group (a set of words) to be verified and a character candidate network, and the shortest path on the cost network is obtained. When a cost network of all words is constructed in advance, the storage capacity required for DP calculation is proportional to the candidate net size × total grammar length. Although the capacity increases, this is sufficient for simple word matching. In DP calculation, cost network construction takes the longest time.

2) Stack table type (Fig. 4 (c))
A method of scanning a description string of a grammar (a group of words to be collated with a structure such as a trie) from the left and possessing a DP calculation table between two consecutive characters on the word string. In the trie structure, a grammar structure descriptor (characters such as “()” and “|”) is included, so two consecutive characters are not necessarily adjacent to each other in the grammar. When scanning, it is necessary to save the DP calculation table to the stack at an appropriate grammatical break.The storage capacity required for DP calculation is at least the candidate net size × 2, when storing a path string Proportional net size x maximum word length.

  The cost network pre-construction type has an easy algorithm, but requires a storage capacity proportional to the product | N | × | C | of the number of edges of the candidate network N and the grammar C. In this patent, a stack table type calculation method is adopted in order to make the keyword extraction engine compact (dictionary capacity, calculation time capacity). In the stack table type, this capacity is minimum | N | × 2, and even when a word path is stored, | N | × maximum word length can be reduced.

  Next, the DP verification algorithm using the stack table is described. The grammar to be analyzed by this algorithm is composed of special symbols S, E, (,), [,], |, and an ordinary grammar (represented by a character code, C). In the DP collation algorithm suitable for the tri-type grammar, the grammar description string is scanned from right to left, and the DP table is pushed and popped onto the stack in a timely manner.

  The DP table is a table that holds the optimum cost (and path) when it is assumed that the currently scanned grammar cj corresponds to each character pattern nqεN on the candidate network. If the DP table for the grammar child cj is represented by Dpt (cj),

It is. The optimum cost is calculated by sequentially calculating the DP table Dpt (cj) for the grammar cj from the first grammar S. In order to calculate the DP table for the grammar cj of interest, the previous DP table Dpt (ci) corresponding to the previous grammar ci is required. This previous DP table can be obtained by referring to the stack. The stack scanning timing is shown in FIG. Further, the grammar S (12 | 3 [4 | 5] | 67) E in which the word set {12, 3, 34, 35, 67} is expressed in a trie structure is taken as a specific example, and the processing contents are shown in FIG. FIG. 7 shows how the contents of the stack transition. The grammar-driven DP type character string notation process has been described by the above DP cost calculation formulas (Formulas 7 and 8) and stack scanning timing (FIG. 5).

  The DP collation engine allows N characters to be read, skipped, and omitted so that words that could not be picked up by conventional RTN collation-type knowledge processing can be picked up. Furthermore, since a semi-regular grammar is allowed as a notation, a compact expression of notation knowledge using a trie expression is possible, and it is possible to deal with fluctuations in notation. This makes it possible to pinpoint the keywords required for document inspection from the document image, and perform searches and content inspections based on the read keywords, which is difficult with conventional OCR due to problems such as non-reading. Automatic document processing can be realized.

  Finally, based on FIG. 8, a construction example of a document processing system using the technique proposed in this patent will be described. FIG. 8 shows an example of a configuration in which a document processing system is configured in such a manner that an OCR device and a document processing device are separated by the method proposed in this patent. The upper part of FIG. 8 shows a configuration example of the OCR apparatus, and the lower part of FIG. 8 shows a configuration example of the document image processing apparatus.

  First, in the upper OCR device, a document is converted into electronic data by an image input device (0801), stored in an external storage device (0804) and a memory (0805), and read by a central processing unit (0806). The definition of the document format is stored in the external storage device (0804), and the definition stored here is referred to for the document structure analysis. These processes can be operated by humans through the operation terminal device (0802), and the processing results and the like are displayed through the display terminal device (0803), and stored in the external storage device or sent to the external device through the communication device (0807). It is done. The result read by the OCR can be output as a text file as in the conventional apparatus, but can also be output as OCR reading hypothesis data. The OCR reading hypothesis data is stored in an external storage device or sent to an external device through a communication device. At this time, it is assumed that the document ID code corresponding to the document (or image) read by OCR is assigned to the OCR reading hypothesis data. By using this document ID code, the correspondence between the paper document or document image and the OCR reading hypothesis data can be taken.

  The document image processing apparatus in the lower part of FIG. 8 performs document search and document browsing using the text file or OCR reading hypothesis data output from the OCR function apparatus. For the document once the OCR output data is generated. Has a function to search and browse repeatedly (as long as OCR output data exists). This document image processing device reads OCR additional data from the communication device (0815) and the external storage device (0812), loads it into the memory (0813), and performs search / view processing by the central processing unit (0814). . Words and document search rules to be searched are stored in the external storage device or can be input from the operation terminal device (0810). The word search result is displayed through the display terminal device (0811), and data can be transmitted to the external device through the communication device, or the search result can be stored in the external storage device. These devices are connected by a communication bus (0807, 0808, 0809, 0815, 0816). The grammar driven DP type notation analysis processing described in this patent may be incorporated in the upper part of FIG. 8 or may be incorporated in the lower part of FIG.

The processing flow figure of character string recognition. The conceptual diagram of the notation knowledge process using a character string hypothesis. The conceptual diagram of a character string hypothesis. Notation knowledge processing as an optimal cost path search problem. Calculation timing table for grammar-driven DP type knowledge processing. Grammar driven DP type notation knowledge processing process based on specific examples. The stack state transition diagram in the grammar-driven DP type notation knowledge processing. 2 shows a configuration example of an OCR device and a document processing device.

Explanation of symbols

0101: Image input unit, 0102 ... Document structure analysis unit, 0103 ... Character line extraction unit, 0104 ... Character string hypothesis creation unit, 0105 ... Grammar driven DP type character string notation analysis unit, 0106 ... Text output unit, 0101 ... Conventional Paper document 0301 input to the document processing system ... Character pattern on character string hypothesis, 0302 ... Pattern boundary on character string hypothesis, 0303 ... Character identification result on character string hypothesis, 0304 ... Character identification similarity on character string hypothesis 0305 ... word retrieved from the character string hypothesis 0801 ... image input device in the OCR device unit, 0802 ... operation terminal device in the OCR device unit, 0803 ... display terminal device in the OCR device unit, 0804 ... external in the OCR device unit Storage device, 0805, memory in OCR device unit, 0806, CPU in OCR device unit, 080 Communication device in OCR device unit, 0808 ... Communication bus in OCR device unit, 0809 ... Network unit, 0810 ... Operation terminal device in document image processing device unit, 0811 ... Display terminal device in document image processing device unit, 0812 ... Document image An external storage device in the processing device unit, 0813 ... a memory in the document image processing device unit, 0814 ... a CPU in the document image processing device unit, 0815 ... a communication device in the document image processing device unit, and 0816 ... a communication bus in the document image processing device unit.

Claims (4)

A character recognition device having an image input unit, a character string hypothesis creating unit, a character string notation analysis unit, and a notation knowledge dictionary storage unit,
The character string hypothesis creating unit outputs a character string hypothesis including character extraction and character recognition uncertainty based on the image input to the image input unit,
The character string notation analysis unit uses dynamic knowledge using character string notation knowledge in which a plurality of character strings stored in the notation knowledge dictionary storage unit are expressed together by a grammar that means parallel, set, or omission. A character string recognition device, wherein the character string hypothesis is analyzed by a programming algorithm and character string recognition is performed. The character string recognition device according to claim 1,
In the process of interpreting the character string notation grammar, the character string notation analysis unit sequentially develops the character string notation analysis result of the character string hypothesis using a stack for each character of the character string notation knowledge. A character string recognition device characterized in that the amount of calculation for cost calculation is reduced. A character string recognition method in a character recognition device having an image input unit, a character string hypothesis creation unit, a character string notation analysis unit, and a notation knowledge dictionary storage unit,
A first step of outputting a character string hypothesis including character extraction and character recognition uncertainty based on the image input to the image input unit;
The character string hypothesis is obtained by a dynamic programming algorithm using character string notation knowledge in which a plurality of character strings stored in the notation knowledge dictionary storage unit are expressed together by a grammar indicating parallel, set, and omission. And a second step of analyzing the character string. The character string recognition method according to claim 3,
In the process of interpreting the character string notation grammar, the second step sequentially develops the character string hypothesis notation analysis result using a stack for each character of the character string notation knowledge. A character string recognition method characterized by reducing a calculation amount of cost calculation.

Images