JP2011248609A - Form recognition device and form recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To read a character string robustly against a character recognition error and with fewer errors for forms which have obscure character string arrangements, without needing to pre-define a described position and attributes of a reading-target character string for a form group in which various layouts coexist.SOLUTION: A form recognition device performs the steps of: detecting a character string area from a form image (S120); calculating, for the detected character strings, an item name likelihood representing likeness of an item name and an item value likelihood representing likelihood of an item value (S140, S150); calculating, for a character string pair constituted by a combination of the detected character strings, an arrangement likelihood representing validity of the arrangement relationship of the character string pair as an item name-item value relationship (S160); calculating an evaluation value of the item name-item value relationship based on the values of the item name likelihood, item value likelihood and arrangement likelihood (S170); and determining the item name-item value relationship in the form image (S180).

Description

  The present invention relates to a form recognition device and a form recognition method, and more particularly, to a technique for understanding attributes of a character string described on a form image and character string recognition.

  A conventional form recognition apparatus reads a character string to be read and the character string by “form definition” in which the user registers in advance the position and attribute of the character string to be read on the form image in advance. Had to understand the attributes.

  In the form processing business, when the layout of the form to be processed, that is, the description position of the character string, the description position of the frame, and the arrangement of the frames are unified, and the description position of the character string to be read in the form image is fixed, The position of the character string to be read is detected and the attributes of the character string are understood by registering the form definition in the apparatus in advance. On the other hand, there is a form processing operation in which various layouts are mixed and a form processing operation in which the layout of the form to be processed is unknown. The various layouts mean that the description position of the character string to be read differs for each form as in the form examples 200 and 201 shown in FIG. The example of FIG. 2 is an example of reading the “transfer account number”, “delivery amount”, and “delivery date” from the form image. However, since the description positions are different, it is necessary to create a form definition for each form. . Depending on the business, there may be several tens of thousands of forms layout types, and the recognition by the form definition cannot be used due to the enormous cost of creating the form definition.

  As a technique for recognizing various layout forms without using a form definition, for example, as in the technique disclosed in Patent Document 1, a character string in a form image is collated with an item name word registered in an item name dictionary. A character string that succeeds in item name word matching is determined as an item name, and a character string that fails in item name word matching is determined as an item value (represented as “data” in Patent Document 1), and an item name and item value candidate. There is a method in which the correspondence between the item name and the item value is determined from the arrangement relationship and the attribute of the corresponding item value is determined from the attribute of the item name registered in the item name dictionary.

  Further, in the method of Patent Document 2, a logical structure composed of logical elements (item names and item values) that logically constitute a form is obtained by using a character string that is a logical element, an appearance frequency of the character string, and a relative relationship between the logical elements. By creating a dictionary defined by the frequency of position for each form type and comparing the character string in the form image with the logical structure in the dictionary, the probability that the character string in the form image is a logical element in the logical structure The character string to be read is read from the form image.

JP 2008-204226 A JP 2008-33830 A

  In Patent Document 1, when a character string recognition error occurs due to adverse effects such as noise or low image quality of a form image, a matching error occurs in item name word matching, and the character string in the form image is the correct item name. Nevertheless, it may be determined that it is not an item name. Furthermore, in the method of Patent Document 1, a character string that becomes an item name is used to determine an item name-item value correspondence from the arrangement relationship between a character string determined as an item name and a character string determined as an item value candidate. If the determination is incorrect, the error is directly connected to the determination error of the item name-item value correspondence. Further, when there are a plurality of item value candidates associated with one item name due to the arrangement relationship, there is a possibility that an association error between the item name and the item value relationship may occur.

  Further, in the method of Patent Document 2, the appearance frequency of the logical elements in the logical structure dictionary and the frequency of the relative position between the logical elements are defined for each form type, resulting in a huge dictionary creation cost. Also, in the case of a form with a layout that is inconsistent with the logical structure dictionary, a correct recognition result cannot be obtained, and versatility decreases.

The present invention has been made in view of such problems.
That is, the present invention provides a form recognition method for correctly extracting an item name-item value relationship from a form image even when a character string recognition error occurs in a character string recognition process, which is a process prior to item name matching. It is a first problem to provide the above.
In addition, the present invention provides a correspondence error even when the arrangement relationship of the item name-item value relationship is ambiguous, that is, when there are a plurality of item value candidates associated with the arrangement relationship for one item name. A second problem is to provide a form recognition method for extracting the item name-item value relationship.
It is a third object of the present invention to provide a form recognition method that can recognize a document with high versatility even for forms having various layouts with a minimal dictionary creation cost.

  In order to solve the above-described problem, the form recognition apparatus of the present invention is a form recognition apparatus for inputting a form image and performing recognition processing of a character string in the form image, and detecting a character string region from the form image. A character string detection unit that recognizes individual characters in the character string region, and a character string recognition unit that calculates the probability that the character string is an item name for the character string in the form image The item name likelihood calculating unit, the item value likelihood calculating unit for calculating the item value likelihood representing the probability that the character string is the item value for the character string in the form image, and the character string in the form image For a pair, an arrangement likelihood calculating unit that calculates an arrangement likelihood indicating whether the arrangement relation of the character string pair is valid as an item name-item value relationship, and the item name likelihood, item value likelihood, arrangement Based on the likelihood, the character string pair's item name-an evaluation representing the likelihood as the item value. The item name-item value relationship evaluation value calculation unit for calculating the value and the evaluation value output from the item name-item value relationship evaluation value calculation unit associates the item name-item value relationship in the form image. It has an item name-item value relationship determination unit to be determined.

  Further, in the form recognition apparatus of the present invention, the placement likelihood calculation unit is configured such that the placement relationship or size of the frame of the item name character string and the item value character string of the character string pair, or the placement relationship or size of the character string rectangle. The placement likelihood is calculated based on a penalty rule that is a rule representing the invalidity of the item name-item value relationship.

  In the form recognition device of the present invention, the item name likelihood calculating unit calculates the item name likelihood for the character string by collating with an item name dictionary describing item name words, and the item value The likelihood calculating unit calculates the item value likelihood for the character string by collating it with a notation dictionary describing grammar notation rules for item value words and character strings.

  According to the present invention, a form can be recognized without a strict form definition in a form processing operation in which forms of various layouts are mixed. Further, it is possible to recognize a form that is robust against a character recognition error and has an ambiguous item name-item value relationship with few errors.

It is a flowchart of the form recognition process in the Example of this invention. It is a figure which shows the example of various layout forms. It is a block diagram of the form recognition apparatus in the Example of this invention. It is a block block diagram of the form recognition part in the Example of this invention. It is a figure which shows the example of the description structure of an item name dictionary. It is a flowchart of item name likelihood calculation in the Example of this invention. It is a figure which shows the example of a data structure of the item name likelihood table in the Example of this invention. It is a figure which shows the example of the description structure of a description dictionary. It is a flowchart of item value likelihood calculation in the Example of this invention. It is a figure which shows the example of a data structure of the item name likelihood table in the Example of this invention. It is a flowchart of arrangement | positioning likelihood calculation in the Example of this invention. It is an example of a penalty rule in arrangement likelihood calculation. It is a figure which shows the example of a data structure of the arrangement | positioning likelihood table in the Example of this invention. It is a figure which shows the example of a data structure of the item name-item value relationship evaluation value table in the Example of this invention. It is a figure which shows the example of a form image

  Embodiments of the present invention will be described below. Note that the present invention is not limited thereby. The outline of the present invention will be described before describing the details of specific processing.

  The present invention reads a character string to be read and determines the attribute of the character string without a form definition for registering in advance the description position of the character string to be read and the attribute of the character string. Is to do. For this purpose, it is necessary to extract pairs of item values and corresponding item names from the form image. In this embodiment, for all the character strings in the form image, the item name likelihood indicating the probability that the character string is the item name and the item value likelihood indicating the probability that the character string is the item value are calculated. For all character string pairs, an arrangement likelihood representing whether the arrangement relationship between the two character strings forming the character string pair is valid as the item name-item value relationship is calculated. Further, the association between the item name-item value relationship in the form image is determined based on the value of the item name-item value relationship evaluation value calculated based on the item name likelihood, the item value likelihood, and the placement likelihood. .

Specifically, the association of the item name-item value relationship in the form image is determined in the following order.
(1) The item name word in the item name dictionary, which is a list of item names registered in advance by the user, is matched with the character string in the form image, and all the item name words in the item name dictionary and the form image The item name likelihood is calculated for all combinations of character strings.
(2) For example, a notation dictionary defined by general-purpose grammar notation rules such as date, amount, and account number is collated with a character string in a form image. Item value likelihood is calculated for all combinations of character strings.
(3) A penalty rule describing a rule in which the arrangement relationship between two character strings is an invalid arrangement relationship as an item name-item value relationship, and all character string pairs composed of two character strings in the form image The placement likelihood is calculated for all the character string pairs in the form image.
(4) Based on the item name likelihood, the item value likelihood, and the placement likelihood, an evaluation indicating whether the character string pair has an item name-item value relationship with respect to all the character string pairs in the form image. A value is calculated, and an item name-item value relationship is extracted from the form image based on the evaluation value.
Since (1), (2), and (3) are processed independently, the order does not depend on the above order.

  Hereinafter, a form recognition apparatus and a form recognition method according to an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 3 is a hardware configuration example of the form recognition apparatus of the present invention. The form recognition apparatus according to the present embodiment includes an input apparatus 301 for inputting command commands and data, an image input apparatus 302 for inputting a form to be recognized, character string detection and character string recognition, and item name-item value relationships. A form recognition unit 300 for analyzing the above, a recognition dictionary 303 for storing a character recognition dictionary and an item name word dictionary, and a display device 304 for displaying the recognition result of the form image. The form recognition unit 300, the input device 301, the image input device 302, the recognition dictionary 303, and the display device 304 may be connected via a network or the like without depending on physical connection means.

  FIG. 4 shows a detailed block diagram of the form recognition unit 300. The form recognition unit 300 includes a character string detection unit 420, a character string recognition unit 430, an item name likelihood calculation unit 440, an item value likelihood calculation unit 450, an arrangement likelihood calculation unit 460, and an item name-item value relationship evaluation value calculation. 470, an item name-item value relationship determining unit 480, and the like.

  Here, the character string detection unit 420 detects a character string region from the form image. The character string recognition unit 430 recognizes individual characters in the character string area. The item name likelihood calculating unit 440 calculates an item name likelihood representing the probability that the character string is a word registered in the item name dictionary for the character string in the form image. The item value likelihood calculation unit 450 is an item value likelihood that is the probability that the character string in the form image matches a grammar rule such as a word, amount, date, or account number registered in the notation dictionary. The degree is calculated. The placement likelihood calculation unit 460 calculates the placement likelihood, which is the validity of the item name-item value relationship of the placement relationship of the character string pair, for the character string pair in the form image. The item name-item value relationship evaluation value calculation unit 470 calculates an evaluation value representing the likelihood as the item name-item value of the character string pair based on the item name likelihood, the item value likelihood, and the placement likelihood. It is to calculate. The item name-item value relationship determination unit 480 determines the item name-item value relationship in the form image based on the evaluation value output from the item name-item value relationship evaluation value calculation unit.

  FIG. 1 shows a process flow diagram of form recognition in the form recognition unit 300. First, in step S110, a frame is detected from the input form image. Next, in step S120, the character string detection unit 420 detects a character string region from the form image. The character string area is a rectangular area including a certain character string. As specific examples of steps S110 and S120, for example, as in the technique disclosed in Japanese Patent Application Laid-Open No. 11-53466, a ruled line is extracted from a form image, and an intersection and an end point of two ruled lines are extracted. By detecting the upper right corner, the upper left corner, the lower right corner, and the lower left corner corresponding to, a method of detecting a frame from the form image can be used. In step S130, the character string recognition unit 430 recognizes each character in the character string area detected in step S120 using the character recognition dictionary 131. As a specific example of step S130, for example, non-patent document: F. Kimura et. Al. “Modified quadratic discriminant functions and the application to chinese character recognition” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol.9, pp.149- As in the technique disclosed in 153, by determining the identification function, which is a distance measure derived from Bayes' theorem, for each character category, and classifying unknown characters into the character category based on the value output by the identification function, There are methods for recognizing individual characters.

  In step S140, the item name likelihood calculation unit 440 collates the item name dictionary 141 for each character string recognition result of each character string recognized in step S130, and the character string is registered in the item name dictionary 141. The item name likelihood representing the probability of being a word is calculated. In step S150, the item value likelihood calculation unit 450 collates with the notation dictionary 151 for each character string recognition result of each character string recognized in step S130, and the character string is described in the notation dictionary 151. The item value likelihood representing the probability of matching the grammar notation rule of the word or character string is calculated. The processing of step S140 and step S150 will be described in detail later.

  In step S160, two character strings forming a character string pair in the form image based on the coordinates of the frame detected in step S110 and the coordinates of the character string region detected in step S120 by the placement likelihood calculation unit 460. The placement likelihood representing the validity as the item name-item value relationship of the placement relationship is calculated. The placement likelihood is calculated based on the size and placement relationship of the frame to which the two character strings belong, and the size and placement relationship of the two character string rectangles. The process of step S160 will be described in detail later.

  In step S170, the item name-item value relation evaluation value calculation unit 470 uses the item name likelihood calculated in step S140, the item value likelihood calculated in step S150, and the placement likelihood calculated in step S160. Then, the evaluation value of the item name-item value relationship of the two character strings is calculated from the numerical value of each likelihood. For example, the evaluation value is calculated by using an evaluation function that outputs an evaluation value by substituting item name likelihood, item value likelihood, and placement likelihood. Further, there is a method of setting the evaluation value to 1 (setting a flag) when the item name likelihood, the item value likelihood, and the placement likelihood all exceed a predetermined threshold value. The process of step S170 will be described in detail later.

  In step S180, the item name-item value relationship determination unit 480 determines the item name-item value relationship in the form image based on the evaluation value calculated in step S170. For example, a character string pair whose evaluation value is greater than or equal to a certain threshold value is determined as an item name-item value relationship. In addition, among the item name-item value relationship candidates having the same attribute, a character string pair having the largest evaluation value is determined as the item name-item value relationship. By determining the item name-item value relationship, the recognition result of the character string and the attribute of the character string are determined.

  In FIG. 1, step S140, step S150, and step S160 are independently calculated in parallel. Note that step S120 is subsequent to step S110, step S130 is subsequent to step S120, steps S140 and S150 are subsequent to step S130, steps S160 are subsequent to step S120, and steps S170 and S170 are subsequent to steps S140, S150, and S160. If step S180 is processed in the subsequent stage, the processing flow does not depend on the format of FIG.

  Hereinafter, the processing flow of the item name likelihood calculation in step S140 in FIG. 1, the processing flow of the item value likelihood calculation in step S150, the processing flow of the placement likelihood calculation in step S160, and the item name-item value relationship in step S170. A calculation example of evaluation value calculation will be described in detail.

First, the processing flow of the item name likelihood calculation in step S140 will be described using FIG. 5, FIG. 6, and FIG.
FIG. 5 is an example of an item name dictionary, FIG. 6 is a flowchart of item name likelihood calculation, and FIG. 7 is an example of an item name likelihood table. In step S1401 of FIG. 6, it is determined whether or not there is a character string for which the item name likelihood is not calculated in the form image. If there is no remaining character string, the item name likelihood calculation process is terminated. In step S1402, it is determined whether or not there is an item name word that has not been collated with respect to the character string. In step S1403, the character string in the form image is compared with the item name in the item name dictionary, and the item name likelihood is calculated. As a specific example of word collation in step S1403, for example, the technique disclosed in Japanese Patent Application Laid-Open No. 2004-171316 can be used. In addition, as a method for calculating the item name likelihood, for example, the character string recognition result in step S130 of FIG. 1 has the character identification likelihood of each individual character, and the character of each individual character in the character string path obtained by word matching A method of using the average value of the identification likelihood as the item name likelihood can be used. In addition, the item name likelihood may be calculated based on the identification likelihood of individual characters, the likelihood of extraction into individual characters, the size and aspect ratio of the individual character rectangle, and the like. In step S1404, the item with the maximum item name likelihood obtained in step S1403 is recorded in the item name likelihood table of FIG. When the item name likelihood is calculated for the character string in the form image of FIG. 15 using the item name dictionary of FIG. 5, an example of the item name likelihood table is as shown in FIG. In the case of the item name likelihood table of FIG. 7, N item strings in the form image (N = 3 in this embodiment) are collated with all item name words, and the item name likelihood is calculated. To do.

Next, the processing flow of the item value likelihood calculation in step S150 of FIG. 1 will be described using FIG. 8, FIG. 9, and FIG. The basic concept is the same as the item name likelihood calculation described above.
In step S1501 of FIG. 9, it is determined whether there is a character string for which the item value likelihood is not calculated in the form image. If there is no remaining character string, the item value likelihood calculation process is terminated. In step S1502, it is determined whether or not there is a notation dictionary that is not collated for the character string. If there is no remaining dictionary, the process proceeds to the item value likelihood calculation of the next character string in the form image. In step S1503, the character string in the form image is compared with the notation dictionary, and the item value likelihood is calculated. Specific examples of word matching in step S1503 include, for example, non-patent literature: Takahashi et al., “Development of a character path search method using a recursive transition network”, IEICE Technical Report Vol.109 No. As in the technology disclosed in .418 pp.141-146, the optimal character from the state transition network is matched by matching the identification candidate character network in which individual character identification candidates are regarded as nodes and the notation dictionary expressed by the state transition network. There is a method of obtaining a character string recognition result by selecting a column path. As an example of item value likelihood calculation, the item name likelihood is based on the identification likelihood of individual characters, the likelihood of extraction into individual characters, the size and aspect ratio of individual character rectangles, etc. There is a method to calculate In step S1504, the item value likelihood obtained in step S1503 is registered in the item value likelihood table. The processes in steps S1503 and S1504 described above are performed on combinations of character strings and notation dictionaries in all form images (steps S1505 and S1506). Examples of the notation dictionary include a regular expression of a character string notation rule as indicated by 801 in FIG. 8, a word network indicated by 802, an item value word list indicated by 803, and the like. When the item value likelihood is calculated for the character string in the form image of FIG. 15 using the notation dictionary 801 of FIG. 8, the item value likelihood table is as shown in FIG.

  Next, the processing flow of the placement likelihood calculation in step S160 of FIG. 1 will be described using FIG. 11 and FIG. In step S1701 of FIG. 11, the presence / absence of a character string pair in the form image for which the placement likelihood is not calculated is determined. If there are no remaining character string pairs, the placement likelihood calculation process is terminated. In step S1702, the placement likelihood is initialized. In this method, for example, the placement likelihood is initialized with the maximum value that the placement likelihood can take, and the placement relationship between the two character strings forming the character string pair is determined to be invalid as the item name-item value relationship. There is a method of defining a possible arrangement pattern as a penalty rule, and calculating as a value obtained by subtracting a penalty value obtained by the penalty rule from an initial value. Therefore, in step S1702, the initial value of the placement likelihood is set as the maximum value that the placement likelihood can take. In step S1704, it is determined whether the arrangement relationship between the two character strings corresponds to the penalty rule referred to. If applicable, in step S1705, a penalty value is calculated and subtracted from the current placement likelihood. As described above, step S1703, step S1704, and step S1705 are repeated by the number of penalty rules, and the arrangement likelihood of the character string pair is calculated. The above processing is executed for all character string pairs in the form image.

  An example of the penalty rule is shown in FIG. The penalty rule is calculated based on the arrangement relation and size of the frame to which the two character strings belong, and the arrangement relation and size of the two character string rectangles. For example, in the case of the rule 1101, when two character strings exist in frames adjacent to each other, and the center coordinate of the character string serving as the item name is shifted from the center coordinate of the frame, the shift distance is set. A penalty is added accordingly. This is based on the assumption that when two character strings originally have an item name-item value relationship, the character string serving as the item name exists near the center of the frame. In the case of the rule 1102, if the height of the frame to which the character string that is the item name belongs is larger than the height of the frame to which the character string that is the item value belongs, a penalty is added according to the ratio of the frame height. This is based on the assumption that the frame height to which the item value belongs is larger than the frame height to which the item name belongs. In the case of rule 1103, when two character strings exist in the same frame, if the character string that is the item value exists to the left or above the character string that is the item name, the character protrudes to the left or upward. Penalties are added according to the distance. This is based on the assumption that when two character strings are in the item name-item value relationship, the item value exists in the lower right direction of the character string serving as the item name. In the case of rule 1104, when two character strings do not belong to any frame in the form image and exist outside the frame and close to each other, the height of the character string as the item name and the character as the item value If the column height is different, a penalty is added according to the ratio of the character string height. This is based on the assumption that the height of the character string of the item name and the item value is substantially equal to each other. The rule 1105 adds a penalty according to the distance between two character strings when the distance between the character string as the item name and the character string as the item value is long. This is based on the assumption that when two character strings are in the item name-item value relationship, the two character strings are close to each other. The rule 1106 adds a penalty according to the deviation distance when a character string as an item value exists in the upper left direction from a character string as an item name. This is based on the assumption that the item value exists in the lower right direction of the item name.

  As described above, the penalty rule assumes a plausible arrangement relationship when two character strings are in the item name-item value relationship, and when two character strings are arranged in an arrangement relationship that deviates from the assumption, Generated to add a penalty. The penalty rule is not limited to 1101 to 1106 shown in FIG. 12, but is calculated from the layout relationship and size of the frames to which the two character strings belong, the layout relationship and size of the two character string rectangles, etc. Assuming a plausible arrangement relationship when the two character strings are in the item name-item value relationship, the calculation method added when the two character strings deviate from the assumption does not depend on this.

  When the placement likelihood is calculated for the character string pair in the form image of FIG. 15 using the penalty rule of FIG. 12, the placement likelihood table is as shown in FIG. For example, in the case of the character string pair of item name character string number “1” −item value character string number “2”, it does not correspond to any of the penalty rules 1101 to 1106 in FIG. It is set to “1.00”. Further, in the case of the character string pair of item name character string number “1” −item value character string number “3”, it corresponds to the penalty rule 1101 of FIG. 12, so the penalty calculated by the penalty rule 1101 is determined from the initial value. Subtraction is performed, and the placement likelihood becomes “0.90”.

  Next, the processing flow of the item name-item value relationship evaluation value calculation in step S170 of FIG. 1 will be described with reference to FIG. The item name-item value relationship evaluation value is calculated by, for example, an evaluation function that receives item name likelihood, item value likelihood, and placement likelihood. As an example of the evaluation function E (LL, VL, AL) where LL is the item name likelihood, VL is the item value likelihood, and AL is the placement likelihood, for example, Equation (1), Equation (2), Equation There is what is shown in (3).

E (LL, VL, AL) = (LL + VL) x AL (1)
E (LL, VL, AL) = LL + VL + AL (2)
E (LL, VL, AL) = √ (LL × VL) × AL (3)
The evaluation function is not limited to the above (1), (2), and (3), and the two character strings have an item name-item value relationship based on the item name likelihood, the item value likelihood, and the placement likelihood value. Any format that can calculate certain certainty is not limited to this.

  When the item name likelihood table of FIG. 7, the item value likelihood table of FIG. 10, and the placement likelihood table of FIG. 13 are obtained, the item name-item value for the form image of FIG. 15 obtained by the above equation (1). The evaluation value table is as shown in FIG. The item name-item value relationship evaluation value is calculated for each predefined attribute in the item name dictionary of FIG. 5 and the notation dictionary of FIG. 8, and is registered in the item name-item value evaluation value table. In this embodiment, when any of the item name likelihood, the item value likelihood, and the placement likelihood is “0.00”, the item name-item value evaluation value is not calculated. For example, in the case of the item name character string number “1” and the item value character string number “2”, E (LL, VL, AL) = (0.25 + 0.85) × 1.00 = 1.10 for the attribute ID “002”. For the attribute ID “003”, E (LL, VL, AL) = (0.28 + 0.20) × 1.00 = 0.48. In the case of the item name character string number “1” and the item value character string number “3”, E (LL, VL, AL) = (0.28 + 0.92) × 0.90 = 1.08 for the attribute ID “003”. . The item name-item value evaluation value calculation by the evaluation function as described above is performed for all character string pairs in the form image and registered in the item name-item value evaluation value table. The item name-item value relationship candidates for the form image in FIG. 15 are “payment amount”-“17,420” of the item name character string number “1” and the item value character string number “2”, and the item name character string number “ “1” and item value character string number “3” are “delivery date” − “21.11.13”. In this case, since the item name character string number “1” is duplicated in both candidates, a character string pair having a high evaluation value is selected (step 180 in FIG. 1), and the final item name-item value relationship extraction result is: The item name character string number “1” and the item value character string number “2” are “payment amount” − “17,420” (attribute ID: 002).

  As described above, according to the present invention, the item name likelihood of the item name likelihood of the character string in the form image and the item value likelihood of the item value likelihood are calculated for all character strings, and the character in the form image is calculated. Item name-item that is calculated based on the item name likelihood, item value likelihood, and arrangement likelihood. A non-tabular layout form that is robust against character recognition errors and has an ambiguous layout relation of the item name-item value relation by the form recognition method that extracts the item name-item value relation in the form image by the value relation evaluation value Can be recognized with few errors. In addition, the module configuration for independently calculating the item name likelihood, the item value likelihood, and the placement likelihood can provide a highly versatile form recognition method with a small number of definitions.

131 ... Character recognition dictionary, 141 ... Item name dictionary, 151 ... Notation dictionary, 200, 201 ... Form example, 300 ... Form recognition unit, 301 ... Input device, 302 ... Image input device 303 ... Recognition dictionary 304 ... Display device 420 ... Character string detection unit 430 ... Character string recognition unit 440 ... Item name likelihood calculation unit 450 Item value likelihood calculation unit, 460 ... placement likelihood calculation unit, 470 ... item name-item value relationship evaluation value calculation unit, 480 ... item name-item value relationship determination unit, 801 ... Regular expression of character string notation rule, 802... Word network, 803... Item value word list, 1101 to 1106.

Claims (6)

A form recognition device that inputs a form image and performs recognition processing of a character string in the form image,
A character string detection unit for detecting a character string region from the form image;
A character string recognition unit for recognizing individual characters in the character string region;
An item name likelihood calculating unit that calculates an item name likelihood representing the probability that the character string is an item name for the character string in the form image;
An item value likelihood calculating unit that calculates an item value likelihood representing the probability that the character string is an item value for the character string in the form image;
An arrangement likelihood calculating unit for calculating an arrangement likelihood indicating whether the arrangement relation of the character string pair is valid as the item name-item value relation for the character string pair in the form image;
An item name-item value relation evaluation value calculation unit for calculating an evaluation value representing the likelihood as the item name-item value of the character string pair based on the item name likelihood, the item value likelihood, and the placement likelihood; ,
An item name-item value relationship determining unit that determines an association of an item name-item value relationship in a form image based on the evaluation value output from the item name-item value relationship evaluation value calculation unit, Form recognition device to do. The form recognition device according to claim 1,
The placement likelihood calculation unit determines whether the item name character string and the item value character string of the character string pair are arranged in relation to the size or size of the character string rectangle, or the relationship between the character string rectangle and the item name-item value relationship of the size. A form recognizing device, wherein the placement likelihood is calculated based on a penalty rule which represents a rule. The form recognition device according to claim 1,
The item name likelihood calculating unit calculates the item name likelihood for the character string by collating with an item name dictionary describing item name words,
The item value likelihood calculating unit calculates the item value likelihood for the character string by collating with a notation dictionary describing grammar notation rules for item value words and character strings. . A form recognition method for inputting a form image and recognizing a character string in the form image,
A character string detection step for detecting a character string region from the form image;
A character string recognition step for recognizing individual characters in the character string region;
An item name likelihood calculating step for calculating an item name likelihood representing the probability that the character string is an item name for the character string in the form image;
An item value likelihood calculating step for calculating an item value likelihood representing a probability that the character string is an item value for the character string in the form image;
An arrangement likelihood calculating step for calculating an arrangement likelihood indicating whether the arrangement relation of the character string pair is valid as an item name-item value relation for the character string pair in the form image;
An item name-item value relationship evaluation value calculating step for calculating an evaluation value representing the likelihood as the item name-item value of the character string pair based on the item name likelihood, the item value likelihood, and the placement likelihood; and ,
The item name-item value relationship evaluation value calculating step includes an item name-item value relationship determining step for determining an association between the item name-item value relationship in the form image based on the evaluation value output from the item name-item value relationship evaluation value calculating step. Form recognition method. The form recognition method according to claim 4,
In the placement likelihood calculation step, the invalidity of the layout relation and size of the frame of the item name character string and the data character string of the character string pair, or the relation of the character string rectangle and the item name-item value relation of the size. A form recognition method, wherein the placement likelihood is calculated based on a penalty rule which is a rule to be expressed. The form recognition method according to claim 4,
The item name likelihood calculating step calculates the item name likelihood for the character string by matching with an item name dictionary describing item name words.
In the item value likelihood calculating step, the item value likelihood is calculated for the character string by collation with a notation dictionary describing rule of grammar of the item value word and character string.

Images