JP6487100B1 - Form processing apparatus and form processing method - Google Patents

Abstract

To provide a form processing apparatus excellent in specifying a form model.
An information recognition apparatus according to an embodiment includes an input unit, a detection unit, and an identification unit. The input unit inputs a form image of a form to be identified. The detection unit detects an input feature point from the input of the form image. The identification unit determines the similarity between each form model and the form image based on an angle formed between model feature points of each form model corresponding to a plurality of types of forms and between the input feature points, A predetermined form model corresponding to the form image is identified based on the determination result.
[Selection] Figure 1

Description

  Embodiments described herein relate generally to a form processing apparatus and a form processing method.

  As an application of optical character recognition (Optical Character Recognition), a task (data entry task) is known in which the contents of a form are converted into data by performing a character recognition process on the form image. In such a technique, a form identification technique for specifying the type of the input form is required in order to perform character recognition of various forms.

  In form identification, a position that can be used as an identification key, in which a character string unique to the form is printed, is pre-defined as a model, and when an unidentified form is input, the pre-defined part is recognized as a character. In general, a method for identifying a corresponding model is known. A character string that can be an identification key in this case does not necessarily match a general character recognition target (eg, billing address, date, amount, etc. in an invoice).

JP 2001-229341 A

However, the above-described methods have the following problems, and there is a desire to solve at least one of these problems.
-If there is no form-specific character string for each model, the identification key cannot be defined and the form cannot be identified.
-It is necessary to execute the character recognition process for the identification key for the number of defined models. If the number of models is enormous, the process takes time.
-If the identification key is printed at a location deviated from the defined position due to expansion or contraction of the form or printing deviation, an accurate character recognition result may not be obtained and form identification may fail.

  An object of the present invention is to provide a form processing apparatus and a form processing method excellent in specifying a form model.

  The information recognition apparatus according to the embodiment includes an input unit, a detection unit, and an identification unit. The input unit inputs a form image of a form to be identified. The detection unit detects an input feature point from the input of the form image. The identification unit determines the similarity between each form model and the form image based on an angle formed between model feature points of each form model corresponding to a plurality of types of forms and between the input feature points, A predetermined form model corresponding to the form image is identified based on the determination result.

FIG. 1 is a block diagram illustrating an example of a schematic configuration of a form processing apparatus according to an embodiment. FIG. 2 is a block diagram illustrating an application example of the form processing apparatus according to the embodiment. FIG. 3 is a flowchart illustrating an example of model definition processing by the form processing apparatus according to the embodiment. FIG. 4 is a flowchart illustrating an example of a form identification process and a character recognition process performed by the form processing apparatus according to the embodiment. FIG. 5A is a diagram illustrating an example of read definition information generated by the model definition unit. FIG. 5B is a diagram illustrating an example of ruled line intersection information detected by the model definition unit. FIG. 6 is a diagram illustrating an example of a ruled line intersection corresponding to a form image. FIG. 7 is a diagram illustrating an example of the definition information display area. FIG. 8A is a diagram illustrating an example of an input form image. FIG. 8B is a diagram illustrating an example of an input form image. FIG. 9A is a diagram illustrating the concept of each form model (model definition information) corresponding to a plurality of types of forms. FIG. 9B is a diagram illustrating the concept of each form model (model definition information) corresponding to a plurality of types of forms. FIG. 10 illustrates the concept of determining the similarity between each form model and the form image based on the angle formed between the model feature points and the input feature points, and identifying (specifying) the form model based on the determination result. FIG.

  Hereinafter, embodiments will be described with reference to the drawings.

FIG. 1 is a block diagram illustrating an example of a schematic configuration of a form processing apparatus according to an embodiment.
As shown in FIG. 1, the form processing apparatus 1 includes a form image capturing unit 2, a form image storage unit 3, a model definition unit 4, a model storage unit 5, a form identification unit 6, a character recognition unit 7, and a character recognition result storage. Part 8 is provided. For example, the storage unit S may include a form image storage unit 3, a model storage unit 5, and a character recognition result storage unit 8, and the storage unit S may be configured with one or more hard disk drives.

  The form image capturing unit 2 is an input unit that inputs a form image of a form Doc that is a form model, and is also an input unit that inputs a form image of a form Doc (unidentified form Doc) to be identified. The form image capturing unit 2 includes a scanner that optically reads an image of the form Doc. The form image capturing unit 2 optically reads a form Doc such as an invoice or a statement by a scanner and outputs a form image corresponding to the form Doc. Note that the form image capturing unit 2 may input a form image read by an external scanner or the like. In this case, the form image capturing unit 2 may not include a scanner. The form image storage unit 3 stores the form image output from the form image capture unit 2.

  The model definition unit 4 is composed of one or more processors. The model definition unit 4 defines (registers) a plurality of form models corresponding to a plurality of types of form Doc based on a plurality of types of form images stored in the form image storage unit 3. The model storage unit 5 stores a plurality of form models defined by the model definition unit 4.

  For example, the user selects a form image to be modeled from a plurality of types of form images displayed on a display or the like via an information input device such as a touch panel or a mouse. In addition, the user specifies the reading position and reading attributes (attributes such as numbers, names, or addresses) and the recognition position of characters, etc., for the displayed model target form image via the information input device. To do. The model definition unit 4 detects feature points such as ruled line intersections by image processing or the like from the model image of the model object, and defines a form model including the detected feature points. The model definition unit sets information indicating the specified reading attribute as a feature point at the specified reading position, and defines a form model including the set feature point. Further, the model definition unit defines a form model including a recognition position of a designated character or the like. That is, each form model defines a form model by associating a feature point such as a ruled line intersection, a feature point indicating a reading attribute designated at a designated reading position, a recognition position, and the like.

  The form identifying unit 6 includes one or more processors, and identifies (specifies) which form model the form image output from the form image capturing unit 2 corresponds to. That is, the form identifying unit 6 identifies (specifies) what type of form the form image output from the form image capturing unit 2 is. The form identifying unit 6 detects input feature points from the form image of the form to be identified, compares the angle between the model feature points of each form model corresponding to a plurality of types of forms and the input feature points with a threshold, The similarity between each form model and the form image is determined based on the comparison result, and a predetermined form model corresponding to the form image is identified (specified) based on the similarity determination result. For example, the similarity is lower as the formed angle is larger than the threshold, and the similarity is higher as the formed angle is smaller than the threshold. For example, the form model having the highest similarity is specified.

  The character recognition unit 7 includes one or more processors, and recognizes various information such as characters, numbers, and symbols included in the form image output from the form identification unit 6. The character recognition unit 7 efficiently recognizes various information such as characters included in the form image by character recognition processing according to the form model (form type). The character recognition unit 7 performs a recognition process on the form image using the recognition position of the character or the like associated with the predetermined form model identified by the form identification unit 6. By targeting a position corresponding to a recognition position of a character or the like associated with a predetermined form model, the recognition process can be executed efficiently. Note that the recognition position of a character or the like associated with a predetermined form model may be enlarged by a predetermined size, and the recognition process may be executed on the recognition position of the enlarged character or the like. The character recognition result storage unit 8 stores the character recognition result by the character recognition unit 7.

  In addition, you may make it a 1 or several processor perform a process by the model definition part 4, the form identification part 6, and the character recognition part 7 collectively. Note that the processing by the model definition unit 4, the form identification unit 6, and the character recognition unit 7 is not necessarily distributed.

  The storage unit S stores one or more programs that control the operation of the processor of each unit. For example, the storage unit S stores one or more programs that control operations such as model definition processing by the model definition unit 4, form identification processing by the form identification unit 6, and character recognition processing by the character recognition unit 7. In other words, the model definition unit 4, the form identification unit 6, and the character recognition unit 7 operate based on one or more programs stored in the storage unit S.

  For example, the above-described processing can be easily realized simply by installing and executing the above-described program in the form processing apparatus 1 through a computer-readable storage medium. For example, the form processing apparatus 1 can download the program via a network, store the downloaded program, and complete the installation of the program. Alternatively, the form processing apparatus 1 can read the program from the information storage medium, store the read program, and complete the installation of the program.

FIG. 2 is a block diagram illustrating an application example of the form processing apparatus according to the embodiment.
As shown in FIG. 2, the form processing apparatus 1 includes a form image capturing unit 2, a form image storage unit 3, a model definition unit 4, a model storage unit 5, a form identification unit 6, a character recognition unit 7, and a character recognition result storage. Unit 8, screen control unit 9, and user interface unit 10.

  The form image capturing unit 2 includes a scanner 21 and a form image writing unit 22. The scanner 2 optically reads a form Doc such as an invoice and outputs a form image corresponding to the form Doc. The form image writing unit 22 assigns an ID (identification information) to the form image output from the scanner 2 and writes the form image associated with the ID in the form image storage unit 3. For example, the form image storage unit 3 stores the form image in association with the ID.

  The form image storage unit 3, the model definition unit 4, the model storage unit 5, and the form identification unit 6 are as described above, and detailed description thereof is omitted.

  The screen control unit 9 includes a form image input / output unit 91 and a screen generation unit 92. The form image input / output unit 91 reads out the form image stored in the form image storage unit 3 and outputs it to the screen generation unit 92. The screen generation unit 92 generates a screen based on the read form image and outputs it to the user interface unit 10 (display 101).

  The user interface unit 10 includes a display 101, a touch panel 102, a keyboard 103, a mouse 104, and the like. The display 101 displays a screen based on the generated form image. The touch panel 102, the keyboard 103, and the mouse 104 select a button displayed on the screen of the display 101 or specify an area on the screen in accordance with an input from the user.

  The character recognition unit 7 includes a dictionary database 71 and an OCR engine 72. Based on the dictionary data registered in the dictionary database 71, the OCR engine 72 recognizes various information such as characters, numbers, and symbols included in the form image output from the form identifying unit 6. The OCR engine 72 efficiently recognizes various information such as characters included in the form image by the character recognition process according to the form model (form type). Character recognition processing according to the form model is a character recognition process with a character recognition position specified according to the form model, a character recognition process with a character size specified according to the form model, and a recognition target according to the form model. This is a character recognition process in which the type of is specified. Although three character recognition processes are given as examples, at least one character recognition process can be a character recognition process corresponding to a form model. The character recognition result storage unit 8 stores the character recognition result by the character recognition unit 7.

  FIG. 3 is a flowchart illustrating an example of model definition processing by the form processing apparatus according to the embodiment.

  In response to execution of the model definition mode, the form image capturing unit 2 captures the form image of the form Doc to be model defined (STJ1), and stores the form image of the form Doc in the form image storage unit 3. .

  In response to execution of the model definition mode, the model definition unit 4 receives a form image from the form image storage unit 3 and causes the display 101 to display the form image of the form image storage unit 3 via the screen control unit 9. . FIG. 6 is a diagram for explaining an example of a form image (image display area) displayed on the display and for explaining feature points and reading position designation.

  The model definition unit 4 performs model definition based on the form image (STJ2). The model definition unit 4 generates read definition information as a feature point based on a user input from the user interface unit 10. Further, the model definition unit 4 detects ruled line intersection information as a feature point from the form image. FIG. 5A is a diagram illustrating an example of read definition information generated by the model definition unit. FIG. 5B is a diagram illustrating an example of ruled line intersection information detected by the model definition unit. The model definition unit 4 generates read definition information as feature points associated with the position information of the form image, and detects ruled line intersection information as feature points associated with the position information of the form image.

  FIG. 6 is a diagram illustrating an example of a ruled line intersection corresponding to a form image. As shown in FIGS. 5B and 6, a plurality of ruled line intersections are detected from the form image, and position information (X coordinate and Y coordinate) is associated with the ruled line intersections.

  The user designates a reading position and a recognition target position such as a character via the touch panel 102 or the mouse 104 while referring to the form image (display 101) in FIG. 6, and the model definition unit 4 associates with the position information. The read position and the recognition target position such as characters are detected. Further, the user inputs an item name such as an amount, name, address, or remarks column corresponding to the reading position via the keyboard 103 or the like while referring to the definition information display area (display 101) in FIG. Further, a reading attribute corresponding to the reading position is selected from the pull-down list. For example, numbers, names, addresses, or characters are selected as reading attributes.

  The model definition unit 4 generates a form model including ruled line intersections associated with the position information of the form image, read definition information associated with item names and attributes, and recognition target positions, and stores the form model as a model. Stored in part 5 (STJ3).

  When there are a plurality of types of forms, the processes of STJ1 to STJ3 are repeated, and the form models corresponding to these types of forms are stored in the model storage unit 5.

  FIG. 4 is a flowchart illustrating an example of a form identification process and a character recognition process performed by the form processing apparatus according to the embodiment.

  The form image capture unit 2 captures the form image of the form Doc to be identified in response to the execution of the form identification and character recognition mode (STJ4), and inputs the form image to the form identification unit 6. Alternatively, the form image capture unit 2 takes out the form image stored in the form image storage unit 3 and inputs the form image to the form identification unit 6 in response to the execution of the form identification and character recognition mode.

  The form identification unit 6 receives a form image and receives a defined form model from the model storage unit 5. 8A and 8B are diagrams illustrating an example of an input form image. 9A and 9B are diagrams showing the concept of each form model (definition information) corresponding to a plurality of types of forms.

  The form identification unit 6 compares the input feature point of the form image shown in FIGS. 8A and 8B with the model feature point in each form model shown in FIGS. 9A and 9B, and has the highest similarity to this form image. Identify high form models. For example, as shown in FIG. 10, the form identifying unit 6 uses each form model and form based on the angle formed between the model feature points of each form model corresponding to a plurality of types of forms and the input feature points of the form image. The similarity with the image is determined, and a predetermined form model corresponding to the form image is identified (specified) based on the determination result of the similarity (STJ5). For example, the degree of similarity may be determined by comparing the angle formed between the model feature points corresponding to the ruled line intersection and the input feature points corresponding to the ruled line intersection, or between the model feature points corresponding to the reading definition information The similarity may be determined by comparing the angle formed between the input feature points corresponding to the reading definition information. Further, the similarity may be determined by comparing the angle formed between the model feature points corresponding to all of the ruled line intersection and the read definition information and the input feature point corresponding to all of the ruled line intersection and the read definition information. .

  The character recognition unit 7 recognizes information included in the form image based on recognition target position information such as characters associated with the predetermined form model identified in STJ5 (STJ6). The character recognition unit 7 stores the character recognition result obtained in STJ6 in the character recognition result storage unit 8 (STJ7).

  Here, details of detection of feature points and determination of similarity based on angles formed between feature points will be described.

  The form identifying unit 6 performs character recognition on the form image (unidentified) based on the reading position included in the form model and the reading attribute associated with the reading position, and information corresponding to the reading attribute is recognized. In such a case, the recognition result is detected as a feature point. For example, if the reading attribute is a number, when the number is recognized from the reading position, the recognition result is detected as a feature point. In addition, the form identification unit 6 performs image processing on the form image (unidentified) and detects ruled line intersection points as feature points.

  The form identifying unit 6 compares the model feature points from each form model with the input feature points from the form image (unidentified), and identifies a predetermined form model corresponding to the form image. For example, the form identifying unit 6 determines the correspondence between each model feature point and each input feature point based on each model feature point associated with the position information and each input feature point associated with the position information. The similarity between each form model and the form image is determined based on the angle formed between the model feature points and the input feature points that are in a correspondence relationship.

  For example, the form identifying unit 6 sorts each model feature point sequentially based on the position information of each model feature point, sorts each input feature point sequentially based on the position information of each input feature point, A correspondence relationship between input feature points is determined.

  For example, the form identifying unit 6 sorts the model feature points sequentially based on the coordinate information, and similarly sorts the input feature points sequentially based on the coordinate information, and the model feature points and the input feature points when traced in order. The angle formed by is calculated by the inner product. The degree of similarity between the form model and the form image is calculated by adding points if the angle difference is small and subtracting if the angle difference is large, and the form model having the highest degree of similarity is adopted as the identification result.

  For example, the similarity P for each form model is calculated by the following equation.

The definition in the formula is as follows.
Va: vector formed by input feature points Vb: vector formed by model feature points θ: inner product threshold S: 1 if inner product of Va and Vb is larger than threshold, 0 otherwise
P: A value obtained by dividing the total value of S by the number n of vectors.

According to the form processing apparatus of the present embodiment, the following operational effects can be obtained.
For example, even if there is no unique character string in the form, it can be identified if there is a ruled line. Alternatively, even if there is no unique character string in the form, it can be identified if there is a character recognition target such as a number. Alternatively, even if there is no unique character string in the form, it can be identified if there are character recognition targets such as ruled lines and numbers.

  In addition, character recognition processing is not executed for form identification, and the similarity is obtained based on a comparison between an angle formed between model feature points and input feature points of each form model corresponding to a plurality of types of forms and a threshold value. Even if the number of form models is enormous, it is hardly affected by the processing time. For example, by obtaining the similarity by calculating the inner product of the corresponding feature points, it is possible to make it less susceptible to the processing time even if the number of form models is enormous.

  In addition, the similarity is calculated based on a comparison between the threshold value and the angle between the model feature points and the input feature points of each form model corresponding to multiple types of forms, so it is robust against the effects of expansion and contraction and printing misalignment. . Further, since expansion / contraction and printing misalignment can be obtained by inner product calculation, the reading position can be corrected when character recognition is performed after form identification to prevent misreading due to expansion / contraction or printing misalignment.

Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.
Hereinafter, the invention described in the scope of claims at the beginning of the application of the present application will be added.
[C1]
An input part for inputting a form image of the form to be identified;
A detection unit for detecting an input feature point from the input of the form image;
The similarity between each form model and the form image is determined based on the angle formed between the model feature points of each form model corresponding to a plurality of forms and the input feature points, and based on the determination result of the similarity An identification unit for identifying a predetermined form model corresponding to the form image;
A form processing apparatus comprising:
[C2]
The identification unit determines a correspondence between each model feature point and each input feature point based on each model feature point associated with the position information and each input feature point associated with the position information, The form processing apparatus of [C1] that determines the similarity between each form model and the form image based on an angle formed between the model feature points and the input feature points.
[C3]
The identification unit sequentially sorts each model feature point based on position information of each model feature point, sorts each input feature point sequentially based on the position information of each input feature point, and each model feature point and each input feature point [C2] form processing apparatus for determining the correspondence relationship.
[C4]
The form processing device according to any one of [C1] to [C3], wherein the model feature point and the input feature point are intersections of ruled lines.
[C5]
The form processing apparatus according to any one of [C1] to [C4], comprising a recognition unit that recognizes information included in the form image based on recognition target position information associated with the predetermined form model.
[C6]
[C5] The form processing apparatus according to [C5], wherein the recognition unit recognizes information included in the form image using at least one of characters, numbers, and symbols as information to be recognized.
[C7]
Enter the form image of the form to be identified,
An input feature point is detected from the input of the form image,
The similarity between each form model and the form image is determined based on the angle formed between the model feature points of each form model corresponding to a plurality of forms and the input feature points, and based on the determination result of the similarity A form processing method for identifying a predetermined form model corresponding to a form image.

DESCRIPTION OF SYMBOLS 1 ... Form processing apparatus 2 ... Scanner 3 ... Form image storage part 4 ... Model definition part 5 ... Model storage part 6 ... Form identification part 7 ... Character recognition part 8 ... Character recognition result storage part 9 ... Screen control part 10 ... User interface Unit 21 ... Scanner 22 ... Form image writing unit 71 ... Dictionary database 72 ... Engine 91 ... Form image input / output unit 92 ... Screen generation unit 101 ... Display 102 ... Touch panel 103 ... Keyboard 104 ... Mouse

Claims (8)

An input unit for inputting a model target form image and an identification target form image corresponding to a plurality of types of forms;
A model definition unit that defines each form model corresponding to the plurality of types of forms based on the model target form image;
A detection unit for detecting an input feature point from the input of the identification target form image;
The similarity between each form model and the identification target form image is determined based on the angle formed between the model feature points of each form model and the input feature points, and the identification target form image is determined based on the similarity determination result. An identification unit for identifying a corresponding predetermined form model;
Equipped with a,
The identification unit is
In determining the correspondence between each model feature point and each input feature point based on each model feature point associated with the location information and each input feature point associated with the location information, the location information of each model feature point The model feature points are sorted in order based on the input feature points, the input feature points are sorted in order based on the position information of the input feature points, the correspondence between each model feature point and each input feature point is determined,
A form processing apparatus that determines the degree of similarity between each form model and the form image to be identified based on an angle formed between the model feature points and the input feature points in correspondence . The form processing apparatus according to claim 1 , wherein the model feature point and the input feature point are intersections of ruled lines. It said predetermined based on the recognized position information associated with the form model, the identification document processing apparatus according to claim 1 or 2 comprising a recognition unit recognizes the information included in the target document image. The form processing apparatus according to claim 3 , wherein the recognition unit recognizes information included in the identification target form image using at least one of characters, numbers, and symbols as information to be recognized.   The form processing apparatus according to claim 1, further comprising an interface for designating a reading position for the model target form image in response to an input from a user.   The form processing apparatus according to claim 5, wherein the interface is a touch panel or a mouse. The model definition unit is configured to specify the reading position, the reading attribute, and the model feature point based on designation of a reading position and a reading attribute for the model target form image and detection of the model feature point from the model target form image. one of form processing apparatus of claims 1 to 5 defining the form model that associates. Enter the model target form image corresponding to multiple types of forms,
Define each form model corresponding to the plurality of types of forms based on the model target form image,
Enter the form image to be identified,
Detecting input feature points from the input of the identification target form image,
Determining the similarity between each form model and the form image to be identified based on the angle formed between the model feature points of each form model corresponding to a plurality of types of forms and the input feature points ;
A method for identifying a predetermined form model corresponding to the identification target form image based on a similarity determination result ,
Determining the similarity is:
In determining the correspondence between each model feature point and each input feature point based on each model feature point associated with the location information and each input feature point associated with the location information, the location information of each model feature point Sorting each model feature point in order, sorting each input feature point sequentially based on the position information of each input feature point, determining the correspondence between each model feature point and each input feature point;
A form processing method comprising: determining a similarity between each form model and the form image to be identified based on an angle formed between the model feature points and the input feature points in a correspondence relationship .