JP2020087112A - Document processing apparatus and document processing method - Google Patents

Abstract

To improve accuracy of reading even if various kinds of documents coexist, while suppressing increase of cost.SOLUTION: A position collation unit 104 receives document image data 113 and document definition data 110b, and extracts a character string image 114 serving as candidates of character string to be read, from the document image data 113, on the basis of position information 111b included in the document definition data 110b. A character recognition unit 105 receives the character string image 114, and performs character recognition processing on the basis of the character string image 114. An attribute collation unit 106 receives a character code 115 and the document definition data 110b, and determines a read result 116 on the basis of a collation result on the character code 115 using attribute information 112b included in simple document definition data 110b.SELECTED DRAWING: Figure 1

Description

The present invention relates to a form processing device and a form processing method capable of reading a form.

Conventionally, when recognizing a character string on a form, the position, attribute, character type of the character string to be recognized, the size and type of the border line around the character string, the pre-printed characters written in the border line, etc. It is registered as definition data in advance, and the character area is determined and recognized based on the form definition data.

Also, when processing is performed in an environment in which many types of forms are mixed, it is necessary to register multiple form definition data, specify the form type to be processed, and then select the appropriate form definition data. .. On the other hand, there is also a technique of automatically extracting candidate item names and item values in a form without using the form definition data and determining a likely read item based on their positional relationship.

As a technique for creating the form definition data, there is a technique described in Patent Document 1, for example. In Patent Document 1, "definition data is automatically created based on layout information such as preprinted characters around or inside a designated reading area, entered characters and layout information such as frames and ruled lines automatically extracted when inputting definition image data." There is a description.

As a technique for specifying the form type, there is a technique described in Patent Document 2, for example. In Patent Document 2, "In the present invention, a frequency distribution of pixel values of a registration color form image and a frequency distribution of pixel values of a color form image to be processed are created. Correlation coefficient of frequency distribution for each color component Then, the similarity between color images is calculated from the correlation coefficient.When the highest similarity is a predetermined value or more, the registered color image with the highest similarity is the same type as the color form image to be processed. It is determined that it is."

As a technique for recognizing a character string on a form without using the form definition data, for example, there is a technique described in Patent Document 3. Patent Document 3 describes "a character string detection unit that detects a character string region from a form image, a character string recognition unit that recognizes each character in the character string region, and a character string in a form image with respect to the character. An item name likelihood calculator that calculates the likelihood of an item name representing the probability that the column is an item name, and a grammatical notation of the word or character string registered in the dictionary for the character string in the form image An item value likelihood calculation unit that calculates an item value likelihood that represents the probability of matching the rule, and a character string pair in the form image, based on the frame of the character string of the character string pair or the character string rectangle, Based on the above-mentioned item name likelihood, item value likelihood, and placement likelihood, a placement likelihood calculation unit that calculates a placement likelihood that indicates whether the placement relationship of character string pairs is valid as an item name-item value relationship. , An item name of the character string pair-an item name-item value relationship evaluation value calculation unit that calculates an evaluation value representing likelihood as an item value, and the evaluation output by the item name-item value relationship evaluation value calculation unit It is characterized by having an item name-item value relationship determining unit that determines the correspondence of the item name-item value relationship in the form image depending on the value."

JP 2004-258706 A JP, 2002-24829, A JP, 2002-24829, A

However, the technologies of Patent Documents 1 and 2 do not assume various situations of the form type to be processed.

That is, although the technique of Patent Document 1 realizes partial automation of the form definition data creation, a high cost is required to create various form definitions. In the technique of Patent Document 2, there are many types of forms, and it is difficult to accurately identify a character string when various types of forms are mixed.

In Patent Document 3, it is difficult to read the form in which the item name does not exist and only the item value is written. In addition, when there are a plurality of likely item value candidates corresponding to the item name, the reading accuracy of the form may decrease.

The present invention has been made in view of the above circumstances, and an object thereof is a form processing apparatus capable of improving the reading accuracy of a form while suppressing an increase in cost even when various types of forms are mixed. And to provide a form processing method.

In order to achieve the above object, the form processing apparatus according to the first aspect provides a comparison result between position information of a character string image extracted from form image data and position information defined for a read target character string. A position matching unit that determines a character string image that is a candidate for the reading target character string, a character recognition unit that performs character recognition based on the character string image that is a candidate for the reading target character string, and the character recognition Attribute matching that determines the reading result of the character string image that is a candidate for the reading target character string, based on the matching result of the attribute of the character recognition result by the section and the attribute information defined for the reading target character string. And a section.

According to the present invention, even when various types of forms are mixed, it is possible to improve the reading accuracy of the forms while suppressing an increase in cost.

FIG. 1 is a block diagram showing the configuration of the form processing apparatus according to the first embodiment. FIG. 2 is a diagram showing a screen display example of the form image data presented to the creator of the form definition data. FIG. 3 is a diagram showing an example of the data structure of simplified form definition data used in the form processing apparatus according to the first embodiment. FIG. 4 is a diagram showing a correspondence relationship between the feature amount extracted from the form image data and the form definition data. FIG. 5 is a flowchart showing the processing of the form processing apparatus according to the first embodiment. FIG. 6 is a block diagram showing the configuration of the form processing apparatus according to the second embodiment. FIG. 7 is a flowchart showing the processing of the form processing apparatus according to the second embodiment. FIG. 8 is a block diagram showing the hardware configuration of the form processing apparatus according to the third embodiment.

Embodiments will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are essential to the solution means of the invention. Not necessarily.

FIG. 1 is a block diagram showing the configuration of the form processing apparatus according to the first embodiment.
In FIG. 1, the form processing apparatus includes a scanner 101, a feature extraction unit 102, a form definition database 103, a position matching unit 104, a character recognition unit 105, an attribute matching unit 106, a reading result database 107, and a form definition data creation unit 108. .. The form definition data creation unit 108 is connected to the display 109.

The scanner 101 receives a form as an input, irradiates the form with light using a lamp (not shown), converts the transmitted light into an electric signal using an image pickup device (not shown), and outputs the form image data 113. The form image data 113 includes a character string image of a character string entered on the form. The scanner 101 may output color form image data or monochrome form image data. The scanner 101 may apply pre-processing for reducing noise, moiré, show-through, etc. to the form image data 113.

The feature extraction unit 102 receives the form image data 113 as input, extracts feature amounts 114a and 114b relating to a plurality of predetermined features from the form image data 113, and outputs them to the form definition database 103. As characteristics of the form image data 113, for example, brightness histogram information of the entire form, connection relations of frame lines described in the form, connection relations of intersections of frame lines, and the like can be set. The feature extraction unit 102 extracts the feature amounts 114a and 114b by digitizing the features included in the form image data 113.

The feature amount 114a is stored in the form definition database 103 in a state of being associated with the form definition data 110a created by the form definition data creating unit 108. The feature amount 114b is used to select the form definition data 110b output to the position matching unit 104 and the attribute matching unit 106 from the form definition data 110a. At this time, the form definition database 103 can compare the feature amount 114b with the feature amount 114a and select the form definition data 110b based on the comparison result.

The form definition database 103 can improve the reliability of extraction of the form definition data 110b by selecting the form definition data 110b based on the comparison result of the feature amount 114a and the feature amount 114b.

The feature extraction unit 102 may extract features from the form image data 113 using machine learning. For example, a large number of form image data 113 are collected and classified into form types. Next, a neural network model with a convolutional layer in the front stage and a fully connected layer in the rear stage is set, and the input is the form image data 113, the output is the form type, and the collected form image data is used as teacher data to create a multi-class classifier. To construct. The convolutional layer of the neural network model constructed by the above procedure can be used as a feature extractor. At this time, when the form image data 113 is input to the convolutional layer of the neural network model, the feature quantities 114a and 114b are output.

As described above, by extracting features using machine learning, it is possible to automate feature selection and feature extractor construction. Further, when the multi-class classifier can be constructed with high accuracy, the convolutional layer extracted from the multi-class classifier can be expected to be a feature extractor that accurately extracts the features of the form image.

The form definition data creation unit 108 receives the form image data 113 and presents the form image data 113 to the creator of the form definition data 110a. Then, the form definition data creation unit 108 creates the form definition data 110 a based on the position and attribute specification of the read target character string by the creator and outputs it to the form definition database 103.

FIG. 2 is a diagram showing a screen display example of the form image data presented to the creator of the form definition data.
In FIG. 2, the screen 301 is displayed on the display 109 of FIG. The screen 301 displays the form image data 113, the cursor 303, and the confirm button 304.

The creator of the form definition data 110a can move the cursor 303 on the screen 301 using a pointing device (not shown). As the pointing device, for example, a mouse, a touch pen, a touch panel, or the like can be used.

It is assumed that the creator of the form definition data 110a creates, for example, the form definition data 110a in which the character strings entered in the positions P1 to P4 of the form image data 113 are defined as the read target character strings.

At this time, the creator of the form definition data 110a specifies the positions P1 to P4 of the form image data 113 and the attributes of the character strings at the positions P1 to P4.

For example, the creator of the form definition data 110a specifies the position P2 of the form image data 113 and the attribute of the character string at the position P2. At this time, the creator of the form definition data 110a points the upper left coordinate P2-1 of the position P2 with the cursor 303, and presses the button of the pointing device so that the upper left coordinate P2-1 of the position P2 of the form image data 113 is changed. , And is defined as the position information 111a of the character string to be read at the position P2. Further, the creator of the form definition data 110a points the lower right coordinate P2-2 of the position P2 with the cursor 303, and presses the button of the pointing device, whereby the lower right coordinate P2-2 of the position P2 of the form image data 113. Is defined as position information 111a of the read target character string at position P2.

At this time, the screen 301 displays the range indicated by the position information 111a in a rectangular shape. For example, the screen 301 displays a rectangle having an upper left coordinate P2-1 and a lower right coordinate P2-2 as opposite vertices for the read target character string at the position P2.

Next, the creator of the form definition data 110a moves the cursor inside the rectangle representing the position P2 with the upper left coordinate P2-1 and the lower right coordinate P2-2 set for the read target character string at the position P2, for example. A predetermined attribute group is displayed by pointing at 303 and pressing the button of the pointing device. Then, the creator of the form definition data 110a specifies an attribute from the attribute group, and the specified attribute is defined as the attribute information 112a of the read target character string at the position P2. At this time, the attribute information 112a of the read target character string at the position P2 is displayed near the rectangle representing the position P2. For example, when the attribute information 112a of the character string to be read at the position P2 is the branch name, the attribute information 112a called the branch name is displayed near the rectangle representing the position P2.

Further, the creator of the form definition data 110a points to the rectangle representing the position information 111a with the cursor 303 in the state where the attribute information 112a is set for the read target character string, and presses the button of the pointing device in advance. Display the set attributes. Then, the creator of the form definition data 110a specifies an attribute different from the currently set attribute information 112 from the attribute group, so that the attribute different from the currently set attribute information 112a is set to the cursor 303. It is defined as the attribute information 112a of the character string to be read at the position pointed to by. At this time, the newly defined attribute information 112a is displayed near the rectangle representing the position pointed by the cursor 303.

By repeating the above operation, the creator of the form definition data 110a defines a set of position information 111a and attribute information 112a of each read target character string for each position P1 to P4 of the form image data 113.

Then, the creator of the form definition data 110a sets a set of the position information 111a of the read target character string and the attribute information 112a for each of the positions P1 to P4 of the form image data 113, and sets the confirm button 304 with the cursor 303. By pointing and pressing the button of the pointing device, the set of the position information 111a and the attribute information 112a of the read target character string is fixed.

When the set of the position information 111a of the read target character string and the attribute information 112a is determined, the form definition data creation unit 108 defines the set of the position information 111a of the read target character string and the attribute information 112a. The data 110a is created and output to the form definition database 103.

In this way, by adopting the screen configuration of FIG. 2, the creator of the form definition data 110a can easily set the position information 111 and the attribute information 112 by a simple operation using a pointing device, and the form definition data 110a can be easily set. The definition data 110a can be created at low cost.

When creating the form definition data 110a, the form definition database 103 stores the form definition data 110a for the form image data 113 in association with the feature amount 114a of the form image data 113. Further, the form definition database 103 selects the form definition data 110b based on the feature amount 114b of the form image data 113 when reading the form. Then, the position information 111b included in the form definition data 110b is output to the position matching unit 104, and the attribute information 112b included in the form definition data 110b is output to the attribute matching unit 106.

That is, when creating the form definition data 110a, the form definition database 103 stores the form definition data 110a and the feature amount 114a extracted by the feature extraction unit 102 from the form image data 113 used when the form definition data 110a was created. As an input, the form definition data 110a and the feature amount 114a are stored in association with each other.

The form definition database 103 stores a plurality of form definition data 110a by linking and adding the form definition data 110a and the feature amount 114a each time the form definition data 110a and the feature amount 114a are newly input.

Further, the form definition database 103 receives the feature amount 114b as an input at the time of reading the form, and extracts one or more form definition data from the form definition data 110a based on the input feature amount 114b to form the form definition. The data 110b is output.

That is, the form definition database 103 calculates the distance D from the input feature amount 114b for all the stored feature amounts 114a according to Formula 1, and the distance D from the feature amount 114b is equal to or less than a preset threshold TD. The form definition data 110a associated with the feature amount 114a of 1 is output as the form definition data 110b.

Here, Fa is a vector indicating the characteristic amount 114a, Fb is a vector indicating the characteristic amount 114b, and · is a dot product.

At this time, the form definition database 103 can extract the form definition data 110b associated with the form image data 113 that is similar in terms of the feature amount.

The form definition database 103 may consider that acquisition of the form definition data 110b has failed when one or more form definition data 110b cannot be extracted. The operation when the acquisition of the form definition data 110b fails will be described later with reference to the flowchart of FIG.

The position matching unit 104 receives the form image data 113 and the form definition data 110b as input, and based on the position information 111b included in the form definition data 110b, extracts the character string image 114 that is a candidate for the read target character string from the form image data 113. And output to the character recognition unit 105.

At this time, the position matching unit 104 does not need to extract all the character strings included in the form image data 113, and may just extract the read target character strings defined in the form definition data 110b without omission. In other words, the position matching unit 104 does not have to extract the form name, the item name in the table, the character string image related to various cautionary notes, and the like that are not normally read target character strings. Further, since the character string to be read is rarely included near the end of the form, the position matching unit 104 may extract the character string image 114 from a position other than the end of the form.

By defining the position of the reading target character string in this way, the character string image 114 extracted from the form image data 113 can be reduced, and the processing speed can be increased while maintaining the reading accuracy. it can.

Any method can be used to extract the character string image from the form image data 113 in the position matching unit 104. For example, the method described in Japanese Patent No. 2991761 may be used.

Next, the position matching unit 104 determines the character string image 114, which is the shortest distance from the position specified by the position information 111b, for the character string image extracted from the form image data 113 according to Formula 2, and the character recognition unit 105. Output to. That is, the position matching unit 104 extracts the character string image 114 existing closest to the position specified by the position information 111b, and outputs it to the character recognition unit 105.

Here, La is the central coordinate of the extraction source of the character string image, and Lb is the central coordinate of the position information 111b included in the form definition data 110b. The center coordinates are obtained by an average value of the upper left coordinates P2-1 and the upper left coordinates P2-2 included in the position information 111b. When there are a plurality of character string images extracted from the form image data 113, those character string images are distinguished by the subscript i.

When the form definition data 110b includes a plurality of position information 111b, the position matching unit 104 determines the shortest distance character string image 114 for each position information 111b according to Formula 2, and the character recognition unit Output to 105.

When there are a plurality of form definition data 110b output from the form definition database 103, the position matching unit 104 determines the character string image having the shortest distance to each form definition data 110b by the method described above.

Next, the position matching unit 104 calculates an evaluation value E for each form definition data 110b according to Formula 3, and determines the character string image 114 based on the form definition data 110b having the smallest evaluation value E, It is output to the character recognition unit 105.

Here, L'a indicates the center coordinates of the extraction source of the character string image determined for each form definition data 110b, and Lb indicates the center coordinates of the position information 111b. The center coordinates are obtained by an average value of the upper left coordinates P2-1 and the upper left coordinates P2-2 included in the position information 111b. When there are a plurality of position information 110b in the form definition data 110b, they are distinguished by the subscript j.

The position matching unit 104 may consider that the position matching has failed when the progress of the process and the situation where the result is regarded as unreliable. For example, the position matching unit 104 can consider that the position matching has failed when the minimum evaluation value E exceeds the preset threshold value TE. The operation when the position verification fails will be described later with reference to the flowchart of FIG.

The character recognition unit 105 receives the character string image 114 as input, performs character recognition processing based on the character string image 114, and outputs the character recognition result as a character code 115.

That is, the character recognition unit 105 cuts out the character string image 114 for each character, converts each character into a feature amount, and calculates a distance from the feature amount of the character registered in a character database (not shown), The character code 115 of the shortest distance is output to the attribute matching unit 106.

The character recognition unit 105 may consider that the character recognition has failed when the progress of the process and the situation in which the result is considered unreliable. For example, when the shortest distance between feature amounts exceeds a preset threshold, the character recognition unit 105 can assume that character recognition with high reliability cannot be performed, and consider that character recognition has failed. .. The operation when the character recognition fails will be described later with reference to the flowchart of FIG.

The attribute matching unit 106 receives the character code 115 and the form definition data 110b as input, determines the reading result 116 based on the matching result for the character code 115 using the attribute information 112b included in the simplified form definition data 110b, and reads the reading result. Output to the database 107.

That is, the attribute matching unit 106 prepares a verification process according to the type of the attribute of the read target character string, performs the matching process of the character code 115 according to the attribute information 112b, and the attribute indicated by the character code 115 seems to be certain. In that case, the character code 115 is output as the reading result 116.

For example, the attribute collation unit 106 inquires a bank name database that stores bank names nationwide (not shown) for a character string as a verification process for the attribute "bank name", and the bank name database contains the character string. If so, it is possible to prepare a verification process in which the character string is considered as the attribute "bank name".

In addition, for example, the attribute matching unit 106 prepares, as the verification processing of the attribute “account number”, the verification processing “if the character string is composed of 7 digits or less, the character string is considered to be certain”. You can

The attribute collating unit 106 may consider that the attribute collation has failed when the progress of the process and the situation in which the result is regarded as unreliable. For example, in the verification process of the attribute “bank name”, the attribute matching unit 106 can consider that the attribute matching has failed if the bank name database does not include a character string. The operation when the attribute matching fails will be described later with reference to the flowchart of FIG.

The reading result database 107 receives the reading result 116 and stores the reading result 116. Further, the read result database 107 presents the read result 116 to the user and calls or stores the read result 116 from a read result correction unit (not shown) that stores the read result corrected by the user in the read result database 107. If requested, follow each request.

FIG. 3 is a diagram showing an example of the data structure of simplified form definition data used in the form processing apparatus according to the first embodiment.
In FIG. 3, the form definition data 110a has a data structure in which a plurality of sets of position information 111a and attribute information 112a of a read target character string are stored. For example, the character strings written in the respective positions P1 to P4 of the form image data 113 in FIG. 2 can be set as the read target character strings 1 to 4. At this time, in the form definition data 110a, a set of position information 111a and attribute information 112a is registered for each of the read target character strings 1 to 4.

The position information 111a can represent the upper left coordinate P2-1 and the upper left coordinate P2-2 of the read target character string in units of one pixel in the resolution of the form image data 113, respectively.

However, the position information 111a is not limited to one pixel unit. For example, the position information 111 may hold the upper left coordinate P2-1 and the upper left coordinate P2-2 in units that are rougher than the resolution of the form image data 113, for example, in units of 2 pixels or 5 pixels. At this time, the screen 301 may display a rectangle indicating the position of the read target character string in units used for holding the position information 111a.

Further, the position information 111a may be held in millimeters with the form input to the scanner 101 of FIG. 1 as a reference. In this case, the screen 301 may perform an appropriate scaling process to display a rectangle indicating the position of the read target character string. The setting of this unit is related to the reading accuracy of the entire form reading apparatus, and if the unit is roughened, the form definition can be performed quickly, but since the reading accuracy will be lowered, it is recommended to set this balance. Good.

By setting the position information 111a of the read target character string in this way, not only the information amount of the form definition data 110a can be reduced and the recording amount can be reduced, but also in the form definition data generation unit 108, with higher precision than necessary. It is possible to suppress the time and effort to generate the form definition data 110a.

The attribute information 112a is information indicating the type of the read target character string. For example, in a financial report, a bank name, a branch name, an account type, an account number, a transfer date, a name and an address can be set as the attribute information 112a.

As described above, the form definition data 110a includes the position information 111a and the attribute information 112a, and includes information such as the frame shape, the number of characters, handwriting/printing characters, preprinted characters, and character pitch, which are included in normal definition data. Absent. Therefore, the form definition data creating unit 108 can easily create the form definition data 110a, and the labor and cost for creating the form definition data 110a can be reduced.

FIG. 4 is a diagram showing a correspondence relationship between the feature amount extracted from the form image data and the form definition data. In addition, in the example of FIG. 4, the case where each feature amount 114a, 114b is composed of two feature amounts A, B is shown.
In FIG. 4, the form definition database 103 stores each form definition data 110a in association with the feature amount 114a of the form image data 113 used to create the form definition data 110a. When the feature amount 114b is input from the feature extraction unit 102, the form definition database 103 selects the form definition data 110a associated with the feature amount 114a whose distance D from the feature amount 114b is equal to or less than the threshold TD, The selected form definition data 110a is output as the form definition data 110b.

FIG. 5 is a flowchart showing the processing of the form processing apparatus according to the first embodiment.
In FIG. 5, in step S101, the scanner 101 scans a form, converts it into form image data 113, and outputs the form image data 113 to the feature extraction unit 102, the position matching unit 104, and the form definition data creation unit 108.

Next, in step S102, the feature extraction unit 102 extracts the feature amounts 114a and 114b from the form image data 113 and outputs them to the form definition database 103.

Next, in step S103a, the form definition database 103 extracts one or more form definition data 110b corresponding to the feature amount 114b.

Next, in step S103b, the form definition database 103 determines whether acquisition of the form definition data 110b in step S103a has failed. If the form definition data 110b is successfully acquired, the process proceeds to step S104a. If the form definition data 110b is not successfully acquired, the process proceeds to step S108a.

Next, in step S104a, the position matching unit 104 extracts the character string image 114 from the form image data 113 based on the position information 111b included in the form definition data 110b.

Next, in step S104b, the position matching unit 104 determines whether the position matching in step S104a has failed. When the position matching is successful, the process proceeds to step S105a, and when the position matching is unsuccessful, the process proceeds to step S108a.

Next, in step S105a, the character recognition unit 105 character-recognizes the character string image 114 and converts it into the character code 115.

Next, in step S105b, the character recognition unit 105 determines whether or not the character recognition in step S105a has failed. If the character recognition is successful, the process proceeds to step S106a. If the character recognition is unsuccessful, the process proceeds to step S108a.

Next, in step S106a, the attribute matching unit 106 verifies the character code 115 based on the attribute information 112b included in the form definition data 110b, and obtains the reading result 116 of the character code 115.

Next, in step S106b, the attribute matching unit 106 determines whether the attribute matching in step S106a has failed. If the attribute collation succeeds, the process proceeds to step S107. If the position collation fails, the process proceeds to step S108a.

Next, in step S107, the read result database 107 stores the read result 116 output from the attribute matching unit 106.

Next, in step S108a, the form definition creating unit 108 determines whether the form definition data 110a needs to be created. If the form definition data 110a needs to be created, the process proceeds to step S108b, and if the form definition data 110a does not need to be created, the process ends.

The form definition data needs to be created when it is estimated that the form could not be read correctly. Specifically, it is the following case.
(1) When acquisition of the form definition data 110b fails in step S103a.
(2) In step S104a, when the position verification fails.
(3) When character recognition fails in step S105a.
(4) In step S106a, when collation of attributes fails.
(5) In step S103a, when the output form definition data 110b is unreliable. That is, when the threshold value TD′ satisfying TD′<TD is set in advance, the relationship of threshold value TD′<distance D<threshold value TD is satisfied.
(6) In step S104a, when the output character string image 114 is not reliable. That is, when the threshold TE'<TE'<TE is preset and the relationship of threshold TE'<distance E<threshold TE is satisfied.
(7) In the case where the read result 116 stored in the read result database 107 is corrected in the read result correction step (not shown).

Next, in step S108b, the form definition creation unit 108 generates the form definition data 110a based on the form image data 113.

Next, in step S103c, the form definition database 103 stores the form definition data 110a, and the process ends.

Note that step S104b, step S105b, or step S106b can be omitted. When this operation is adopted, when there are a plurality of reading target character strings, the processing can be completed even if reading of a part of the character strings fails.

As described above, according to the above-described first embodiment, it is possible to realize high-accuracy form reading at low cost even in an environment in which various forms are mixed. That is, the cost of creating the form definition data 110a can be reduced by generating the form definition data 110a including only the position information 111a and the attribute information 112a by a simple method. Further, the accuracy of reading is ensured by extracting candidate character strings from the form based on the position information 111b of the form definition data 110b and verifying the character strings based on the attribute information 112b of the form definition data 110b. You can Here, the reading result 116 of the character string image extracted from the form can be determined based on the position information 111b and the attribute information 112b defined in the form definition data 110b, and the form reading can be performed even when the item name is not in the form. Can be realized. Further, when the reading process fails, the form definition data 110a is newly generated, so that it is possible to increase the possibility of successful reading when the same form is input next time.

If the process fails in step S103b, step S104b, step S105b, or step S106b, the new form definition data 110a is stored in the form definition database 103 in step S103c, and then the process proceeds to step S102 and the same form image data 113 is stored. May be processed.

By this operation, by simply creating a new form definition data 110a, the re-reading process is automatically executed for the form that has once failed to be read. Therefore, it is troublesome for the user to manually correct the failed read result. Can be reduced, and it is possible to immediately confirm whether the form can be read by the newly created form definition data 110a. Further, it becomes possible to easily correct the form definition data 110a until the form can be read, so that the quality of the form definition data 110a can be improved.

FIG. 6 is a block diagram showing the configuration of the form processing apparatus according to the second embodiment.
In the form processing apparatus of FIG. 6, a definitionless reading unit 201 is added to the form processing apparatus of FIG. When it is determined that the processing has failed in any of steps S103b, S104b, S105b, and S106b in FIG. 5, the definitionless reading unit 201 operates.

The definitionless reading unit 201 receives the form image data 113 as an input, converts the predetermined read items in the form into character codes, and outputs the read results 116. The definitionless reading unit 201 estimates a read item without using the form definition data 110b by using a keyword such as an item name in a form or matching a character recognition result with a dictionary for each attribute created in advance. To do. The definitionless reading unit 201 can be realized by the method described in Japanese Patent No. 5621169, for example.

The definitionless reading unit 201 may consider that the reading of the form has failed if the progress of the process and the situation in which the result is considered unreliable. The operation when reading of the form fails will be described later with reference to the flowchart of FIG. 7.

For example, when the definitionless reading unit 201 is realized by the method described in Japanese Patent No. 5621169, the definitionless reading unit 201 determines that the evaluation value calculated in step S170 of Japanese Patent No. 5621169 is less than or equal to a predetermined threshold value. If it is, it can be considered that the reliable reading of the form could not be performed and that the reading of the form failed.

FIG. 7 is a flowchart showing the processing of the form processing apparatus according to the second embodiment.
In the processing of FIG. 7, steps S201a and S201b are added to the processing of FIG.

In step S103b, the form definition database 103 proceeds to step S201a when acquisition of form definition data fails in step S103a. In step S104b, when the position verification unit 104 fails in the position verification in step S104a, the process proceeds to step S201a. In step S105b, when the character recognition unit 105 fails in character recognition in step S105a, the process proceeds to step S201a. In step S106b, the attribute matching unit 106 proceeds to step S201a when the attribute matching in step S106a fails.

Next, in step S201a, the definitionless reading unit 201 character-codes the read items of the form image data 113 and obtains the read result 116.

Next, in step S201b, the definitionless reading unit 201 determines whether the reading of the form in step S201a has failed. If the form has been successfully read, the process proceeds to step S107. If the form has not been successfully read, the process proceeds to step S108a.

Next, in step S108a, the form definition creating unit 108 determines whether the form definition data 110a needs to be created. If the form definition data 110a needs to be created, the process proceeds to step S108b, and if the form definition data 110a does not need to be created, the process ends.

The form definition data needs to be created when it is estimated that the form could not be read correctly. Specifically, it is the following case.
(1) When acquisition of the form definition data 110b fails in step S103.
(2) In step S104a, when the position verification fails.
(3) When character recognition fails in step S105a.
(4) In step S106a, when collation of attributes fails.
(5) In step S103a, when the output form definition data 110b is unreliable. That is, when the threshold value TD′ satisfying TD′<TD is set in advance, the relationship of threshold value TD′<distance D<threshold value TD is satisfied.
(6) In step S104, when the output character string image is not reliable. That is, when the threshold value TE′<TE is set in advance and the relationship of threshold value TE′<distance E<threshold value TE is satisfied.
(7) In the case where the reading result 116 stored in the reading result database 107 is corrected in the reading result correction step (not shown).
(8) When the reading of the form has failed in step S201a.

As described above, according to the second embodiment described above, by providing the definitionless reading unit 201, the form is read even if the form definition data 110a corresponding to the form to be processed is not created. It becomes possible. For example, even if the form definition data 110a has not been sufficiently created at the beginning of the operation of the form reading apparatus, the operation of the form reading apparatus can be started. Alternatively, even if a new type of form is input during operation of the form reading device, the form can be read.

Further, step S103b, step S104b, step S105b, step S106b, and step S201b may be omitted, and the operation may be changed so that step S201a is executed after step S106a and before step S107.

At this time, the position matching unit 104 changes the configuration so that information corresponding to the evaluation value of the position matching result is calculated and output together with the character string image 114. The character recognition unit 105 calculates the information corresponding to the evaluation value of the character recognition result, adds the information to the input evaluation value, and outputs the information together with the character code 115. The attribute matching unit 106 calculates the information corresponding to the evaluation value of the attribute matching result, adds the information to the input evaluation value, and outputs the read result 116. The definitionless reading unit 201 calculates the information corresponding to the evaluation value of the definitionless reading and changes the configuration so as to output the information together with the reading result 116. The reading result database 107 compares the evaluation value output from the attribute matching unit 106 with the evaluation value output from the definitionless reading unit 201, adopts the reading result 116 with the higher evaluation value, and reads the reading result database. The configuration is changed so as to be stored in 107.

By this change, it is possible to adopt the reading result which is considered to be more correct, when the reading is performed by utilizing the form definition data 110a and when the reading is performed without using the form definition data 110a.

In the above-described embodiment, the form reading device in the medical field has been described as an example, but it may be applied to various other reading devices. For example, it may be applied to a reading device such as a receipt, medical receipt, check or slip.

FIG. 8 is a block diagram showing the hardware configuration of the form processing apparatus according to the third embodiment.
8, the form processing apparatus 110 includes a processor 11, a communication control device 12, a communication interface 13, a main storage device 14 and an external storage device 15. The processor 11, the communication control device 12, the communication interface 13, the main storage device 14, and the external storage device 15 are connected to each other via an internal bus 16. The main storage device 14 and the external storage device 15 are accessible from the processor 11.

Further, outside the form processing apparatus 110, a pointing device 20, a display 21 and a scanner 22 are provided. The pointing device 20, the display 21, and the scanner 22 are connected to the internal bus 16 via the input/output interface 17.

The processor 11 is hardware that controls the operation of the entire form processing apparatus 110. The main storage device 14 can be composed of, for example, a semiconductor memory such as SRAM or DRAM. The main storage device 14 can store a program being executed by the processor 11 or can be provided with a work area for the processor 11 to execute the program.

The external storage device 15 is a storage device having a large storage capacity, and is, for example, a hard disk device or an SSD (Solid State Drive). The external storage device 15 can hold execution files of various programs and data used for executing the programs. The external storage device 15 can store a form processing program 15A and form definition data 15B. The form processing program 15A may be software that can be installed in the form processing apparatus 110, or may be incorporated in the form processing apparatus 110 as firmware.

The communication control device 12 is hardware having a function of controlling communication with the outside. The communication control device 12 is connected to the network 19 via the communication interface 13. The network 19 may be a WAN (Wide Area Network) such as the Internet, a LAN (Local Area Network) such as WiFi, or a mixture of WAN and LAN.

The input/output interface 17 converts data input from the pointing device 20, the display 21 and the scanner 22 into a data format that the processor 11 can process, and processes data output from the processor 11 by the pointing device 20 and the display 21. Convert to a possible data format.

The processor 11 reads the form processing program 15A into the main storage device 14 and executes the form processing program 15A to generate a character string image that is a candidate for a character string to be read based on the position information included in the form definition data 15B. The reading result 116 is determined by extracting from the form image data, performing character recognition processing of the extracted character string image, and collating the attributes of the character recognition result based on the attribute information included in the simplified form definition data 15B. be able to. At this time, the form processing program 15A can realize the functions of the position matching unit 104, the character recognition unit 105, and the attribute matching unit 106 of FIG. In the collation of the attribute of the character recognition result, the processor 11 may access the database used for the attribute verification processing via the network 19 and acquire the information used for the attribute verification processing.

The execution of the form processing program 15A may be shared by a plurality of processors or computers. Alternatively, the processor 11 may instruct a cloud computer or the like via the network 19 to execute all or part of the form processing program 15A and receive the execution result.

It should be noted that the present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of a certain embodiment. Further, with respect to a part of the configuration of each embodiment, other configurations can be added/deleted/replaced. Further, the above-described respective configurations, functions, processing units, processing means, etc. may be realized by hardware by designing a part or all of them, for example, with an integrated circuit.

101 Scanner, 102 Feature Extraction Unit, 103 Form Definition Database, 104 Position Matching Unit, 105 Character Recognition Unit, 106 Attribute Matching Unit, 107 Read Result Database, 108 Form Definition Data Creation Unit

Claims (11)

A character string image that is a candidate for the read target character string is determined based on the matching result of the position information of the character string image extracted from the form image data and the position information defined for the read target character string. A position matching unit,
A character recognition unit that performs character recognition based on a character string image that is a candidate for the reading target character string;
The reading result of the character string image that is a candidate for the reading target character string is determined based on the matching result between the attribute of the character recognition result by the character recognition unit and the attribute information defined for the reading target character string. Form processing apparatus having an attribute collating unit for performing. The position matching unit, based on the distance between the position of the character string image extracted from the form image data and the position defined for the reading target character string, a character string that is a candidate for the reading target character string. The form processing apparatus according to claim 1, which determines an image. The said position collation part determines the character string image with the shortest distance with the said reading target character string as a candidate for the said reading target character string, when the said character string image exists in multiple in a form image. Form processing device. 2. The form definition data creating unit, further comprising: a form definition data creating unit that creates form definition data in which position information and attribute information of the read target character string is defined based on a position and an attribute of the read target character string. Form processing device. A form definition database that stores form definition data in which position information and attribute information of the reading target character string are defined in association with the feature amount of the form image data,
A feature extraction unit that extracts the feature amount of the form image data,
The form processing apparatus according to claim 4, wherein the form definition database selects the form definition data based on a feature amount of the form image data and outputs the selected form definition data to the position matching unit. In the form definition database, the form definition data corresponding to the second feature amount in which the distance between the first feature amount of the form image data and the second feature amount associated with the form definition data is equal to or less than a threshold value. 6. The form processing apparatus according to claim 5, wherein is selected and output to the position matching unit. When a plurality of the form definition data are selected, the position matching unit determines the position based on the distance between the position of the character string image extracted from the form image data and the position defined for the read target character string. The form processing apparatus according to claim 6, wherein an evaluation value is calculated for each form definition data, and a character string image based on the form definition data having the smallest evaluation value is determined as a candidate for the read target character string. The form processing apparatus according to claim 5, further comprising a definitionless reading unit that determines a reading result of the character string image based on a character recognition result of a character string image including an item name extracted from the form image data. When the position matching unit fails to match the position information,
When the character recognition unit fails in the character recognition,
When the attribute matching unit fails to match the attribute information,
When the form definition database fails to select the form definition data,
The form processing apparatus according to claim 8, wherein the definitionless reading unit executes reading of the form image data. When the position matching unit fails to match the position information,
When the character recognition unit fails in the character recognition,
When the attribute matching unit fails to match the attribute information,
When the form definition database fails to select the form definition data,
When the definitionless reading unit fails to read the form image data,
The form processing apparatus according to claim 8, wherein the form definition data creation unit executes creation of the form definition data. A form processing method executed by a processor, comprising:
The processor is
A character string image that is a candidate for the read target character string is determined based on the matching result of the position information of the character string image extracted from the form image data and the position information defined for the read target character string. ,
Character recognition is performed based on a character string image that is a candidate for the reading target character string,
A form processing method for determining a reading result of a character string image that is a candidate for the reading target character string, based on a matching result between the attribute of the character recognition result and the attribute information defined for the reading target character string. .

Images