JP7149721B2 - Information processing device, character recognition engine optimization method and program - Google Patents

Description

The present disclosure relates to form reading technology.

An information processing apparatus connected to a feature database that accumulates feature data relating to the appearance features of a form for each form type includes an image data acquisition unit that acquires image data of a form to be processed, A feature data acquisition unit that acquires feature data related to a form, and a feature data that matches or is similar to the feature data related to the process target form is specified from the feature data accumulated in the feature database. and a feature data addition unit that adds feature data related to the target form to the feature database when the form type of the target form is not estimated by the estimation unit. A technique has been proposed that makes it possible to estimate the form type without creating a format definition (see Patent Document 1).

JP 2012-198684 A

2. Description of the Related Art Conventionally, there is a technique of generating image data of a form generated as a paper medium using a scanner, a camera, or the like, identifying the type of the form, and performing OCR (Optical Character Recognition) processing.

However, when such technology is actually introduced into business, it takes time and effort to evaluate multiple types of character recognition engines using forms that are actually used in business and select the most suitable character recognition engine. was In addition, even after introduction, it is necessary to make adjustments over time in order to obtain good results of document reading, and the effect of technical introduction cannot be obtained immediately. Furthermore, it is difficult to change a character recognition engine once it has been introduced into business, and the upper limit of form recognition accuracy depends on the performance and features of the character recognition engine that has been introduced.

In view of the above problems, the present disclosure aims to optimize a character recognition engine applied to each form or each item in a form.

An example of the present disclosure is a form data accumulation means for accumulating form data related to a form or an item in the form in association with a character recognition engine suitable for recognizing the form or the item among a plurality of character recognition engines. image data acquisition means for acquiring image data of a form to be processed; a first character recognition means for reading a character string contained in a character string, and among character recognition results obtained by the first character recognition means, a correct character string and a character image including at least the correct character string are associated with each other as correct data. correct data accumulation means for accumulating; second character recognition means for reading a character string included in the character image accumulated by the correct data accumulation means using each of the plurality of character recognition engines; engine changing means for changing the character recognition engine associated with the form type or the form item related to the correct data based on the result of character recognition by the character recognition means.

The present disclosure can be understood as an information processing device, a system, a method executed by a computer, or a program to be executed by a computer. The present disclosure can also be understood as recording such a program in a recording medium readable by a computer, other device, machine, or the like. Here, a computer-readable recording medium is a recording medium that stores information such as data and programs by electrical, magnetic, optical, mechanical or chemical action and can be read by a computer. Say.

According to the present disclosure, it is possible to optimize the character recognition engine applied to each form or each item in the form.

1 is a schematic diagram showing the hardware configuration of a system according to an embodiment; FIG. It is a figure which shows the outline of the functional structure of the information processing apparatus which concerns on embodiment. 4 is a diagram showing the relationship between form data accumulated in a form data bank and form images in the embodiment; FIG. 3 is a diagram showing the configuration of a form type data table in a form data bank used in the embodiment; FIG. 4 is a diagram showing the configuration of a form item data table in the form data bank used in the embodiment; FIG. FIG. 4 is a diagram showing the configuration of a correct data table in a form data bank used in the embodiment; 4 is a diagram showing the configuration of an OCR engine evaluation data (form type) table in the form data bank used in the embodiment; FIG. 4 is a diagram showing the configuration of an OCR engine evaluation data (form item) table in the form data bank used in the embodiment; FIG. 4 is a flowchart A showing an overview of the flow of engine selection processing according to the embodiment; 4 is a flowchart B showing an overview of the flow of engine selection processing according to the embodiment; 4 is a flowchart A showing an overview of the flow of form reading processing according to the embodiment; 10 is a flowchart B showing an overview of the flow of form reading processing according to the embodiment; 4 is a flowchart showing an overview of the flow of data analysis processing according to the embodiment; 4 is a flowchart showing an overview of the flow of new engine evaluation processing according to the embodiment; 4 is a flowchart showing an overview of the flow of engine optimization processing according to the embodiment;

Hereinafter, embodiments of an information processing device, method, and program according to the present disclosure will be described based on the drawings. However, the embodiments described below are examples of embodiments, and do not limit the information processing apparatus, method, and program according to the present disclosure to the specific configurations described below. For implementation, a specific configuration may be appropriately adopted according to the mode of implementation, and various improvements and modifications may be made.

In the description of the embodiments, the information processing apparatus, method, and program according to the present disclosure estimate the form type of the form related to the image data transmitted from the user terminal, and select the optimum OCR engine determined based on the estimation result. An embodiment in the case of being implemented in a form processing system that performs character reading using the method will be described. Note that the information processing apparatus, method, and program according to the present disclosure can be widely used for techniques for reading forms, and the application target of the present disclosure is not limited to the examples shown in the embodiments.

<System configuration>
FIG. 1 is a schematic diagram showing the hardware configuration of a form processing system 1 according to this embodiment. The form processing system 1 according to the present embodiment includes a plurality of user terminals 90 (hereinafter referred to as "nodes 90") installed at a plurality of different bases and used for operations using different types of forms, and a server 50 communicably connected to the node 90 via a network.

In the form processing system 1 according to this embodiment, the server 50 connected from the node 90 is remotely connected via the Internet or a wide area network, and is provided by, for example, an ASP (Application Service Provider). , server 50 need not necessarily be remotely connected. For example, server 50 may be connected to the local network on which node 90 resides.

The server 50 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 13, a ROM (Read Only Memory) 12, an EEPROM (Electrically Erasable and Programmable Read Only Memory), HDD (Hard Disk 1), etc. , a communication unit such as a NIC (Network Interface Card) 15, and the like.

The server 50 according to the present embodiment receives image data transmitted from the node 90 and executes processing described later. The form type of the form related to the image data transmitted from the node 90 is estimated by the server 50 executing a process described later. Then, the server 50 selects the OCR engine most suitable for the form type based on the estimation result (the form type of the form related to the image data), and uses the engine to perform character reading. Specifically, when the server 50 presumes that the form type of the form related to the image data is the "quotation sheet of the AAA company", based on this estimation result, when the server 50 reads the "quotation sheet of the AAA company" Then, the OCR engine with the best performance is selected, and character reading is performed using the engine.

FIG. 2 is a diagram showing an outline of the functional configuration of the server 50 according to this embodiment. In server 50, a program recorded in storage device 14 is read out to RAM 13 and executed by CPU 11 to control each hardware provided in server 50, so that form data storage unit 21, correct data storage unit 22, image data acquisition unit 23, feature data acquisition unit 24, form data search unit 25, first character recognition unit 26, second character recognition unit 27, correction unit 28, correction reception unit 29, format/ It functions as an information processing device including a rule updating unit 30 and an engine changing unit 31 . In this embodiment and other embodiments described later, each function of the server 50 is executed by the CPU 11, which is a general-purpose processor. may be performed.

The form data accumulation unit 21 accumulates form data related to a form or an item in the form in association with an OCR engine suitable for recognizing the form or the item among a plurality of OCR engines (character recognition engines). For each combination of the OCR engine and the form type or the form item, the form data storage unit 21 calculates the results (this implementation In the form, the average recognition rate) is accumulated, and the accumulated results are updated based on the character recognition result by the second character recognition unit 27 .

The correct data storage unit 22 stores a correct character string (for example, a character string obtained when a correction by the user is accepted) and at least the correct character string among the character recognition results obtained by the first character recognition unit 26. The included character images are associated with each other and stored as correct answer data.

In this embodiment, the data accumulated by the form data accumulation unit 21 and the correct data accumulation unit 22 are accumulated in a form data bank, which is a database accessible from the server 50 .

FIG. 3 is a diagram showing the relationship between the form data accumulated in the form data bank and the form image in this embodiment. Here, it is assumed that the form data includes not only data related to the entire form but also data for each item in the form. A form image is an image obtained by digitizing a filled-in form using means such as imaging, and includes information in the form as an image. Note that the form image is not limited to an image obtained by capturing an image of a paper medium using a scanner or camera. The form image may be, for example, an image of a form filled in using a so-called tablet touch panel display or the like.

In this embodiment, by extracting a partial image by image processing, reading characters by an OCR engine, reading metadata, etc., from this form image, a format image, an item image, a mark image, and an entire OCR A result, an item OCR result, a form size, and the like are obtained. Here, the format image is an image related to the format excluding margins and entry information from the form image (an image containing only ruled lines, or an image including ruled lines, background patterns, logos, marks, etc.). It is an image that has been cut out (cropped) from the reading area for each item to be OCRed, and the mark image is an image that is useful for identifying a form, such as a logo, a seal, and a tint block, which is used by the form creator. be. The overall OCR result is a character string obtained by reading the entire form image using OCR, and the item OCR result is a character string obtained by reading the item image using OCR. Also, the form size is the size of an image obtained by excluding margins from the form image, and is the same as the size of the form image.

The form data bank includes a form type data table, a form item data table, a correct answer data table, an OCR engine evaluation data table, and a synonym dictionary. Here, the form type data table is a table that stores information for specifying the form type and the OCR engine used for the relevant form type. 1 is a table that stores OCR engines used for . The correct data table is a table for accumulating combinations of original image data and correct character strings included in the image data. It is a table for accumulating average recognition rates. The synonym dictionary is a dictionary that holds synonyms of form names, item names, and form identification keywords in association with each other.

FIG. 4 is a diagram showing the configuration of the form type data table in the form data bank used in this embodiment. Form type data includes form type ID, form name (character string data), form identification keyword (character string data), format image (image data), emblem image (image data), form size, OCR engine, and others. information is included.

Here, the form type ID is used to identify the form type in the process described later. A form identification keyword is information that is printed on a form and serves as an important key in combination with the form name in determining the type of form, such as the name of the issuing company of the form. A plurality of form identification keywords may be specified for one form type. The OCR engine set in the form type data designates the OCR engine used for reading a new form (process target form) related to the relevant form type. In the form processing system 1 according to the present embodiment, the processing described later is repeatedly executed, and the OCR engine specified for the target form is updated to a more appropriate one, thereby improving the reading accuracy of the target form. go.

FIG. 5 is a diagram showing the configuration of the form item data table in the form data bank used in this embodiment. The form item data includes form type ID, form name, form item ID, item name (character string data), reading area, item format, item rule, OCR engine, and other information.

Here, the form item ID is used to identify the form item in the process described later. The reading area is information specifying an area in the form in which the item is written. For example, if it is a rectangular area in the form, it can be specified using coordinates and distance. In addition, the item format specifies the format (date format, usable character types, attributes of information to be entered, number of digits, etc.) that the description content of the target item should follow. can be defined. The item rule specifies the description rule (value range, relationship with other items, etc.) to be followed by the description content of the target item, and can be defined using a conditional expression or the like. Multiple reading areas, item formats, and item rules may be specified for one form item. If multiple items are specified, one of the multiple reading areas and item formats is applied, and all item rules are applied.

The OCR engine set in the form item data designates the OCR engine used for reading the target item in the new form (processing target form) related to the relevant form type. That is, in the present embodiment, it is possible to execute reading using an OCR engine other than the OCR engine specified in the form type data for each item in the form. For example, it is possible to select an OCR engine that is good at reading handwritten characters for items written by the user, and select an OCR engine that is good at that language for items written in a specific language. is.

FIG. 6 is a diagram showing the configuration of the correct data table in the form data bank used in this embodiment. Correct data includes OCR engine, form type ID, form name, form item ID, item name, reading area, form image (image data), OCR result (character string data), correct character string (character string data), and Contains other information.

Here, the OCR result is the OCR result (regardless of whether it is correct or not) when the item related to the form item ID of the record is read using the OCR engine specified for the record (for this reason, If there is an error in the reading area, the OCR result may be blank). Also, the correct character string is the correct character string entered by the user and described in the item (for this reason, if the OCR result is correct, the correct character string may be blank).

7 and 8 are diagrams showing the configuration of the OCR engine evaluation data table in the form data bank used in this embodiment. In this embodiment, the OCR engine evaluation data includes OCR engine evaluation data (form type) that indicates the evaluation of the OCR engine for each form type, and OCR engine evaluation data (form item) that indicates the evaluation of the OCR engine for each form item. Two types are prepared. The OCR engine evaluation data (form type) includes a form type ID, form name, format image, OCR engine, number of read items, number of evaluated forms, average recognition rate, and other information. The OCR engine evaluation data (document item) includes a document type ID, a document name, a document item ID, an item name, an item image, an OCR engine, the number of evaluation items, an average recognition rate, and other information.

Here, the number of read items is the number of items to be read using the OCR engine, included in the target form. Also, the average recognition rate is the average value of the accuracy rate (recognition rate) of the OCR result by the target OCR engine for the form type or form item related to the OCR engine evaluation data. The average recognition rate is calculated and updated using, for example, the formula shown below.
Total number of read items = Number of evaluation items x Number of evaluation sheets + Number of items read this time Total number of correct answers = Number of evaluation items x Number of evaluation sheets x Average recognition rate + Number of correct answers this time ×100

The image data acquisition unit 23 acquires image data of a form to be processed that is captured by a node 90 such as a mobile camera or a scanner and transmitted to the server 50 .

The characteristic data acquisition unit 24 acquires a plurality of types of characteristic data related to the processing target form from the image data of the processing target form. Specifically, the characteristic data acquiring unit 24 obtains, as characteristic data of the processing target form, a character string included in the image data of the processing target form, a format image obtained by excluding character images from the image data of the processing target form, a logo image, and The form size and the like are acquired using a predetermined OCR engine and image processing function.

The form data search unit 25 searches the form data accumulated by the form data accumulation unit 21 using the feature data, thereby retrieving form data similar to the form to be processed. For example, the form data search unit 25 searches the form data accumulated by the form data accumulation unit 21 using the character string, the format image, the logo image, the form size, or the like acquired by the characteristic data acquisition unit 24. , to retrieve form data similar to the form to be processed. At this time, the form data search unit 25 searches form data using a plurality of types of feature data or combinations of feature data to obtain search results for each feature data or combination of feature data. are evaluated in a predetermined order, and the retrieval result that satisfies the predetermined conditions first is taken as form data similar to the form to be processed.

The first character recognition unit 26 performs OCR associated with the form data related to the form type or form item corresponding to the image data of the form to be processed, which is retrieved by the form data search unit 25 among the plurality of OCR engines. Using the engine, the character string included in the image data of the form to be processed is read.

The second character recognition section 27 uses each of the plurality of OCR engines to read character strings included in the character images accumulated by the correct data accumulation section 22 . Further, when a new OCR engine that can be used in the information processing apparatus is added, the second character recognition unit 27 uses the added OCR engine to recognize the character images accumulated by the correct data accumulation unit 22. Read the string contained in .

The correction unit 28 corrects the character recognition result by the first character recognition unit 26 so as to conform to the format or description rule.

A correction accepting unit 29 accepts a user's correction of the character recognition result by the first character recognizing unit 26 .

The format/rule updating unit 30 extracts the correct character string for the same item from the correct character strings obtained by correction by the user and accumulated by the correct data accumulation unit 22, and extracts the correct character string for the same item from the correct character string. , the format or description rule of the item is updated so that the correct character string obtained by modification conforms to the format/rule of the item.

The engine change unit 31 changes the OCR engine associated with the form type or form item to the OCR engine with the highest result for the form type or form item based on the results accumulated by the form data accumulation unit 21. .

<Process flow>
Next, the flow of processing executed by the server 50 according to this embodiment will be described. It should be noted that the specific content and processing order of the processing described below are examples for carrying out the present disclosure. Specific processing contents and processing order may be appropriately selected according to the embodiment of the present disclosure.

9 and 10 are flowcharts showing an overview of the flow of engine selection processing according to this embodiment. The processing shown in this flowchart is executed when the server 50 receives a new form (hereinafter referred to as a "process target form").

In step S101, a form image is obtained. The image data acquisition unit 23 acquires image data (form image) of a form to be processed which is captured by the node 90 such as a mobile camera or a scanner and transmitted to the server 50 . After that, the process proceeds to step S102.

In steps S102 to S106, feature data is acquired from the form image. The feature data acquisition unit 24 acquires feature data related to the form to be processed from the image data (form image). Specifically, the feature data acquisition unit 24 uses a predetermined OCR engine (for example, one of a plurality of OCR engines available from the form processing system 1) to extract all OCR-readable data contained in the form image. character as feature data (overall OCR result) (step S102). The overall OCR result obtained here includes the form name and form identification keyword (company name related to the form, etc.).

Furthermore, the feature data acquisition unit 24 acquires the form image obtained by removing the character image from the form image as feature data (step S103). In this embodiment, the feature data acquisition unit 24 identifies one or more areas recognized as characters by the OCR engine as a result of character recognition in step S102, and deletes the image of the area from the form image. to get the format image. In addition, various methods can be adopted as specific methods for extracting ruled lines. For example, from the image data of the form, the part related to the ruled line and the part other than the ruled line (characters, symbols, photographs, etc.) may be distinguished by image analysis, and image data of only the ruled line may be generated.

Further, the feature data acquisition unit 24 acquires mark images such as logos, seals, and background patterns from the format image as feature data (step S104). Specifically, the mark image is obtained by removing the frame line and text images from the form image, and then using image analysis to identify and cut out images such as logos, seals, and background patterns from the remaining image. to get At this time, a machine learning model constructed using learning data such as logos, seals, and background patterns may be used to extract the mark image.

Furthermore, the feature data acquisition unit 24 acquires the form size and other parameters from the format image as feature data (steps S105 and S106). Other parameters are, for example, the language used in the form, the number of pages in the form, and the like. These parameters can be obtained from the overall OCR results or metadata contained in the form image. After that, the process proceeds to step S107.

In step S107, the form data bank is searched using the feature data. The form data search unit 25 searches the form type data in the form data bank for each type of characteristic data acquired in the processing from step S102 to step S106. Here, the form data search unit 25 searches for form data using the feature data in descending order of priority among the plurality of types of feature data. A search result is obtained for each feature data, and the search result that first satisfies a predetermined condition is taken as form data similar to the form to be processed. An example of priority of feature data and search processing in the present embodiment is as follows in descending order of priority. Note that the priority of (search 3) and (search 4) may be reversed.

(Search 1) Form name and form identification keyword: The form data search unit 25 uses the form name and form identification keyword included in the form type data to search the entire OCR result obtained in step S102, thereby Form type data whose name and form identification keyword are included in the overall OCR result of the process target form are retrieved. A synonym dictionary may also be used for searching.

(Search 2) Form image: The form data search unit 25 calculates the degree of similarity between the form image acquired in step S103 and the form image included in the form type data. Search for the form type data related to.

(Search 3) Mark image: The form data search unit 25 calculates the similarity between the mark image acquired in step S104 and the mark image included in the form type data. form type data related to the mark image exceeding the

(Search 4) Form size: The form data search unit 25 compares the form size acquired in step S105 with the form size included in the form type data, and determines the form type associated with the form size whose difference is within the range of the threshold value. Search for data.

(Search 5) Other parameters such as language and number of pages: The form data search unit 25 compares the other parameters acquired in step S106 with the other parameters included in the form type data to determine whether they match or have similarity. Document type data related to other parameters exceeding the threshold or whose difference is within the range of the threshold are retrieved.

In steps S108 to S111, the most similar data is narrowed down from the retrieved form type data. The form data search unit 25 determines the number of search results in step S107 for each search condition or combination of search conditions.

Specifically, first, the form data search unit 25 refers to the number of forms type data obtained as a result of "search 1" (step S108), and if the number is one, the process proceeds to step S114. . If the number of form type data obtained as a result of "search 1" is not 1 (zero or more), the form data search unit 25 determines the number of form type data obtained as a result of "search 2" (step S109), and if there is one case, the process proceeds to step S114. If the number of document type data obtained as a result of "search 2" is 0, the process proceeds to step S112. If the number of items of form type data obtained as a result of "search 2" is plural, the form data search unit 25 is included in both the result of "search 1" and the result of "search 2" (i.e., "Search 1" and "Search 2") The number of document type data is referred to (step S110), and if the number is one, the process proceeds to step S114. On the other hand, if the number of form type data included in both the results of "search 1" and the result of "search 2" (i.e., "search 1" and "search 2") is 0 or more, the form The data search unit 25 further narrows down the search results by adding the conditions of "search 3" to "search 5" in descending order of priority (step S111). proceed.

In this way, by narrowing down the form type data similar to the form to be processed according to the priority of the feature data used for retrieval, the form type of the form to be processed can be determined using more accurate conditions. I can.

On the other hand, in step S111, if the search result cannot be narrowed down to one even if all the conditions of "search 3" to "search 5" are added, the narrowed down result so far is discarded, and the process proceeds to step S112. . However, the result of narrowing down in step S111 may be added to the search result in the process of next step S112.

In step S112, an appropriate OCR engine is selected based on the form name, item name, item image and form image. The first character recognition unit 26 searches the OCR engine evaluation data (form type) table in the form data bank using the form name and the format image, and averages the OCR engine evaluation data (form type) with a high degree of similarity. Of the OCR engines with accumulated recognition rates, the OCR engine with the highest average recognition rate is determined as the OCR engine for processing the form to be processed.

Also, the first character recognition unit 26 searches the OCR engine evaluation data (form item) table in the form data bank using the form name, item name, and item image, and recognizes the OCR engine evaluation data ( Out of the OCR engines for which the average recognition rate is accumulated for the form item), the OCR engine with the highest average recognition rate is determined as the OCR engine for processing the item of the target form. If the OCR engine for processing cannot be determined even after searching the OCR engine evaluation data, the first character recognition unit 26 presents the search result to the user, and the OCR engine for processing the form to be processed. may be selected by the user. After that, the process proceeds to step S113.

In step S113, the form type data and the form item data are added. The form data accumulating unit 21 generates form type data and form item data corresponding to the form to be processed, associates them with the OCR engine determined in step S112, and adds them to the form data bank. The form name of the form type data and form item data to be added is determined according to the selected OCR engine evaluation data (form type), and the form identification keyword is selected from the overall OCR result. For other items included in the form type data and the form item data, the data acquired in the processes from step S102 to step S106 are used. Note that the form data storage unit 21 may estimate, among the overall OCR results, character strings with large fonts and character strings that are in predetermined positions or areas where keywords are frequently described as form identification keywords. I can. Items that are automatically set in the form type data and the form item data can be updated/corrected in a process described later. After that, the process proceeds to step S114.

In step S114, the OCR engine for processing the form to be processed is determined. The first character recognition unit 26 recognizes the OCR engine associated with the form type data retrieved in the processes from step S107 to step S111 or the OCR engine determined or selected in step S112 to process the form to be processed. Select the OCR engine for After that, the processing shown in this flowchart ends.

11 and 12 are flowcharts showing an outline of the flow of the form reading process according to this embodiment. The processing shown in this flowchart is executed following the engine selection processing described above.

In step S201, each item defined in the form item data is read. The first character recognition unit 26 recognizes the characters of each item defined in the form item data using the OCR engine determined as the processing OCR engine for the form to be processed in step S114 of the engine selection process described above. Read from the reading area set in the form item data. At this time, if an engine different from the processing OCR engine for the processing target form determined in step S114 is specified in the form item data as the OCR engine for the target item, the specified OCR engine is used to The characters in the item are read. After that, the process proceeds to step S202.

In steps S202 and S203, the OCR results are corrected. The correction unit 28 corrects the characters read in step S201 according to the item format defined in the form item data (step S202). For example, when the item format set in the form item data of the item is "date", the correction unit 28 checks whether the read character string conforms to the format of the date. automatically corrects it if it does not follow the format of Further, the correction unit 28 corrects the characters read in step S201 according to the item rule defined in the form item data (step S203). If the rule is "Future than issue date", check whether the read character string indicates a future date than the date read from the item "Issuance date" in the same form. Automatically correct this if it doesn't follow the rules. After that, the process proceeds to step S204.

In step S204, it is determined whether or not all the items in the form to be processed have been read. The first character recognition unit 26 confirms whether or not the processing from step S201 to step S203 has been completed for all items related to the relevant form type defined in the form item data. If processing for all items has not been completed, the process returns to step S201. On the other hand, if the processing for all items has been completed, the processing proceeds to step S205.

In steps S205 to S208, the OCR results are checked by the user and corrected if necessary. The correction accepting unit 29 stores the form image acquired in step S101, the item image of the reading area specified when the item is read in step S201, and the item OCR result acquired in the processing from step S201 to step S204 to the node. It is output so that the user can check it by a method such as displaying it on the screen of 90 (step S205). Then, the correction accepting unit 29 accepts the confirmation result input by the user who confirmed the output OCR result (step S206). Here, the user can use the node 90 or the like to input whether or not the OCR result is correct for each item of the OCR result. If the OCR result is correct, the process proceeds to step S212. On the other hand, if there is an item with an incorrect OCR result (an erroneous OCR recognition), the correction accepting unit 29 accepts input of a correct character string and/or correct reading area from the node 90 (step S208). After that, the process proceeds to step S209.

In steps S209 to S211, the form data bank is updated based on the content of the correction made by the user. If the item corrected in step S208 is the form identification keyword, the form data accumulation unit 21 adds the character string obtained by misrecognition to the form identification keyword of the form type data (step S209). By doing so, even if the OCR engine makes similar erroneous recognition, it is possible to correctly recognize the form type.

In addition, the correct data storage unit 22 adds the correct character string input by the user as correct data (step S210). Specifically, when the reading area is corrected in step S208, the correct data storage unit 22 generates correct reading area input by the user (re-designated reading area) and correct data including the form image, and Add to databank. Further, when the character string of the item OCR result is corrected in step S208, the correct data storage unit 22 stores the form image, the OCR result of the incorrectly recognized character string before correction, and the correct character string input by the user. Generates correct data containing In this embodiment, the correct character string input by the user at the time of correction by the user is registered in the correct answer data table. It may be accumulated in the correct answer data table.

Furthermore, if the item corrected in step S208 is a form name or an item, the form data storage unit 21 stores the incorrectly recognized character string (for example, "word name") before correction as entered by the user. The correct character string (for example, "product name") is registered as a synonym in the synonym dictionary (step S211). After that, the process proceeds to step S212.

At step S212, the OCR engine evaluation data is updated. The form data accumulation unit 21 stores OCR engine evaluation data (form items) and OCR engine evaluation data (form type) based on the number of items read in step S201 and the evaluation of the OCR results obtained in step S206. to update.

Specifically, the form data storage unit 21 stores the number of items newly read in S201 (10 if one form containing 10 items is read) as the corresponding form type, form item, and OCR engine. Add to the number of evaluation items in the OCR engine evaluation data (form item) related to the combination, and the average recognition rate of the OCR engine evaluation data (form item) based on the correct/wrong OCR result of the newly read item to update. For example, if the number of correct items is 18,000 when 20,000 items are read, the average recognition rate is 90%.

In addition, the form data accumulation unit 21 calculates the number of newly read forms (for example, 1) in S201 as the number of evaluated forms in the OCR engine evaluation data (form type) associated with the combination of the corresponding form type and OCR engine. and the average recognition of the OCR engine evaluation data (form type) based on the accuracy rate of the OCR result of the newly read form (for example, 90% when 9 out of 10 items are correct) Update rate. For example, if the number of correct items is 180,000 when 20,000 forms (that is, 200,000 items) of 10 read items are read, the average recognition rate is 90%.

The OCR engine evaluation data (form item) and the average recognition rate of the OCR engine evaluation data (form type) updated in the process of step S212 are referred to in the engine optimization process, which will be described later. Contributes to optimization of the OCR engine used. After that, the processing shown in this flowchart ends.

FIG. 13 is a flowchart showing an overview of the flow of data analysis processing according to this embodiment. The processing shown in this flowchart is automatically executed when it is detected that the OCR result has been corrected as described in steps S208 to S211. However, the timing at which the processing shown in this flowchart is executed does not have to be immediately after the correction of the OCR result. For example, the processing shown in this flowchart may be executed at a timing specified by an administrator, or may be executed periodically.

In step S301, correct data having a common item name are extracted. The format/rule update unit 30 extracts correct data for the same form type and the same item from the form data bank. After that, the process proceeds to step S302.

In steps S302 to S305, the correct answer data is analyzed, and the item format and item rules in the form item data are updated. The format/rule updating unit 30 analyzes the correct character string group in the correct data extracted in step S301, adds the item format and item rule of the form item data according to the analysis result, and improves the reading accuracy by OCR. . Specifically, the format/rule updating unit 30 determines whether or not the correct character string group in the correct answer data extracted in step S301 satisfies the item format of the form item data (step S302). If so, a new item format is registered in the form item data (step S303). For example, if the correct character string extracted from the correct answer data is a Japanese calendar date, but the item format set in the form item data is only a Christian calendar date, the format/rule updating unit 30 updates the Japanese calendar date. Add to item format. Also, if the correct character string extracted from the correct data includes alphabetic characters, but the item format set in the form item data is only numbers, the format/rule updating unit 30 adds the alphabetic characters to the item format. do. Also, the format/rule updating unit 30 can add a new number system or add a regular expression pattern according to the content of the correct character string group extracted from the correct answer data.

The format/rule updating unit 30 also determines whether the correct character strings in the correct answer data extracted in step S301 satisfy the item rule of the form item data (step S304). If so, a new item rule is registered in the form item data (step S305). For example, if the correct character string extracted from the correct answer data does not fall within the numerical upper limit, lower limit, and number of digits set in the item rule, the format/rule updating unit 30 updates the item so that the correct character string is included. change the rules. Further, the format/rule update unit 30 does not include characters such as "\" and "," in the correct character string extracted from the correct answer data, but if these characters are set in the item rule, these characters are not included. Exclude characters from item rules. Also, if the correct character string extracted from the correct answer data is a company name but is a new company name not registered in the item rule (company name dictionary), the format/rule updating unit 30 updates the item rule (company name dictionary) with Register the company name. After that, the processing shown in this flowchart ends.

FIG. 14 is a flowchart showing an overview of the flow of new engine evaluation processing according to this embodiment. The processing shown in this flowchart is executed when an OCR engine available from the server 50 is added to the form processing system 1 .

In steps S401 to S404, a new OCR engine is evaluated using correct data. The second character recognition unit 27 uses a new OCR engine to read item images in the correct data stored in the form data bank (step S401). Then, the form data storage unit 21 compares the OCR result by the new OCR engine with the correct character string in the correct data (step S402), and it is determined that they match (the OCR result by the new OCR engine is correct). If there is, the OCR engine evaluation data (form item) for the new OCR engine is added (step S403). Specifically, the form data accumulation unit 21 sets the identification information (name, etc.) of the new OCR engine to the OCR engine in the OCR engine evaluation data (form item), and sets the form type and form item to the evaluation Set the form type and the form item acquired from the correct data used in , set the number of items newly read in S401 to the number of evaluation items, and set the number of newly read items to the average recognition rate. A value calculated based on the correct/wrong OCR result is set.

On the other hand, as a result of the comparison, if it is determined that the OCR result by the new OCR engine and the correct character string in the correct data do not match (the OCR result by the new OCR engine is erroneous), the correct data storage unit 22 registers new correct data based on the OCR result (error) and the correct character string (step S404). Specifically, the correct data storage unit 22 generates OCR results, which are character strings erroneously recognized by the new OCR engine, and correct data including correct character strings, and adds them to the form data bank. Here, the correct character string can be obtained from the correct data that is the object of comparison. After that, the process proceeds to step S405.

In step S405, it is determined whether or not the evaluation of a predetermined amount of correct data has been completed. The second character recognition unit 27 confirms whether or not the processing from step S401 to step S404 has been completed for a predetermined amount (for example, all) of correct data accumulated in the form data bank. If the predetermined amount of correct data has not been processed, the process returns to step S401. On the other hand, if the processing for the predetermined amount of correct data has been completed, the process proceeds to step S406.

In step S406, OCR engine evaluation data (form type) is added. Specifically, the form data accumulation unit 21 sets the identification information (name, etc.) of the new OCR engine to the OCR engine in the OCR engine evaluation data (type of form), and sets the type of form to be used for evaluation. The form type acquired from the correct data obtained from the correct data is set, the number of evaluation items is set to the number of forms newly read in S401, and the average recognition rate is set to indicate the correct/wrong OCR result of the newly read form. Set a value calculated based on After that, the processing shown in this flowchart ends.

In the new engine evaluation process shown in this flowchart, the average recognition rate of the OCR engine evaluation data (form item) and the OCR engine evaluation data (form type) updated in the processes of steps S403 and S406 is the engine optimum By being referenced in the conversion process, it contributes to the optimization of the OCR engine used for the form type or form item.

FIG. 15 is a flowchart showing an overview of the flow of engine optimization processing according to this embodiment. The processing shown in this flowchart is executed periodically (for example, once a day) or at timing designated by the administrator.

In steps S501 and S502, each existing OCR engine is evaluated using correct data. The second character recognition unit 27 reads the item images in the correct answer data accumulated in the form data bank using all OCR engines available from the server 50 (step S501). However, in consideration of the processing load of the form processing system 1, etc., an OCR engine that is used infrequently may be excluded from the target. Then, the form data storage unit 21 compares the OCR result of each OCR engine with the correct character string in the correct answer data, and the results match (the OCR result by each OCR engine is correct) or do not match (each The OCR result by the OCR engine is erroneous) is recorded for each OCR result (step S502). After that, the process proceeds to step S503.

In step S503, it is determined whether or not the evaluation of a predetermined amount of correct data has been completed. The second character recognition unit 27 confirms whether or not the processing of steps S501 and S502 has been completed for a predetermined amount (for example, all) of correct data accumulated in the form data bank. If the predetermined amount of correct data has not been processed, the process returns to step S501. On the other hand, if the processing for the predetermined amount of correct data has been completed, the process proceeds to step S504.

In steps S504 and S505, the average recognition rate of the OCR engine evaluation data (form item) and the average recognition rate of the OCR engine evaluation data (form type) are recalculated and updated. Since the specific update method is substantially the same as the content described in step S212, the description is omitted. After that, the process proceeds to step S506.

In step S506, the optimum OCR engine for the form type is updated. The engine change unit 31 acquires the average recognition rate for each OCR engine for the same form type from the OCR engine evaluation data (form type) accumulated in the form data bank, and compares them to read the form type. determines the OCR engine with the highest average recognition rate (optimal OCR engine), and if the determined optimal OCR engine is different from the OCR engine currently set in the document type data for the document type, Update the OCR engine set for the data to the optimal OCR engine. However, if the correct data evaluated in the processing from step S501 to step S503 is biased, there is a possibility that a high average recognition rate can be obtained by chance. Therefore, OCR engines that are candidates for the optimum OCR engine may be limited to OCR engines that have reached the number of statistically reliable evaluation forms or evaluation items. After that, the process proceeds to step S507.

In step S507, the optimal OCR engine for the form item is updated. The engine change unit 31 acquires and compares the average recognition rate of each OCR engine for the same form item from the OCR engine evaluation data (form item) accumulated in the form data bank, and reads the form item. determines the OCR engine with the highest average recognition rate (optimal OCR engine), and if the determined optimal OCR engine is different from the OCR engine currently set in the form item data for the relevant form item, Update the OCR engine set for the data to the optimal OCR engine. As in step S506, OCR engines that are candidates for the optimum OCR engine may be limited to OCR engines that have reached the number of statistically reliable evaluation forms or evaluation items. After that, the processing shown in this flowchart ends.

<effect>
According to the above-described embodiment, similar form data is retrieved using the characteristics of the form to be processed, and the character recognition engine associated with the retrieved form data among a plurality of character recognition engines is used. By processing the form to be processed, it is possible to select the optimum character recognition engine for each form or item to be processed. In addition, by performing character recognition for the same form/item using each of a plurality of character recognition engines, and changing the character recognition engine applied to the form/item based on the result of character recognition, Alternatively, it is possible to optimize the character recognition engine applied to each item in the form.

1 form processing system 50 servers 90 nodes

Claims (6)

a form data accumulation means for accumulating form data related to the form type of the form or the form item in the form in association with a character recognition engine suitable for recognizing the form type or the form item among a plurality of character recognition engines; ,
image data acquisition means for acquiring image data of a form to be processed;
feature data acquisition means for acquiring feature data relating to the form to be processed from the image data;
By searching the form data accumulated by the form data accumulation means using the characteristic data, a form related to the form type or the form item corresponding to the image data is retrieved from the form data accumulated by the form data accumulation means. form data search means for searching data;
first character recognition means for reading a character string included in the image data using a character recognition engine associated with the form data retrieved by the form data search means , among the plurality of character recognition engines;
correction receiving means for receiving a user's correction of the result of character recognition by the first character recognition means;
correct data storage means for associating a correct character string obtained by modification by the user among character recognition results by the first character recognition means with a character image including at least the correct character string, and accumulating the result as correct answer data;
second character recognition means for reading a character string included in the character image accumulated by the correct data accumulation means using each of the plurality of character recognition engines;
determining that the character recognition result is correct when there is no correction by the user, and using the character recognition engine for each combination of the character recognition engine and the form type or the form item accumulating an average recognition rate which is a ratio of the total number of correct answers to the total number of read items when character strings are read from image data; and a recognition rate management means for updating the accumulated average recognition rate by updating the total number of correct answers;
Based on the average recognition rate accumulated by the recognition rate managing means, the character recognition engine associated with the form data accumulated in the form data accumulating means is operated as the form type or the form item related to the form data. an engine changing means for changing to a character recognition engine with the highest average recognition rate for
Information processing device. The form data includes a format to be followed by the description contents of the items in the form or a description rule to be followed by the description contents of the items,
Further comprising correction means for correcting the character recognition result by the first character recognition means so as to conform to the format or description rule,
The information processing device according to claim 1 . A correct character string for the same item is extracted from the correct character strings corrected by the user and accumulated by the correct data storage means, and if the correct character string does not conform to the format or description rule, , further comprising format/rule update means for updating the format or description rule of the corresponding item so that the correct character string obtained by the correction conforms to the format/rule of the corresponding item;
The information processing apparatus according to claim 2 . When a new character recognition engine that can be used in the information processing apparatus is added, the second character recognition means uses the added character recognition engine to recognize the characters accumulated by the correct data accumulation means. read the string contained in the image,
The form data storage means compares the character recognition result of the character image by the new character recognition engine with the correct character string in the correct data, so that the character recognition result by the new character recognition engine is: Determine whether or not the answer is correct, and read a character string from the image data of the form type or form item using the new character recognition engine for the combination of the new character recognition engine and the form type or form item. accumulating the average recognition rate, which is the ratio of the total number of correct answers to the total number of read items,
The engine changing means changes the character recognition engine associated with the form data accumulated in the form data accumulation means based on the average recognition rate accumulated by the recognition rate management means. change to a character recognition engine with the highest average recognition rate for the form type or the form item ;
The information processing apparatus according to any one of claims 1 to 3 . the computer
a form data accumulation step of accumulating form data related to the form type of the form or the form item in the form in association with a character recognition engine suitable for recognizing the form type or the form item among a plurality of character recognition engines; ,
an image data acquisition step for acquiring image data of a form to be processed;
a feature data acquiring step of acquiring feature data relating to the form to be processed from the image data;
By searching the form data accumulated in the form data accumulation step using the characteristic data, a form related to the form type or the form item corresponding to the image data is retrieved from the form data accumulated in the form data accumulation step. a form data search step for retrieving data;
a first character recognition step of reading a character string included in the image data using a character recognition engine associated with the form data retrieved in the form data search step, among the plurality of character recognition engines;
a correction receiving step for receiving a user's correction of the character recognition result in the first character recognition step;
a correct data storage step of associating a correct character string obtained by modification by a user among the character recognition results in the first character recognition step with a character image including at least the correct character string, and accumulating the result as correct data; ,
a second character recognition step of reading a character string included in the character image accumulated in the correct data accumulation step using each of the plurality of character recognition engines;
determining that the character recognition result is correct when there is no correction by the user, and using the character recognition engine for each combination of the character recognition engine and the form type or the form item accumulating an average recognition rate that is a ratio of the total number of correct answers to the total number of read items when character strings are read from image data; and a recognition rate management step of updating the accumulated average recognition rate by updating the total number of correct answers;
Based on the average recognition rate accumulated in the recognition rate management step, the character recognition engine associated with the form data accumulated in the form data accumulation step is operated for the form type or the form item related to the form data. an engine change step of changing to a character recognition engine with the highest average recognition rate ;
Character recognition engine optimization method to perform. the computer,
a form data accumulation means for accumulating form data related to the form type of the form or the form item in the form in association with a character recognition engine suitable for recognizing the form type or the form item among a plurality of character recognition engines; ,
image data acquisition means for acquiring image data of a form to be processed;
feature data acquisition means for acquiring feature data relating to the form to be processed from the image data;
By searching the form data accumulated by the form data accumulation means using the characteristic data, a form related to the form type or the form item corresponding to the image data is retrieved from the form data accumulated by the form data accumulation means. form data search means for searching data;
first character recognition means for reading a character string included in the image data using a character recognition engine associated with the form data retrieved by the form data search means , among the plurality of character recognition engines;
correction receiving means for receiving a user's correction of the result of character recognition by the first character recognition means;
correct data storage means for associating a correct character string obtained by modification by the user among character recognition results by the first character recognition means with a character image including at least the correct character string, and accumulating the result as correct answer data;
second character recognition means for reading a character string included in the character image accumulated by the correct data accumulation means using each of the plurality of character recognition engines;
determining that the character recognition result is correct when there is no correction by the user, and using the character recognition engine for each combination of the character recognition engine and the form type or the form item accumulating an average recognition rate which is a ratio of the total number of correct answers to the total number of read items when character strings are read from image data; and a recognition rate management means for updating the accumulated average recognition rate by updating the total number of correct answers;
Based on the average recognition rate accumulated by the recognition rate managing means, the character recognition engine associated with the form data accumulated in the form data accumulating means is operated as the form type or the form item related to the form data. an engine changing means for changing to a character recognition engine with the highest average recognition rate for
A program to function as

Images