JP2021005173A - Ocr recognition result confirmation support program, ocr recognition result confirmation support method, and ocr recognition result confirmation support system - Google Patents

Abstract

To make it possible to output a list for facilitating an operation to compare handwritten character image data with its character recognition result.SOLUTION: An OCR recognition result confirmation support device 100 acquires character string image data handwritten on a form, and outputs a list L2 arranged for confirmation image data L21 in a unit of the form in parallel with a character recognition result L22 in which the confirmation image data L21 has been subjected to the character recognition. The confirmation image data L21 of the list L2 is enlarged or reduced in size of each character included in the character string image data of the form, so that the size of each character in the character string is approximately equal, the image data of each character is then concatenated, and thereby a user can easily compare the confirmation image data L21 with the character recognition result L22 and can appropriately determine the correctness of the character recognition result.SELECTED DRAWING: Figure 6A

Description

The present invention relates to an OCR recognition result confirmation support program that supports confirmation of character recognition (OCR: Optical Character Recognition) results, an OCR recognition result confirmation support method, and an OCR recognition result confirmation support system.

Handwritten characters can be converted into data by using OCR. For example, by recognizing the handwritten characters entered in the form, the contents of the form can be converted into text data or the like, and a database of form data can be constructed.

In character recognition by OCR, since the shape and size of handwritten characters are not uniform, character recognition may be erroneous. Therefore, the user confirms the correctness of the character recognition result by comparing the handwritten character image data (for example, image data) with the OCR result data (for example, text data).

As a technology related to confirmation of character recognition results, an operator that clusters OCR recognition results for each pattern such as color and part of speech, and displays and confirms the recognition results of multiple image patterns that have been recognized as the same character pattern side by side. The technology for selecting is disclosed. Further, there is disclosed a technique of clustering similar character images from character images having the same recognition result and displaying the character images determined to be the same cluster side by side. Further, a technique is disclosed in which an image of a character string of a recognition source is decomposed for each character, and images of characters before and after the specified recognition result of one character are concatenated and sorted to be displayed in a list. Further, a technique for displaying the area recognition result of a figure, a ruled line, an image, a table, etc. in a document image and the character recognition result in the area are disclosed.

JP-A-2017-111500 Japanese Unexamined Patent Publication No. 2005-309608 JP-A-2015-55891 Japanese Unexamined Patent Publication No. 11-149520

In the prior art, the confirmation work of the character recognition result obtained by OCR could not be made efficient. For example, if the handwritten characters to be confirmed on the confirmation list consist of multiple character strings and the size of each character is different, simply arranging the images of the same character string will determine the position of each character to be compared. Unlike, it cannot be easily confirmed.

Further, when the size of handwritten characters differs for each character due to the user's habit, even if the images (character shapes) of the same characters are lined up at the same position, the character sizes are different, so that the characters may be mistaken for different character shapes. As described above, in the case where the handwritten character is a character string composed of a plurality of characters, in the prior art, it has not been possible to easily confirm the correctness of the character recognition result at a glance during the confirmation work by the user.

In one aspect, it is an object of the present invention to be able to output a list that facilitates the task of comparing image data of handwritten characters with character recognition results.

According to one aspect of the present invention, image data is acquired, and the size of each character is enlarged or reduced so that the size of each character in the character string included in the image data is substantially equal, and the size of each character is enlarged or reduced. The character recognition result obtained as a result of character recognition of the image data or the confirmation image data by concatenating the characters to generate confirmation image data of the character string can be compared with the confirmation image data. It is a requirement to output a list in display format and perform processing.

According to one aspect of the present invention, it is possible to output a list that facilitates the work of comparing the image data of handwritten characters with the character recognition result.

FIG. 1 is an explanatory diagram showing an embodiment of OCR recognition result confirmation support according to the embodiment. FIG. 2 is a diagram showing a hardware configuration example of the OCR recognition result confirmation support device according to the embodiment. FIG. 3 is a diagram illustrating a conversion process of image data for OCR recognition according to the embodiment. FIG. 4 is a diagram showing an example of conversion processing of image data for OCR recognition according to the embodiment. FIG. 5 is a flowchart showing a processing example of OCR recognition result confirmation support according to the embodiment. FIG. 6A is a diagram showing an example of an OCR result confirmation list output by the OCR recognition result confirmation support device according to the embodiment. (Part 1) FIG. 6B is a diagram showing an example of an OCR result confirmation list output by the OCR recognition result confirmation support device according to the embodiment. (Part 2) FIG. 7 is a diagram showing a list of character recognition results corresponding to the prior art.

(Embodiment)
Hereinafter, embodiments of the disclosed OCR recognition result confirmation support program, OCR recognition result confirmation support method, and OCR recognition result confirmation support system will be described in detail with reference to the drawings.

FIG. 1 is an explanatory diagram showing an embodiment of OCR recognition result confirmation support according to the embodiment. The OCR recognition result confirmation support device 100 recognizes characters handwritten on a form or the like, and supports the confirmation work of the character recognition result performed by the user. In OCR, an error may occur in the character recognition result of the handwritten character, and the OCR recognition result confirmation support device 100 of the embodiment easily confirms the correctness of the character recognition state to the user who confirms the OCR recognition result. Present in a possible display format.

The OCR recognition result confirmation support device 100 can be applied to, for example, a business of aggregating the contents of a large number of forms written in handwritten characters and digitizing them (digitizing them into electronic data). The OCR recognition result confirmation support device 100 can be configured by using a computer device such as a PC or a server having a CPU, a storage unit 101, or the like, and the OCR recognition result confirmation support program is stored in the storage unit 101 or the like. A predetermined OCR recognition process is executed by the CPU executing a program of an OCR recognition result confirmation support program stored in a storage unit such as the storage unit 101.

The OCR recognition result confirmation support system includes the above-mentioned OCR recognition result confirmation support device 100, an output unit 103 connected to the OCR recognition result confirmation support device 100, an operation unit 104, a database (DB) 105, and a scanner 110. And, including. The output unit 103 is, for example, the display shown in FIG. 1, a printer, or the like. The operation unit 104 is, for example, the keyboard shown in FIG. The DB 105 may be configured to be arranged independently of the OCR recognition result confirmation support device 100, or may be configured to use a part of the storage area of the storage unit 101 of the OCR recognition result confirmation support device 100.

With reference to FIG. 1, the flow of the entire process relating to the OCR recognition result confirmation support for the form will be described. First, a staff member (user) U of a municipal office or the like visits a user's house H, hands a predetermined form T to the user, and requests the user to fill in each entry field of the form T. The user writes by handwriting in the entry field of the form T using a writing instrument. Employee U collects the form T on which the handwritten characters are written and takes it back to the government office (step S1).

Employee U collects a plurality of forms T by visiting a plurality of user's homes H. The plurality of staff U for each process shown in FIG. 1 may be the same staff, different staff, or the same staff U may carry out work having a plurality of configurations. Good.

After that, the staff U reads each of the plurality of forms T using the scanner 110 in order to digitize the entry contents of the plurality of forms T collected by using the OCR recognition result confirmation support device 100 (step S2). The image data (image data) IMG of the form T read by the scanner 110 is input to the OCR recognition result confirmation support device 100 (step S3).

The OCR recognition result confirmation support device 100 performs, for example, OCR recognition (character recognition) for the image data IMG for each read form T and processing related to display for confirming the character recognition result (step S4). The storage unit 101 of the OCR recognition result confirmation support device 100 stores processing programs related to these character recognition and display control of the character recognition results, and a processor such as a CPU executes the processing programs.

The processing related to character recognition and display control of the character recognition result (step S4) includes decomposition processing of form T data (step S101), attribute-based conversion result data listing process (step S102), and integration of form T data. The process (step S103) is included.

In the data decomposition process of the form T (step S101), the OCR recognition result confirmation support device 100 performs a character recognition process for each form T. At the time of this character recognition processing, the OCR recognition result confirmation support device 100 holds the image data IMG of the form T and the character recognition result (text data) OCR obtained by character recognition processing of the image data in association with each other.

Here, the OCR recognition result confirmation support device 100 decomposes the character string of the image data into character units before executing the character recognition process, and shapes the image data of each character of the character string to be uniform in size. Process and perform. Details of these processes will be described later.

After that, the OCR recognition result confirmation support device 100 performs character recognition processing and decomposes the image data after character recognition into attributes. For example, the form T has a plurality of entry fields, one entry field having a name, another entry field having an address, and another entry field such as a questionnaire response. For example, the name, address, etc. in the entry fields of these different items correspond to attributes (attribute items). Then, the OCR recognition result confirmation support device 100 holds the image data for each attribute in the storage unit 101 as the conversion result data 102 for each attribute.

In the listing process (step S102), the OCR recognition result confirmation support device 100 uses a plurality of image data and character recognition results (text data) of the same attribute among the conversion result data 102 for each attribute such as name and address. Aggregate from the form T of. Then, the OCR recognition result confirmation support device 100 creates a list sorted by the character string data, and outputs the image data and the character recognition result in a comparable format (step S5). For example, the list is displayed and output on the display 103 as an output unit shown in FIG. 1, or printed out on a printer.

For example, the list L1 shown in FIG. 1 shows a display state in which names are sorted in ascending order as attributes. The OCR recognition result confirmation support device 100 arranges the image data IMG (L11) for confirming the name entered in the form T and the character recognition result OCR (text data) L12 side by side as the name list L1. In the vertical direction, the image data L11 for confirming the name written on a plurality of different forms T and the character recognition result (text data) L12 are arranged.

Further, the list L2 shown in FIG. 1 shows a display state in which addresses are sorted in ascending order as attributes. The OCR recognition result confirmation support device 100 arranges the address confirmation image data L21 entered in the form T and the character recognition result (text data) L22 side by side as the address list L2. In the vertical direction, the image data L21 for confirming the address entered in a plurality of different forms T and the character recognition result (text data) L22 are arranged.

The confirmation image data L11 and L21 shown in FIG. 1 and the image data described later are actually handwritten characters, but are described in a predetermined font for convenience.

For example, the staff U of the government office may be in charge of confirming the character recognition result for each attribute by a different staff. In the example of FIG. 1, the staff U1 confirms the character recognition result regarding the attribute of the name, and the staff U2 confirms the character recognition result regarding the attribute of the address.

Then, the staff U1 refers to the list L1 of the names displayed on the display 103 or the like, and confirms the character recognition result regarding the names. At this time, the staff member U1 compares the confirmation image data L11 displayed in the name list L1 with the character recognition result (text data) L12, thereby comparing the character recognition result (text data) L12 with respect to the confirmation image data L11. Confirm the work.

For example, the staff member U1 compares the confirmation image data L11 arranged side by side with each other on the name list L1 and the character recognition result (text data) L12. Then, it is confirmed whether the character recognition result (text data) L12 is correctly recognized as a character. In the example of FIG. 1, the character recognition result of the name list L1 is normal. Here, for example, when the character recognition result (text data) L12 is "Ryoko Kawaguchi" with respect to the image data "Akiko Kawaguchi" at the top of the name list L1, the staff U1 is assigned to the character recognition result (text data) L12. You can confirm that there is an error.

Here, the confirmation image data L11 of the name list L1 was actually handwritten on the form T, and the size and the like of each character were different, but the above decomposition process (step S101) was performed. The size of each character is made uniform by the image data shaping process. As a result, the staff member U1 can confirm each character of the confirmation image data L11 displayed in the name list L1 without overlooking. At this time, the staff U1 can easily compare the confirmation image data L11 having the same character size and the character recognition result (text data) L12. In this way, the OCR recognition result confirmation support device 100 supports the confirmation work performed by the staff U1 so that the confirmation work can be performed accurately and efficiently.

Here, with respect to the confirmation image data L11 of the name list L1 and the character recognition result (text data) L12, a plurality of different forms T are sorted and arranged in ascending order in the vertical direction. As a result, since the same character strings are continuously arranged in the vertical direction, it is possible to easily confirm (discover) if different characters are mixed when the confirmation image data L11 is visually confirmed in the vertical direction. become. Similarly, when the character recognition result (text data) L12 is visually checked in the vertical direction, it becomes possible to easily find a mixture of different characters (corresponding to erroneous recognition of characters).

Although the details will be described later, when a plurality of different forms T are sorted in ascending order in the vertical direction based on the character recognition result, the same character string is continuously arranged in the confirmation image data L11 in the vertical direction. Therefore, while the same character string is continuous, only the character recognition result (text data) L12 needs to be confirmed in the vertical direction, so that the confirmation work can be further made more efficient.

In addition, the staff U2 also refers to the address list L2 and provides support for easily confirming the character recognition result regarding the address. At this time, the staff member U2 compares the confirmation image data L21 displayed in the address list L2 with the character recognition result (text data) L22. Then, the correctness confirmation work is performed to confirm whether the character recognition result (text data) L22 for the confirmation image data L21 is correctly recognized as a character. For the staff U2, the OCR recognition result confirmation support device 100 supports the confirmation work of the character recognition result related to the address so that it can be accurately and efficiently performed.

Then, if there is an error in character recognition in the character recognition result, the staff U1 and U2 can correct the wrong character recognition results (text data) L12 and L22 by operating the keyboard 104 or the like as an operation unit, for example. It can be modified to the recognition result. At this time, the OCR recognition result confirmation support device 100 changes the data (text data) of the corresponding character recognition result of the attribute-based conversion result data 102 to the character (character string) input from the keyboard 104, and stores the data in the storage unit 101. Store.

In the example shown in FIG. 1, the example in which the attributes are the name and the address and the staff U1 and U2 perform the character recognition confirmation work is shown, but the other staff U performs the character recognition confirmation work for other attributes. You may.

Then, when the confirmation work of character recognition for each attribute of the form T is completed, the OCR recognition result confirmation support device 100 performs the data integration work of the form T using the conversion result data 102 for each attribute (step S103). ), Stored in DB105.

The conversion result data 102 by attribute is data decomposed by attributes such as name and address, but in this integration work, each attribute such as name and address is integrated for each form T, and each original form T is All the items to have are stored in the DB 105 as the character recognition result OCR (text data). Since the image data IMG is used for confirming the character recognition result, it is not necessary to store it in the DB 105, and the image data IMG is deleted when the character recognition confirmation work is completed. Not limited to this, the character recognition result OCR (text data) and the image data IMG corresponding to the character recognition result OCR (text data) may be stored in association with each other in the DB 105.

As a result, the DB 105 stores the digitized data obtained by converting the characters handwritten on the form T into text data. When the external terminal accesses the digitized data stored in the DB 105, the external terminal can easily acquire the information for each form T. The external terminal may be a terminal device provided with the display 103 and the keyboard 104.

FIG. 2 is a diagram showing a hardware configuration example of the OCR recognition result confirmation support device according to the embodiment. The OCR recognition result confirmation support device 100 can be configured by, for example, a general-purpose PC or server composed of the hardware shown in FIG.

The OCR recognition result confirmation support device 100 includes a CPU (Central Processing Unit) 201, a memory 202, a network interface (IF) 203, a recording medium IF204, a recording medium 205, and an input / output IF206. Reference numeral 200 denotes a bus connecting each part.

The CPU 201 is an arithmetic processing unit that functions as a control unit that controls the entire OCR recognition result confirmation support device 100. Memory 202 includes non-volatile memory and volatile memory. The non-volatile memory is, for example, a ROM (Read Only Memory) for storing the program of the CPU 201. The volatile memory is, for example, a DRAM (Dynamic Random Access Memory) used as a work area of the CPU 201, a SRAM (Static Random Access Memory), or the like.

The network IF203 is a communication interface for a network NW such as LAN (Local Area Network), WAN (Wide Area Network), and the Internet. The OCR recognition result confirmation support device 100 communicates and connects to the network NW via the network IF203. For example, the OCR recognition result confirmation support device 100 can communicate with an external terminal via the network NW.

The recording medium IF204 is an interface for reading and writing the information processed by the CPU 201 to and from the recording medium 205. The recording medium 205 is a recording device that assists the memory 202, and an HDD (Hard Disk Drive), an SSD (Solid State Drive), a USB (Universal Bus) flash drive, or the like can be used.

The CPU 201 executes the program recorded in the memory 202 or the recording medium 205 to realize the processing function of the OCR recognition result confirmation support. Further, the memory 202 and the recording medium 205 record and hold the information handled by the OCR recognition result confirmation support device 100. Further, the recording medium 205 corresponds to the storage unit 101 shown in FIG. 1 and stores and holds the conversion result data 102 for each attribute.

The input / output IF206 is an interface for inputting / outputting information to the CPU 201, and a scanner 110, a display 103, a keyboard 104, etc. (see FIG. 1) as external devices are connected via the input / output IF206.

Further, the database 105 shown in FIG. 1 can be configured by using the recording medium 205 shown in FIG. Further, the database 105 may be configured to include each of the other configurations (CPU 201 to recording medium 205) shown in FIG.

FIG. 3 is a diagram illustrating a conversion process of image data for OCR recognition according to the embodiment. FIG. 3A shows an example of the image data IMG (1) of the handwritten character that is the OCR recognition source of the above-mentioned form T. The character string of the image data IMG (1) of this handwritten character is "1-chome, Masuizumi, Kanazawa-shi", and there are a plurality of (n) characters 301 "gold", 302 "sawa", ..., 30n "eyes". It consists of letters. Then, for convenience, each character 301 to 30n shows an irregular state in which the size of each character is different.

The OCR recognition result confirmation support device 100 divides the image data IMG (1) in a state where the sizes of the characters are different and irregular in this way by dividing the image data one character at a time in units that can be recognized as characters (character units). Perform processing. After that, a plurality of conversion processes such as a shaping process for enlarging or reducing each character are performed so that the sizes of the characters are substantially equal, and the confirmation image data IMG (n) is generated. The confirmation image data IMG (n) is shaped so that the sizes of the characters 301, 302, ... 30n of the character string are aligned to a predetermined size. In the shaping process, each character may be enlarged or reduced so that the size of each character is substantially equal to the predetermined (reference) size. In addition, substantially equal means that the difference in height of each character is within a predetermined range, the difference in width of each character is within a predetermined range, and / or the difference in the area of a rectangle surrounding each character is within a predetermined range. And so on.

Then, the OCR recognition result confirmation support device 100 outputs the generated confirmation image data IMG (n) as confirmation image data L11 and L21 of the character recognition confirmation lists L1 and L2 (see FIG. 1). The parentheses (1) and (n) indicate the number of times the initial image data IMG (1) is increased each time a predetermined conversion process is performed, and finally the confirmation image is obtained in the nth conversion process. Data (n) is being generated.

(Example of image data conversion process)
FIG. 4 is a diagram showing an example of conversion processing of image data for OCR recognition according to the embodiment. As shown in FIGS. 4A to 4D, the control unit (CPU201) of the OCR recognition result confirmation support device 100 performs a plurality of (n) conversion processes on the image data IMG of handwritten characters.

As shown in FIG. 4A, first, the control unit generates image data IMG (1) obtained by cutting out an area of handwritten characters on the form T read by the scanner 110. In this example, a character string consisting of a plurality of characters 301 to 308 in a predetermined frame E (one line in the illustrated example) of the attribute "address" of the form T is cut out.

In the example shown in FIG. 4A, in order to emphasize the characteristics generated by the handwritten characters, for convenience, the size and writing state of each character in the character string of one line (the illustrated example expresses each handwritten character in a different font). ), The height position of each character in one line is different.

Next, as shown in FIG. 4B, the control unit generates the image data IMG (2) which has been subjected to the division process of dividing the character string in the frame E into a plurality of characters in units of one character. At this time, the control unit sets the division frames E1 to E8 for each character corresponding to the size of each character of the character 301 "gold", the character 302 "sawa", ... the character 308 "eye".

For example, the control unit sets a square (□) -shaped division frame E1 in which the character 301 “gold” fits the entire character (for example, the outer edge of the character) of the “gold”. Here, for a character smaller than the character 301 "gold" such as the character 302 "sawa", the control unit also compares the division frame E2 with the division frame E1 according to the size of the character "sawa". Set it small.

After that, as shown in FIG. 4C, the control unit receives each of the characters 301 "gold", the characters 302 "sawa", ... the characters 308 "eyes" of the image data IMG (2) after the division process. The image data IMG (3) is generated by performing a shaping process in which the size of characters is enlarged or reduced to make them uniform.

For example, the control unit sets the other characters 302 "sawa" to 308 "eyes" to the same size as the character 301 "gold" with reference to the largest character 301 "gold". In this case, the control unit of the other characters 302 "sawa" of the division frames E2 to the characters 308 "eyes" so as to have the same size (vertical and horizontal) as the division frame E1 of the reference character 301 "gold". The division frames E8 are enlarged or reduced to the same size as the reference. As a result, the size of the character 301 "gold", the character 302 "sawa", ... the character 308 "eye" of the character string becomes the same.

After that, as shown in FIG. 4D, the control unit concatenates the characters of the character string 301 "gold", 302 "sawa", ..., 308 "eyes", and image data IMG (n). To generate. In this case, the image data conversion process is performed four times, and n = 4 of IMG (n). By this process, the image data IMG (3) divided in units of one character is connected to the image data IMG (1) in the state before the division in a state where the sizes of the characters are aligned.

For example, the control unit connects the division frame E1 of the character 301 “gold”, the division frame E2 of the character 302 “sawa”, and the division frame E8 of the character 308 “eye” in the row direction, and one image data IMG (n). ) Is generated. The generated image data IMG (n) corresponds to the confirmation image data IMG described with reference to FIG. 3 (b). Further, the control unit outputs this image data IMG (n) as confirmation image data L21 of the address list L2 shown in FIG.

(Processing example of OCR recognition result confirmation support device)
FIG. 5 is a flowchart showing a processing example of OCR recognition result confirmation support according to the embodiment. An example of processing that is executed by the control unit (CPU) 201 and that mainly supports confirmation of character recognition results for a plurality of forms T is shown.

First, the control unit cuts out the image data IMG of the portion to be OCR (character recognition) for one form T from the plurality of forms T (step S501). This process corresponds to cutting out the image data IMG (1) in the frame E of FIG. 4 (a).

Next, the control unit divides the cut image data into image data in character units (step S502). By this process, the image data IMG (2) divided by the character-by-character division frames E1 to E8 shown in FIG. 4B is generated.

Next, the control unit shapes the size of the character portion in the image data divided into character units (step S503). By this processing, the division frames E1 to E8 of each character shown in FIG. 4C are enlarged or reduced to the same size as the reference, so that the image data IMG (3) having the same size of each character is used. ) Is generated.

Next, the control unit concatenates the image data of one character unit whose size has been shaped into the state before the division (step S504). By this process, the image data IMG (4) in which the divided frames E1 to E8 having the same size of each character shown in FIG. 4D are connected is generated.

This image data IMG (4) corresponds to confirmation image data (see FIG. 3B) for the user U to confirm the character recognition result. For example, when the user U confirms the character recognition result, the control unit outputs the generated image data IMG (4) as the confirmation image data L21 of the address list L2 shown in FIG.

Next, the control unit determines whether all the parts of one form T have been OCR (character recognition) (step S505). The form T has a plurality of entry points, character recognition is performed for all the entry points, and the above-mentioned image data IMG is generated. The plurality of entry points correspond to each attribute such as the name and address described above. The character recognition can also be performed on the initial image data obtained by reading the form T. Then, if all the parts of one form T have been recognized as characters (step S505: Yes), the control unit shifts to the process of step S506. On the other hand, if there is a part where the character recognition is not completed in one form T (step S505: No), the process returns to the process of step S501 and the process of step S501 or less is executed again for the unprocessed portion.

In step S506, the control unit determines whether all the forms T have been processed (step S506). In the above process, character recognition is performed for one form T and image data IMG is generated. Here, it is determined whether the image data IMG is generated by character recognition for a plurality of target forms T. If all the forms T have been processed (step S506: Yes), the control unit proceeds to the process of step S507, and if there is an unprocessed form T (step S506: No), returns to the process of step S501. ..

In step S507, the control unit outputs an OCR (character recognition) result listed in attribute units such as name and address (step S507). By the above processing, the control unit generates an image data IMG for each attribute of the name and address of the handwritten characters entered in the form T, and displays the generated image data IMG on the display 103. Further, it may be printed out via a printer.

This image data IMG is used as confirmation image data on the list when the staff (user) U confirms the character recognition result of the form T. The details will be described later, but for example, the image data IMG is displayed as the confirmation image data L11 of the list L1 of the attribute “name” of the example described in FIG. To do. Similarly, the image data IMG is displayed as the confirmation image data L21 of the list L2 of the attribute "address", and the character recognition result (text data) L22 is displayed and output next to the image data IMG.

As a result, the staff U can confirm the correctness of the character recognition result by comparing the image data displayed in the lists L1 and L2 for each attribute with the character recognition result (text data). For example, the staff U conveys the correctness of the character recognition result to the control unit by operating the keyboard 104 or the like based on the character recognition result. At this time, if there is an erroneous character recognition result (text data), the staff U inputs the correct character and corrects the character recognition result. If there is no error in character recognition, no operation is performed, or the correct character recognition result is input.

Then, the control unit determines whether or not all the attributes have been confirmed (step S508). When the confirmation work of the character recognition for all the attributes of the form T such as the name and address by the staff (user) U is completed (step S508: Yes), the control unit shifts to the process of step S509. If the unprocessed attributes remain in the character recognition confirmation work (step S508: No), the process returns to the process of step S507.

In step S509, the control unit integrates the OCR (character recognition) result into one form T unit (step S509), and ends the above series of processes. In this process, the DB 105 stores electronic data in which image data IMGs of a plurality of attributes such as a name and an address included in one form T are integrated as a set of data in units of this one form T.

The image data IMG is generated for comparison with the character recognition result, and after confirming the correctness of the character recognition result in step S508, all the image data IMGs of the confirmed form T become unnecessary. .. Therefore, the image data IMG generated above may be deleted when the staff U confirms the correctness of the character recognition result. For example, the control unit deletes the image data IMG in the process of step S509. In this case, only the character recognition result of the form T is stored in the DB 105, and the image data IMG is not stored.

6A and 6B are diagrams showing an example of an OCR result confirmation list output by the OCR recognition result confirmation support device according to the embodiment. FIG. 6A shows an output example of a list L2 in which a plurality of forms T are sorted by address attributes. The control unit (CPU201) of the OCR recognition result confirmation support device 100 lists the result No. (601) for each form T, the character recognition (OCR) result (address) L22, and the confirmation image data L21 as the list L2. ..

Each line shows the data of one form T. For example, the top one line is the data of one form T (result No. "104", and the character recognition result L22 is "Masuizumi 1-chome, Kanazawa" (text). Data) 611 is displayed. In addition, image data 612 of handwritten characters of "Masuizumi 1-chome, Kanazawa City" is displayed as confirmation image data L21.

The second line is the data of another form T (result No. 789), and displays "Kanazawa-shi Masuizumi 1-chome" (text data) 613 as the character recognition (OCR) result L22. Further, as the confirmation image data L21, the image data 614 of the handwritten characters of "Masuizumi 1-chome, Kanazawa City" is displayed. The third line displays "Kanazawa-shi Masuizumi 1-chome" (text data) 615 as yet another form T (data of result No. "159" and character recognition result L22. Also, confirmation image data. Image data 616 of handwritten characters of "Masuizumi 1-chome, Kanazawa" is displayed as L21.

The staff (user) U2 who confirms the character recognition result of the address compares the character recognition result L22 arranged side by side with each other on the address list L2 and the confirmation image data L21. For example, in the first line in the form T unit, the text data 611 in the area of the character recognition result L22 and the image data 612 in the area of the confirmation image data L21 are compared. As a result, it is confirmed whether the character recognition result (text data) L12 is correctly recognized as a character.

The confirmation image data L21 is the image data after the decomposition process of decomposing the image data of the character string of the handwritten character for each character and the shaping process of making the size of each character the same (the above IMG). Equivalent to (4)). Here, the confirmation image data L21 was actually handwritten on the form T, and the size of each character was different, but the size of each character was made uniform by the above-mentioned decomposition process and shaping process. Has been done.

As a result, the staff U2 can confirm each character of the confirmation image data L21 displayed in the address list L2 without overlooking. At this time, the staff U2 can easily compare the confirmation image data L21 having the same character size and the character recognition result (text data) L22.

For example, the text data 611 of the character recognition result L22 of the form T (result No. “104”) in the uppermost line is “Masuizumi 1-chome, Kanazawa City”, and the image data 612 of the confirmation image data L21 is “Kanazawa City”. Masuizumi 1-chome ". In this case, the staff U2 can determine that the text data 611 of the character recognition result L22 is correctly recognized as a character.

Further, the list L2 sorts the character recognition results in ascending order of the addresses, and the OCR recognition result confirmation support device 100 continuously sets the same address in the vertical direction of the character recognition results L22 of a plurality of forms T. Deploy. In the illustrated example, a plurality of forms T of the address "Masuizumi 1-chome, Kanazawa" are displayed.

By sorting and displaying in this way, the staff (user) U2 looks at only the confirmation image data L21 in order from the top while the same character recognition result "Masuizumi 1-chome, Kanazawa" is continuous, and the characters. All you have to do is determine if there are any dissimilar ones. In this way, it becomes possible to easily confirm the correctness of the character recognition results of the plurality of forms T.

The moving state of the line of sight P of the staff (user) U2 at this time is illustrated. For example, if "Masuizumi 1-chome, Kanazawa City" is displayed as the character recognition result 611 of the new address in the area of the character recognition result L22, the subsequent line of sight P moves the area of the confirmation image data L21 in order from the top. Just let me do it. That is, it is only necessary to move the image data 612 → the image data 614 → the image data 616 vertically.

Here, in the image data 612,614,616 of each form T displayed in the confirmation image data L21, each character of the character string has the same size by the above-mentioned image data division processing and shaping processing. It is arranged in. Therefore, in the image data 612, 614, 616 of each form T displayed in the vertical position in the area of the confirmation image data L21, each character of the character string is located at substantially the same position. For example, the first character 301 "gold" of the address of the image data 612,614,616 is displayed at the position p1 at substantially the same position in the vertical direction. Similarly, the last character 308 "eye" of the address of the image data 612,614,616 is displayed at the position pn at substantially the same position in the vertical direction. That is, each character in the character string is displayed in the vertical direction without any misalignment.

The image data 612, 614, 616 of each form T displayed in the confirmation image data L21 is a character string composed of a plurality of characters. In the embodiment, the character spacing is made uniform by making each character of the character string the same size, and the characters are displayed without shifting in the vertical direction.

As a result, when the line of sight P is moved, the staff U2 intuitively captures the entire character string that is continuous up and down in the area of the confirmation image data L21 as an image, for example, a figure, and finds a dissimilar figure. Work can be done. Then, the work of confirming the correctness of the character recognition result can be efficiently performed.

FIG. 6A shows a state in which the character recognition result of the result No. “159” of the form T is erroneous recognition. The character recognition result 615 of the form T result No. "159" in the character recognition result L22 area of the character recognition result L22 is "Masuizumi 1-chome, Kanazawa", but the image data in the corresponding confirmation image data L21 area. It is assumed that 616 was "Masuizumi 2-chome, Kanazawa City". In this case, while the staff U2 is looking at the entire character string that is continuous up and down in the area of the confirmation image data L21, the sixth character X in the character string of "Masuizumi 2-chome, Kanazawa City" of the image data 616. It can be easily found that "2" is different from the top and bottom. In this case, the character recognition result 615 is correctly "Masuizumi 2-chome, Kanazawa City", and the confirmation work is performed that the character recognition result of the address of the address No. "159" of the result T of the form T is incorrect.

Since the address list L2 is sorted in ascending order based on the character recognition result, the staff U2 moves the line of sight P up and down within the area of the confirmation image data L21 to display the similarity state in the image (corresponding to the figure). You can easily find an error in the character recognition result just by checking it.

Further, FIG. 6B shows an output example of the list L1 in which a plurality of forms T are sorted by the attribute of the name. The control unit (CPU201) of the OCR recognition result confirmation support device 100 also performs the result No. (601) and character recognition (601) for each form T for the name attribute list L1 as well as the address attribute list L2 described above. OCR) Result (name) L12 and confirmation image data L11 are listed.

At this time, in the image data 622,624,626 of each form T in the area of the confirmation image data L11, each character of the character string has the same size and is located at the same position in the vertical position. For example, the first character 301 "river" of the name of the image data 622,624,626 is displayed at the position p1 at substantially the same position in the vertical direction. Similarly, the second character 302 “mouth” of the name in the image data 622,624,626 is displayed at position p2 at substantially the same position in the vertical direction. In this way, each character in the character string is displayed in the vertical direction without any misalignment.

As a result, the staff (user) U1 can confirm each character of the confirmation image data L11 displayed in the name list L1 without overlooking. At this time, the staff member U1 can easily compare the confirmation image data L11 having the same character size and the character recognition result (text data) L12, and can easily confirm the correctness of the character recognition result.

In addition, when the staff (user) U1 displays "Masuizumi 1-chome, Kanazawa City" as the character recognition result 621 of the new name in the area of the character recognition result L12, as shown in the moving state of the line of sight P, For the subsequent line of sight P, the area of the confirmation image data L11 may be moved in order from the top. That is, it is only necessary to move the image data 622 → the image data 624 → the image data 626 vertically. As a result, when the line of sight P is moved, the staff U1 intuitively finds a dissimilar figure by grasping the entire character string that is continuous vertically in the area of the confirmation image data L11 as an image (corresponding to a figure). Work can be done. Then, the work of confirming the correctness of the character recognition result can be efficiently performed.

(Comparison reference diagram)
FIG. 7 is a diagram showing a list of character recognition results corresponding to the prior art. For convenience, FIG. 7 has the same display format as the address attribute similar to that of the embodiment (FIG. 6A) for comparison with the embodiment. That is, a list L of addresses having the result No. (701) of the form T, the character recognition (OCR) result 702, and the confirmation image data 703, and the character recognition result 702 sorted in ascending order is shown.

Conventionally, by arranging the character recognition result 702 and the confirmation image data 703 for each form T, the correctness of the character recognition result can be confirmed in the form T unit of one line. However, in the prior art, the image data 712,714,716 of each form T displayed in the area of the confirmation image data 703 displays the characters handwritten on the form T as they are as an image.

Therefore, in the image data 712,714,716 displayed in the area of the confirmation image data 703, the size of the characters 301 to 30n is different for each form T, the character spacing is also different, and the size of the entire character string ( Width) is also different. For this reason, conventionally, it takes time and effort to confirm the correctness of the character recognition result, and an error in the correctness judgment has also occurred. As described above, conventionally, it is difficult to compare the character recognition result 702 and the confirmation image data 703, and the confirmation work cannot be performed efficiently and accurately.

Further, in the area of the confirmation image data 703, the characters of the image data 712, 714, 716 of different forms T are not positioned vertically. For example, suppose that the area of the vertical position p1 is viewed with reference to the position of the first character 301 "gold" of the image data 712 of the form T on the first line (result No. "104"). In this case, in the image data 716 of the form T on the third line (result No. "159"), the first character 301 "gold" and the second character 302 "sawa" are located.

Further, when viewed in the area of the vertical position pn of the position of the character 308 "eyes" at the end of the image data 716 of the form T on the third line (result No. "159"), the first line (result No. "104"). The fourth character 304 "increase" of the image data 712 of the form T of the above is located. Here, the end of the image data 712 of the form T of the first line (result No. "104") is compared with the character 308 "eye" at the end of the image data 716 of the form T of the third line (result No. "159"). The character 308 "eye" of is located at a position shifted in the character string direction (horizontal direction).

As described above, since the characters of the image data 712, 714, 716 of the different forms T are not located vertically, the line of sight P is moved in the vertical direction only in the area of the confirmation image data as described in the embodiment. It is not possible to capture different images. As a result, it is not possible to determine whether or not the characters are not similar in the character string of each line, and it is not possible to streamline the work of confirming the correctness of the character recognition result.

In the prior art, even if the display format shown in FIG. 7 is used, in the confirmation image data 703, the same characters are misaligned even if the same character string images are arranged because the size of each character in the character string is different. Do not line up above and below. Characters to be compared (first characters, second characters, ...) Are misaligned in the character string and do not line up one above the other. In addition, in handwritten characters, the size of each character varies depending on the habit of the person who filled out the form, so even if the same characters in the confirmation image data 703 are lined up in the same vertical position, they will look different. You may make a mistake.

As described above, the conventional technique cannot realize the work of confirming the handwritten character string at a glance. In addition, the essence of glance confirmation is to detect a strange "character shape" from the arranged character shapes, and in this essence, irregular character sizes are an obstacle to the confirmation work. For example, in the technique of Patent Document 2 described above, the feature amount of the image is used, but the confirmation accuracy is lowered because it becomes a calculation element of clustering up to an unrelated character size.

On the other hand, in the embodiment, as described with reference to FIGS. 6A and 6B, the image data in the areas of the confirmation image data L11 and L21 of the different forms T have the same upper and lower characters in the character string. It is located in position. As a result, the staff (user) U can perform the work of moving the line of sight P in the vertical direction only in the areas of the confirmation image data L11 and L21 to capture different images. Then, it is possible to easily determine whether or not the characters are not similar in the character string of each line, and it is possible to streamline the work of confirming the correctness of the character recognition result.

In the embodiment described above, in order to support the user in confirming the OCR recognition result, image data is acquired, and each character is substantially equal in size of each character in the character string included in the image data. Enlarge or reduce the size of. Then, the enlarged or reduced characters are concatenated to generate confirmation image data of the character string, and the character recognition result obtained as a result of character recognition of the image data or confirmation image data can be compared with the confirmation image data. Output a list of display formats. As a result, each character of the confirmation image data displayed in the list is aligned to the same size, and when the user confirms the character recognition result, the confirmation image data and the character recognition result of the confirmation image data are displayed. You will be able to easily compare and perform confirmation work appropriately. There are various types of handwritten characters written on forms, etc., and their sizes and handwriting states are different. However, by dividing and shaping the image data of handwritten characters as described above, each character in the character string has the same size. This makes it possible to improve the visibility of the confirmation image data by the user, and it becomes possible to easily and correctly compare the character recognition result with the correctness of the character recognition result.

Further, the acquired image data may be divided for each character, and the size of each divided frame may be enlarged or reduced to a reference size. This processing can be easily performed by general-purpose image processing on the image data, and confirmation image data with improved visibility can be easily generated.

In addition, the form generally has a plurality of handwritten entry fields for each attribute. Corresponding to such a form, in the acquisition of image data, image data for each attribute of a plurality of forms is acquired. Further, in the output of the list, the confirmation image data of a predetermined attribute common to a plurality of forms and the character recognition result of the confirmation image data are output as a list in a comparable display format. As a result, it becomes possible to confirm the character recognition result for each attribute for a plurality of forms. The form is provided with fields for entering multiple attributes such as name and address, and by confirming the character recognition result for each attribute, this confirmation work can be performed efficiently.

Further, in the output of the list of a plurality of forms, the character string included in the character recognition result of the character string of one attribute common to the plurality of forms is sorted. Then, the character recognition results of the plurality of sorted forms and the confirmation image data corresponding to the character recognition results may be output as a list in a comparable display format. For example, by using a list sorted by attributes such as name and address, character recognition results and confirmation image data can be displayed in ascending order of names. Therefore, for example, it is possible to perform an intuitive work of capturing the entire continuous character string as an image, for example, a figure in the display area portion of the confirmation image data over a plurality of forms, and finding a figure that is not similar. Then, the work of confirming the correctness of the character recognition result can be efficiently performed.

Further, based on the confirmation of the correctness of the character recognition result by the user using the output list, the confirmed character recognition result is stored as electronic data of the form. As a result, the form of handwritten characters can be easily digitized and stored as electronic data with correctly recognized characters.

Further, the character recognition results for each of a plurality of attributes may be integrated for each form and stored as electronic data of the form. As a result, it is possible to obtain electronic data for each form including all the handwritten entry fields of one form. In addition, form data can be easily input / output to an external terminal that performs various processes based on the contents of the form.

Further, the OCR recognition result confirmation support system includes an OCR recognition result confirmation support device that performs each of the above processes, and a scanner that reads a character string handwritten on a form and outputs image data. Further, an output unit that displays or prints out the list and an operation unit for user operation to confirm the correctness of the character recognition result based on the output list may be included. By connecting such a general-purpose external device, the user can efficiently confirm the character recognition result.

From these facts, according to the embodiment, it becomes possible to improve the character recognition accuracy of the form described by the handwritten character and digitize it. That is, the user can compare the image data for confirming the handwritten characters of the form with the character recognition result (text data) for character recognition, and the confirmation work for confirming the correctness of the character recognition result can be made more efficient. At this time, the visibility of the confirmation image data itself can be improved, and the user can correctly judge the correctness of the character recognition result, so that it becomes possible to finally obtain electronic data with improved character recognition accuracy of the entire form. ..

The method for supporting OCR recognition result confirmation described in the embodiment of the present invention can be realized by causing a processor such as a server to execute a program prepared in advance. This method is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versailles Disk), or a flash memory, and is read from the recording medium by the computer. Is executed by. Further, this method may be distributed via a network such as the Internet.

The following additional notes are further disclosed with respect to the above-described embodiment.

(Appendix 1) Obtain image data and
Enlarge or reduce the size of each character so that the size of each character of the character string included in the image data is substantially equal.
Image data for confirmation of the character string is generated by concatenating the enlarged or reduced characters.
Outputs a list of display formats capable of comparing the character recognition result obtained as a result of character recognition of the image data or the confirmation image data with the confirmation image data.
An OCR recognition result confirmation support program characterized by having a computer execute processing.

(Appendix 2) The image data has a plurality of handwritten entry fields for each attribute on the form.
In the acquisition of the image data, image data for each of the attributes of the plurality of forms is acquired.
In the output of the list, the confirmation image data of the predetermined attribute common to the plurality of forms and the character recognition result of the confirmation image data are output as a list in a comparable display format.
The OCR recognition result confirmation support program described in Appendix 1 characterized by this.

(Appendix 3) In the output of the list, the character string of the character string of one attribute common to the plurality of forms is sorted by the character string included in the character recognition result.
The character recognition result of the sorted plurality of forms and the confirmation image data corresponding to the character recognition result are output as a list in a comparable display format.
The OCR recognition result confirmation support program described in Appendix 2 characterized by this.

(Appendix 4) Based on the confirmation of correctness of the character recognition result by the user using the output list, the confirmed character recognition result is stored as electronic data of the form.
The OCR recognition result confirmation support program described in Appendix 2 or 3, characterized in that.

(Appendix 5) The character recognition results for each of a plurality of attributes are integrated for each form and stored as electronic data of the form.
The OCR recognition result confirmation support program described in Appendix 4 characterized by the above.

(Appendix 6) The acquired image data is divided into a plurality of division frames including the outer shapes of the plurality of characters.
Enlarge or reduce the size of the divided frame to a reference size.
The OCR recognition result confirmation support program according to any one of Supplementary notes 1 to 5, characterized in that.

(Appendix 7) Obtain image data and
Enlarge or reduce the size of each character so that the size of each character of the character string included in the image data is substantially equal.
Image data for confirmation of the character string is generated by concatenating the enlarged or reduced characters.
Outputs a list of display formats capable of comparing the character recognition result obtained as a result of character recognition of the image data or the confirmation image data with the confirmation image data.
An OCR recognition result confirmation support method characterized in that a computer executes processing.

(Appendix 8) Acquire image data, enlarge or reduce the size of each character so that the size of each character in the character string included in the image data is substantially equal, and concatenate the enlarged or reduced characters. Then, the confirmation image data of the character string is generated, and the character recognition result obtained as a result of character recognition of the image data or the confirmation image data is displayed in a display format in which the confirmation image data can be compared. Control unit to output,
OCR recognition result confirmation support system characterized by being equipped with.

(Appendix 9) A scanner that reads a character string handwritten on a form and outputs the image data.
An output unit that displays or prints out the list,
Based on the output list, an operation unit for user operation to confirm the correctness of the character recognition result, and
The OCR recognition result confirmation support system according to Appendix 8, which is characterized by the above.

100 OCR recognition result confirmation support device 101 Storage unit 102 Conversion result data by attribute 103 Display 104 Keyboard 105 Database 110 Scanner 201 CPU (Control unit)
202 Memory 203 Network interface 205 Recording medium 206 Input / output interface 301-30n characters 611,613,615,621 Character recognition result (text data)
612,614,616,622,624,626 Image data E frame E1 to E8 Divided frame H User's home NW network IMG image data L1 (L1, L2) List L11, L21 Confirmation image data L12, L22 Character recognition result ( Text data)
P line of sight (moving state of line of sight)
T form U (U1, U2) staff (user)

Claims (8)

Get image data
Enlarge or reduce the size of each character so that the size of each character of the character string included in the image data is substantially equal.
Image data for confirmation of the character string is generated by concatenating the enlarged or reduced characters.
Outputs a list of display formats capable of comparing the character recognition result obtained as a result of character recognition of the image data or the confirmation image data with the confirmation image data.
An OCR recognition result confirmation support program characterized by having a computer execute processing. The image data has a plurality of handwritten entry fields for each attribute on the form.
In the acquisition of the image data, image data for each of the attributes of the plurality of forms is acquired.
In the output of the list, the confirmation image data of the predetermined attribute common to the plurality of forms and the character recognition result of the confirmation image data are output as a list in a comparable display format.
The OCR recognition result confirmation support program according to claim 1, characterized in that. In the output of the list, the character string of the character string of one attribute common to the plurality of forms is sorted by the character string included in the character recognition result.
The character recognition result of the sorted plurality of forms and the confirmation image data corresponding to the character recognition result are output as a list in a comparable display format.
The OCR recognition result confirmation support program according to claim 2, characterized in that. Based on the confirmation of correctness of the character recognition result by the user using the output list, the confirmed character recognition result is stored as the form electronic data.
The OCR recognition result confirmation support program according to claim 2 or 3, characterized in that. The character recognition results for each of a plurality of attributes are integrated for each form and stored as electronic data of the form.
The OCR recognition result confirmation support program according to claim 4, characterized in that. Get image data
Enlarge or reduce the size of each character so that the size of each character of the character string included in the image data is substantially equal.
Image data for confirmation of the character string is generated by concatenating the enlarged or reduced characters.
Outputs a list of display formats capable of comparing the character recognition result obtained as a result of character recognition of the image data or the confirmation image data with the confirmation image data.
An OCR recognition result confirmation support method characterized in that a computer executes processing. The image data is acquired, the size of each character is enlarged or reduced so that the size of each character of the character string included in the image data is substantially equal, and the enlarged or reduced characters are concatenated to form the character. A control unit that generates column confirmation image data and outputs a list of display formats that can compare the character recognition result obtained as a result of character recognition of the image data or the confirmation image data with the confirmation image data. ,
OCR recognition result confirmation support system characterized by being equipped with. A scanner that reads the character string handwritten on the form and outputs the image data,
An output unit that displays or prints out the list,
Based on the output list, an operation unit for user operation to confirm the correctness of the character recognition result, and
The OCR recognition result confirmation support system according to claim 7, wherein the OCR recognition result confirmation support system is provided.

Images