JP7088661B2 - Paper form data conversion system, OCR engine learning image generator and image analyzer - Google Patents

Description

The present invention is a paper form data conversion system that extracts and outputs character information from a target image obtained by scanning a form, and an OCR engine learning image that causes an OCR engine that extracts character information from an image to learn to extract character information from an image. The present invention relates to a generator and an image analyzer that extracts character information from a target image.

The forms handled by companies are diverse. Such forms include, for example, invoices issued by other companies and the results of employee health examinations. As for the contents of these forms, electronic data of character information is often not available.

From such paper forms created in-house or received from outside, necessary information may be input to a computer as electronic data and used for various corporate activities. When inputting to a computer, the image of the form may be read by a scanner and then converted into text information by OCR.

JP-A-2015-46027

In recent years, the accuracy of extracting character information by OCR from images of forms such as invoices has improved. However, if there is an error in the extracted character information, it must be manually corrected one by one. Therefore, it is desired to further improve the accuracy of extracting character information from the document image.

Therefore, an object of the present invention is to improve the accuracy of extracting character information from a form such as an invoice.

In order to achieve the above object, the present invention uses an OCR engine to obtain character information from the target image in a paper form data conversion system that extracts character information from the target image obtained by scanning the form and outputs it to an external database. The OCR unit to be extracted, the structural analysis unit that generates structured data in which the character information extracted by the OCR unit is structured based on the position of the character information on the target image, the structured data, and the database. It is characterized by having a mapping unit for inputting the structured data into the database using a mapping table showing a correspondence relationship with the structure.

Further, according to the present invention, in an image generation device for learning an OCR engine that causes an OCR engine that extracts character information from an image to learn the extraction of character information from an image, the present invention applies the learning image conversion to characters of a specific font for learning. A second image is converted to a first image by using a pair of a learning image generator that generates an image for use, a first image including recognized characters, and a second image in which the recognized characters are represented by the specific font. It is characterized by having a learning image generation learning unit that causes a learning image generator to learn the learning image conversion to be converted.

Further, according to the present invention, in an image analyzer that extracts character information from a target image, a learning image generator that generates a learning image by performing the learning image conversion on characters of a specific font and recognized characters are used. Learning to make a learning image generator learn image conversion for learning to convert a second image to a first image using a pair of a first image including the first image and a second image expressing the recognized characters in the specific font. The image generation learning unit for learning and the character recognition learning unit for learning the extraction of characters from the image using the pair of the learning image generated by the learning image generator and the characters corresponding to the learning image. It is characterized by having an OCR engine learned by an OCR engine learning device provided, and an OCR unit that extracts character information from the target image using the OCR engine.

Further, according to the present invention, in an image analyzer that extracts character information from a target image, a first pre-process image in which characters and non-character images are mixed and an image other than the characters in the first pre-process image are removed. The first learning device that causes the first machine learner to learn the first conversion for removing the non-character image from the image in which the character and the non-character image are mixed by using the pair with the processed image, and the first conversion. It is characterized by having a preprocessing unit that applies preprocessing including the target image to the target image, and an OCR unit that extracts character information from the target image.

Further, according to the present invention, in an image analyzer that extracts character information from a target image, a second processed image including a character group represented by a ruled line and a pre-processed image in which the ruled line of the second processed image is removed. A second learning device that causes a second machine learner to learn a second conversion that converts an image of a character group that is represented by a set with an image without using a ruled line into an image of a character group that is represented by a ruled line. It is characterized by having a preprocessing unit that performs preprocessing including the second conversion on the target image, and an OCR unit that extracts character information from the target image.

Further, in the image analyzer for extracting character information from a target image, the present invention forms a frame that does not overlap with each other in a third pre-process image including a character group and each character included in the third pre-process image. A third learning device that causes a third machine learner to learn a third conversion that forms a frame that does not overlap each other in each character included in an image including a character group by using a set with the image after processing, and the third. It is characterized by having a preprocessing unit that performs preprocessing including conversion on the target image, and an OCR unit that extracts character information from the target image.

According to the present invention, it is possible to improve the accuracy of extracting character information from a form such as an invoice.

It is a block diagram in one Embodiment of the form data digitization system which concerns on this invention. It is a top view of the form to be analyzed of one Embodiment of the form data digitization system which concerns on this invention. It is a flowchart which shows the flow of analysis of one Embodiment of the form data digitization system which concerns on this invention. It is a top view of the preprocessed image in one Embodiment of the form data digitization system which concerns on this invention. It is an example of the plan view of the character group expressed without using the ruled line in one embodiment of the form data digitization system which concerns on this invention. It is an example of the plan view of the character group represented by the ruled line in one Embodiment of the form data digitization system which concerns on this invention. It is an example of the plan view of the character group in one Embodiment of the form data digitization system which concerns on this invention. It is an example of a plan view surrounded by a frame character by character in one embodiment of the form data digitization system according to the present invention. It is an image correctly recognized as a specific character in one embodiment of the form data digitization system according to the present invention, and an image expressing the character using a specific font. It is an image for learning in one embodiment of the form data digitization system which concerns on this invention. It is an example of a part of a form in one Embodiment of the form data digitization system which concerns on this invention. It is a mapping screen in one Embodiment of the form data digitization system which concerns on this invention.

An embodiment of the image analyzer according to the present invention will be described with reference to the drawings. It should be noted that this embodiment is merely an example, and the present invention is not limited thereto. The same or similar configurations are designated by the same reference numerals, and duplicate description will be omitted.

FIG. 1 is a block diagram of an embodiment of a paper form data conversion system according to the present invention.

The paper form data conversion system 10 of the present embodiment extracts character information from an atypical document such as a form printed on paper, and outputs image analysis structured according to the structure of the document. It is a device. Further, the paper form data conversion system 10 inputs structured character information into the database.

The paper form data conversion system 10 of the present embodiment maps the image storage unit 20, the image analysis pre-processing unit 21, the pre-processed image storage unit 22, the OCR unit 23, the image analysis post-processing unit 24, and the structural analysis unit 25. It has a unit 26, an extracted character string storage unit 29, a dictionary 30, an OCR engine learning device 70, a scanner 12, a display 13, a keyboard 14, and a mouse 15. The paper form data conversion system 10 is configured on, for example, one computer. A part of the paper form data conversion system 10, for example, an image storage unit 20, an image analysis preprocessing unit 21, a preprocessed image storage unit 22, an OCR unit 23, an image analysis post-processing unit 24, a structural analysis unit 25, and a mapping unit 26. , The extracted character string storage unit 29, and a part or all of the dictionary 30 may be distributed and arranged on a plurality of computers connected to each other by a network. Image storage unit 20, image analysis pre-processing unit 21, pre-processed image storage unit 22, OCR unit 23, image analysis post-processing unit 24, structural analysis unit 25, mapping unit 26, extracted character string storage unit 29, and dictionary 30. The OCR engine learning device 70 is realized by a program that gives a computer each function.

The image analysis preprocessing unit 21 has a first learning device 51, a second learning device 52, and a third learning device 53. The first learning device 51 includes a first machine learning device 61. The second learning device 52 includes a second machine learning device 62. The third learning device 53 includes a third machine learning device 63. The first machine learning device 61, the second machine learning device 62, and the third machine learning device 63 are all machine learning devices including GAN (Generative Adversarial Networks).

The OCR engine learning device 70 has a learning image generation learning unit 71, a learning image generator 72, and a character recognition learning unit 73.

FIG. 2 is a plan view of a form to be analyzed by the form data digitization system of the present embodiment.

The paper form data conversion system 10 of the present embodiment analyzes, for example, an image (document image 90) of a form that summarizes the monthly billing amount printed on paper. It is also possible to analyze not only what is printed on paper but also what is displayed on the display of a smartphone or computer.

Next, the flow of the analysis process of the document image 90 using the paper form data conversion system 10 of the present embodiment will be described.

FIG. 3 is a flowchart showing the flow of analysis of the image analyzer of the present embodiment.

The document image 90 is, for example, scanned by the scanner 12 and stored in the image storage unit 20 of the paper form data conversion system 10 (S1). Instead of the scanner 12, the image may be taken with a camera such as a smartphone. When taking an image with a camera, keystone correction or horizontal correction may be applied to the image. The document image 90 is converted to, for example, grayscale.

Next, the image analysis preprocessing unit 21 performs preprocessing on the document image 90 stored in the image storage unit 20 (S2).

Extracting characters from an image that includes hatching or shading in addition to the characters tends to reduce the extraction accuracy. Therefore, before extracting the characters, the image analysis preprocessing unit 21 removes an image other than the characters from the target image. That is, the image analysis preprocessing unit 21 removes a portion of an image other than the characters, that is, changes the color to a paper surface such as white.

FIG. 4 is a plan view of the preprocessed image in the present embodiment.

The image analysis pre-processing unit 21 previously performs characters such as hatching and shading as shown in FIG. 4 from the first pre-processing image (document image 90) including hatching and shading as shown in FIG. The first conversion for generating the image 54 after the first processing from which the parts other than the above are removed is being learned. The first processed image 54 from which the parts other than the characters have been removed can be generated by, for example, manually removing the parts other than the characters from the document image 90 containing the non-characters by using image retouching software or the like. .. A plurality of pairs of a document image 90 including such hatching and shading and a first processed image 54 from which parts other than characters such as hatching and shading are removed are created, and the first learning device 51 uses these. The first machine learner 61 is made to learn the first conversion of removing the non-character image from the image in which the character and the non-character image are mixed by using the set of. The image analysis preprocessing unit 21 can remove an image other than characters from the target image by performing the first conversion on the document image 90 using the first machine learning device 61.

By using such a first machine learning device 61, noise such as shading can be removed from the document image 90. If noise that reduces the accuracy of character recognition is known other than shading and hatching, noise is generated from the document image 90 by having the first machine learning device 61 learn image conversion to remove such noise. Can be removed. That is, the image analysis preprocessing unit 21 using the first machine learning device 61 functions as a noise removal engine (AI-noise removal engine) using artificial intelligence.

FIG. 5 is an example of a plan view of a character group arranged in a table without using a ruled line in the present embodiment. FIG. 6 is an example of a plan view of a character group represented by a ruled line in the present embodiment.

Further, the image analysis pre-processing unit 21 previously sets an image of a character group that has been arranged without using the ruled lines as shown in FIG. 5 into a character group that has been arranged with the ruled lines as shown in FIG. We are learning the second conversion to convert a character group to an image. The second post-processed image 56 including the character group represented by the ruled lines as shown in FIG. 6 has a ruled line on the second pre-processed image 55 as shown in FIG. 5, for example, manually by using image retouching software or the like. Can be generated by adding. Alternatively, the image 55 before the second processing can be generated by manually removing the ruled lines from the image 56 after the second processing, for example, by using image retouching software or the like. The second learning device 52 causes the second machine learning device 62 to learn the second conversion for generating the second post-processing image 56 from the second pre-processing image 55. A plurality of pairs of the second pre-processing image 55 of the character group tabulated without using the ruled lines and the second post-processing image 56 tabulated by the ruled lines are created, and the second learning device 52 creates these pairs. The second machine learner 62 is made to learn the second conversion of converting the image of the character group tabulated without using the ruled lines into the image tabulated using the ruled lines. The image analysis preprocessing unit 21 uses the second machine learning device 62 to perform the second conversion on the document image 90, thereby converting the table structure of the target image without the ruled lines into the image of the table structure using the ruled lines. You will be able to do it. Here, the ruled line used for the table is preferably a color different from the color of the characters in the document image 90 (for example, blue).

By using such a second machine learning device 62, the table structure portion included in the atypical document can be regarded as a table structure even when there is no ruled line. As a result, the structuring of the document becomes easier.

FIG. 7 is an example of a plan view of a character group in the present embodiment. FIG. 8 is an example of a plan view surrounded by a frame character by character in the present embodiment.

Further, the image analysis pre-processing unit 21 previously surrounds an image of a character group as shown in FIG. 7 (third pre-processing image 57) with a frame in which each character as shown in FIG. 8 does not overlap with each other. We are learning the third conversion to convert to the image (image 58 after the third processing). The third post-processed image 58, as shown in FIG. 8, in which the characters of the character group in the image are surrounded by a frame that does not overlap with each other, is shown in FIG. 7 manually, for example, by using image retouching software or the like. It can be generated by adding a frame to the third pre-processed image 57 as described above. The third learning device 53 causes the third machine learning device 63 to learn the third conversion for generating the third post-processing image 58 from the third pre-processing image 57. A plurality of sets of the third pre-processed image 57 including the character group and the third post-processed image 58 in which each character of the character group is surrounded by a frame are created, and the third learning device 53 uses these sets. , The third machine learning device 63 is made to learn the third conversion of converting an image including a character group into an image in which each character is surrounded by a frame. The image analysis preprocessing unit 21 can convert each character included in the target image into an image surrounded by a frame by performing a third conversion on the document image 90 using the third machine learning device 63. become. Here, the frame surrounding the characters is preferably a color different from the color of the characters in the document image 90 (for example, red).

In this way, the image analysis preprocessing unit 21 performs preprocessing on the document image 90 stored in the image storage unit 20. The preprocessed image obtained by subjecting the document image 90 to the first conversion, the second conversion, and the third conversion is stored in the preprocessed image storage unit 22. The first conversion, the second conversion and the third conversion are performed in this order, for example. Further, in the area where the inverted character whose background color is opposite to that of the character is formed, such as a white character, a pretreatment for inverting the color may be performed. The ruled lines and frames added in the second conversion and the third conversion may be arranged on a layer different from the document image 90. When the format of the image file has a multi-layer structure, a layer other than the actual document image 90 is created and a ruled line or a frame is arranged. When the format of the image file does not have a multi-layer structure, an image file different from the layer (layer) other than the actual document image 90 is created and a ruled line or a frame is arranged.

Next, the OCR unit 23 performs image processing on the preprocessed image 93 to extract character information (S3). The OCR unit 23 uses an OCR (Optical Character Recognition) engine. The OCR engine uses a machine learning device whose accuracy is improved by learning. As the OCR engine, for example, an engine including at least a part of a convolutional neural network (CNN) is used. The OCR engine extracts character information from the target image using a correspondence relationship that converts the image into characters. The accuracy of character recognition is improved by removing the portion other than the characters by the image analysis preprocessing unit 21. The OCR engine is trained in character recognition by the following method. The learning of the OCR engine is performed by the OCR engine learning device 70.

First, prepare a plurality of images that are correctly recognized as specific characters. Next, an image in which the specific character is expressed using a specific font is formed.

FIG. 9 is an image correctly recognized as a specific character in the present embodiment and an image expressing the character using a specific font.

In FIG. 9, the image correctly recognized as a specific character (first image) is the image on the right side, and the image expressing the character using a specific font (second image) is the image on the left side. ..

Image generation for learning The learning unit 71 converts an image (second image) in which a specific character is expressed using a specific font into an original image (first image) that recognizes the specific character (for learning). Image conversion) is trained by the learning image generator 72. The learning image generator 72 generates a learning image by performing a learning image conversion on an image of some or all characters included in a specific font. As a result, it is possible to generate a learning image having characteristics similar to those of an image correctly recognized as a specific character, such as a blurring method.

FIG. 10 is a learning image in the present embodiment.

In FIG. 10, the left side is an image of characters expressed in a specific font, and the right side is an image obtained by converting the image on the left side using a learning image conversion.

By using the learning image conversion and the character image included in the specific font in this way, it is possible to generate learning data (image) even for characters for which no image data exists. The character recognition learning unit 73 uses the learning data to further train the OCR engine on the correspondence relationship for converting an image into characters, thereby improving the accuracy of character recognition.

When the characters are extracted by the OCR unit 23, the ruled lines added by the second machine learning device 62 and the frame added by the third machine learning device 63 are colored differently from the characters in the image to recognize the characters. There is little risk of degrading the accuracy of. Further, since the third machine learning device 63 adds a frame to each character, the possibility that a part or all of the adjacent characters is recognized as one character is extremely reduced. Therefore, the accuracy of character recognition is improved.

If the ruled line and the frame reduce the accuracy of character recognition, the ruled line and the frame may be deleted and the character may be recognized by deleting a specific color or the like. At this time, the accuracy of character recognition can be improved by virtually recognizing the area delimited by the ruled line and the frame as one block.

By using the second machine learning device 62, the table structure portion included in the atypical document can be regarded as a table structure even when there is no ruled line. As a result, the structuring of the document becomes easier. That is, the image analysis preprocessing unit 21 and the OCR unit 23 using the second machine learning device 62 function as a natural language analysis / structuring engine using artificial intelligence.

By using the third machine learning device 63, the characters included in the document image 90 can be grasped character by character, so that the accuracy of character recognition is improved. That is, the image analysis preprocessing unit 21 and the OCR engine using the third machine learning device 63 function as an OCR character recognition engine (AI-OCR character recognition engine) using artificial intelligence.

The character information extracted by the OCR unit 23 is stored in the extracted character string storage unit 29 together with the position of the character information on the document image 90.

Next, the image analysis post-processing unit 24 performs post-processing on the extracted character information (S4). In the post-processing, for example, the correctness of recognition of the extracted character information is checked by using the dictionary 30.

FIG. 11 is an example of a part of a document image in the present embodiment.

For example, consider a case where the image shown in FIG. 11 is recognized as "daily hospitalization fee" as a result of OCR. The same 7-letter words as the "daily amount of hospitalization fee" are extracted from the dictionary 30 and an exclusive OR (XOR) is taken with each word. More specifically, a column in which each character of "Hospitalization Pickup Daily Amount" is expressed in binary and a column in which each character of a word of the same length contained in the dictionary 30 is expressed in binary. And XOR is calculated. If all seven of the XOR calculation results are 0 columns, the extracted character string is included in the dictionary 30.

On the other hand, for example, when "adult illness hospitalization special contract" exists as a 7-character word included in the dictionary 30, if "hospitalization pick-up daily amount" and XOR are taken, all seven calculation results are in one column. As described above, when 1 is included in the calculation result of XOR, the extracted character string is not included in the dictionary 30.

The XOR calculation is repeated for the character strings of the same length until the words in the column whose XOR calculation result is all 0 appear in the dictionary 30. When the word in the column whose XOR calculation result is all 0 exists in the dictionary 30, it is determined that the extracted character string is correct. When the word in the column whose XOR calculation result is all 0 does not exist in the dictionary 30, it is determined that the extracted character string may be incorrect.

If it is determined that the extracted character string may be incorrect and there is a word in which only one 1 exists in the XOR calculation result, it is possible that the word and only one character were incorrectly recognized as an image. Highly sex. Therefore, the character string extracted with the word as correct is corrected.

In this way, the recognition accuracy of the character information is improved. Here, when the length of the word is short, there is a high possibility that there are a plurality of words that differ by only one character, so that it is difficult to determine the correctness. Therefore, for example, a check may be made only for words having a length of 5 characters or more.

Further, in the post-processing, the morphological analysis of the extracted character information may be performed. By morphological analysis, it is possible to analyze the part of speech of the character string included in the extracted character information. The analysis result is stored together with the character string. Not only the part of speech, but also whether or not it corresponds to a proper noun, whether or not it corresponds to a date, whether or not it corresponds to a quantity, and the like may be analyzed.

Next, the structural analysis unit 25 structures the character information extracted by the OCR unit 23 and corrected by the image analysis post-processing unit 24 as needed (S5). Here, structuring is to group textual information into a group of information, and to specify and express the hierarchical relationship of the information. For example, in structured text information, the title of the form is in the uppermost layer, and the document creator, the document creation date, and the main contents are in the lower layer. Items in each hierarchy may include a plurality of hierarchies. For example, if the title of the form is an invoice, the main content includes the invoice item and the total, and each invoice item includes the product number, unit price, quantity, and invoice amount.

When layering, the key and value may be specified. The key can be specified by searching whether or not the target character string is included in the database that stores the character string that can be the item name. The relative positional relationship of each character string is used for associating the key and the value. For each character string, the character strings located on the right side (right), left side (left), upper (above), and lower (below) of the character string are stored as neighbors (neighbor). The value is one of the character strings of the neighbor.

Further, the value candidate list in which the item name is not described may be stored in advance together with the item name in which the item name is not described, such as "product name". When the extracted character information corresponds to one of the candidate lists, the extracted character information may be used as a value and the corresponding item name may be stored as a key.

When morphological analysis or the like is performed in the post-processing (S4), the characteristics of the character string such as the part of speech may be used for the association. For example, it is possible to link using a rule that only the quantity corresponds to the value for a specific key.

Keys and values do not have to be a simple one-to-one relationship. For example, a plurality of values may be combined for one key. The key may have a hierarchical structure (tree structure) in which a subkey is subordinated to the main key. In this case, the number of layers may be 3 or more.

The ruled line added by the second machine learning device 62 can be used for structuring the document. Therefore, the extracted character information can be easily structured even in a table structure document to which no ruled line is added.

Next, the mapping unit 26 maps the character string extracted on the original document image 90 to the database (S6).

FIG. 12 is a mapping screen in the image analyzer of the present embodiment.

The mapping screen 40 is displayed on the display 13. The mapping screen 40 includes an image display unit 41 and a corresponding display unit 42. The document image 90 is displayed on the image display unit 41. Structured character information is displayed on the corresponding display unit 42.

The extracted character string is displayed as structured character information (structured data). The mapping unit 26 further prompts the user to confirm the correspondence between the key and the value combined by the structural analysis unit 25, and makes it possible to correct any error.

Specifically, first, for example, the document image 90 and the extracted character information are displayed on the image display unit 41 of the display 13. The extracted character information is displayed as structured character information on the corresponding display unit 42 on the left side of the document image 90, for example.

In addition, the part where the character information is extracted is colored. When the pointer is moved to the colored part by operating the mouse and clicked, the character information of the key or value corresponding to that part can be distinguished from others in the part where the key and value pair is displayed. , For example, displayed by changing the color. At this time, on the document image 90, the part where the character information associated with the key or value corresponding to the selected part is read is displayed by changing the color, for example, so that the part can be distinguished from the others. .. If there is no mistake in this correspondence, the user leaves it alone or inputs that there is no mistake. If there is a mistake in this correspondence, the key or value value is corrected in the part where the key and value pair is displayed.

When correcting, if there is an error in character recognition, input it from the keyboard or the like. If there is an error in the correspondence (association), the correspondence may be corrected by clicking on the part where the corresponding key or value is written on the document image 90.

The result of character recognition is obtained, for example, as a plurality of candidates and the certainty of each candidate. That is, as a result of recognizing a certain character image, the certainty of the character of the candidate 1 is expressed as 90%, the certainty of the character of the candidate 2 is expressed as 10%, and the like. When displaying the document image 90 and the extracted character information on the image display unit 41 of the display 13, the characters may be color-coded according to the certainty of character recognition. For example, if the certainty is less than a predetermined threshold such as 90%, it may be represented by red characters. By displaying in this way, it becomes easier for the person who confirms the character recognition (verifier) to check the character recognition. Color coding may be performed on a character-by-character basis or on an extracted word basis. Alternatively, the entire document may be color-coded according to the proportion of characters with certainty that is less than a predetermined threshold. In this case, in a document having a large proportion of characters with low accuracy of character recognition, it may be faster for the operator to manually input the characters without using the result of character recognition by this system.

Further, the mapping unit 26 inputs the structured data modified as necessary into the database using the mapping table. Here, the mapping table is a table (table) showing the correspondence between the logical structure of the document and the structure of the database. Since the logical structure of a document is expressed as a structure of structured data, character information can be easily flowed into a database by using a mapping table.

As described above, in the present embodiment, the error can be reduced by the user correcting the result of character recognition.

The form data digitization system of the present embodiment includes an OCR engine and a machine learning device capable of deep learning in the preprocessing unit. Therefore, the accuracy of character recognition can be improved by advancing the learning.

Further, by learning the correction by the user, the accuracy of character recognition and the accuracy of mapping (association) can be improved. For example, if a specific character is mistakenly recognized as another character and the user corrects it more often, the specific character can be correctly recognized. In addition, if the mapping is incorrect, the user can correct the key and value by correcting the association.

As described above, the paper form data conversion system 10 of the present embodiment has a structural analysis unit 25 for associating the character information with the paired character information based on the position of the character information in the document image 90. There is. Therefore, it is possible to grasp the structure of the document represented by the document image 90, which is the image to be analyzed, that is, the correspondence between the item name and the corresponding value. As a result, the recognition accuracy of the character information can be improved based on the correspondence between the item name and the corresponding value.

Further, in the present embodiment, the mapping unit 26 displays the set of character information associated with the structural analysis unit 25 in association with the document image 90, and when there is an error in the combination of the character information set, the user makes a mistake. Accepts correction input. Therefore, it is possible to more accurately grasp the correspondence relationship between the item name and the corresponding value. Furthermore, by learning the history of corrections by the user, it is possible to improve the accuracy of grasping the correspondence relationship between the item name and the corresponding value.

10 ... Paper form data conversion system, 12 ... Scanner, 13 ... Display, 14 ... Keyboard, 15 ... Mouse, 20 ... Image storage unit, 21 ... Image analysis preprocessing unit, 22 ... Preprocessed image storage unit, 23 ... OCR Unit, 24 ... Image analysis post-processing unit, 25 ... Structural analysis unit, 26 ... Mapping unit, 29 ... Extracted character string storage unit, 30 ... Dictionary, 40 ... Mapping screen, 41 ... Image display unit, 42 ... Corresponding display unit, 51 ... 1st learning device, 52 ... 2nd learning device, 53 ... 3rd learning device, 54 ... 1st processed image, 55 ... 2nd pre-processed image, 56 ... 2nd processed image, 57 ... 3rd processing Pre-image, 58 ... 3rd processed image, 61 ... 1st machine learning device, 62 ... 2nd machine learning device, 63 ... 3rd machine learning device, 70 ... OCR engine learning device, 71 ... Image generation learning unit for learning , 72 ... Image generator for learning, 73 ... Character recognition learning unit, 90 ... Document image, 93 ... Preprocessed image

Claims (5)

In a paper form data conversion system that extracts character information from a scanned target image and outputs it to an external database.
An OCR unit that extracts character information from the target image using the OCR engine,
A structural analysis unit that generates structured data in which the character information extracted by the OCR unit is structured based on the position of the character information on the target image.
A mapping unit that inputs the structured data to the database using a mapping table showing the correspondence between the structured data and the structure of the database, and
A paper form data conversion system characterized by having. An image in which characters and non-character images are mixed using a pair of a first pre-process image in which characters and non-character images are mixed and a first post-process image in which images other than characters in the first pre-process image are removed. A first learning device that causes a first machine learning device to learn a first conversion that removes images other than characters from
Have,
The pretreatment includes the first conversion.
The paper form data conversion system according to claim 1, characterized in that. A character group that is tabulated without using a ruled line using a set of a second post-processed image that includes a character group that is tabulated by a ruled line and a second pre-processed image that has the ruled line of the second post-processed image removed. A second learning device that causes a second machine learner to learn a second conversion that converts an image into an image of a character group represented by ruled lines.
Have,
The pretreatment comprises the second transformation.
The form data digitization system according to claim 1 or 2, characterized in that. Each of the images including the character group is included in the image including the character group by using the pair of the third pre-process image including the character group and the third post-process image in which the frames not overlapping each other are formed in the respective characters included in the third pre-process image. A third learning device that causes a third machine learning device to learn a third transformation that forms a frame that does not overlap with each other.
Have,
The pretreatment includes the third transformation.
The paper form data conversion system according to any one of claims 1 to 3, wherein the paper form data conversion system is characterized. A word database that stores words and
The exclusive logical sum of the first data in which each character of the word is expressed in binary and the second data in which each character of the character information extracted by the OCR unit is expressed in binary is obtained, and the exclusive sum is obtained. Image analysis that determines that the character information extracted by the OCR unit is correct when all of the logical sums are 0, and corrects the character information to the word when one of the exclusive logical sums is 1 and the other is 0. Post-processing unit and
The paper form data conversion system according to any one of claims 1 to 4, wherein the system is characterized by having.

Images