KR102561878B1 - Ai blue ocr reading system and method based on machine learning - Google Patents

Abstract

The machine learning-based AI blue OCR reading system according to the present invention receives various types of documents, analyzes the text of the document and the shape of each form of the box using a learning model learned based on machine learning, and converts the input image document an OCR device that reads a plurality of pages at once; and an OCR server that receives and merges learning models from the plurality of OCR devices to generate a learning model with improved document recognition rate and distributes the learning model to the OCR device; to read multiple pages of image documents at once. It has the effect of being able to read documents quickly.

Description

AI BLUE OCR READING SYSTEM AND METHOD BASED ON MACHINE LEARNING}

The present invention relates to a machine learning-based AI blue OCR reading system, and more particularly, to a machine learning-based AI blue OCR reading system in which various document data are input, converted into a database, and the recognition rate for image documents that are subsequently input is improved. It's about the system.

A conventional optical character reader selects a corresponding document to recognize a specific document and then reads the document.

For example, in order to recognize a business registration certificate or a tax invoice, a conventional optical character reading device performs reading as a document of the corresponding form is input only when a user selects a form (form) to be recognized.

That is, the conventional optical character reading device has a cumbersome problem in that a user must select which document to input before reading a specific document.

In addition, since the conventional optical character reading device reads and recognizes only one image, it is not easy to read and recognize a large-capacity image document, and it takes a considerable amount of time to read and recognize a large-capacity image document.

Republic of Korea Patent Registration No. 10-2013329 (2019.08.16)

In order to solve the above problems, the present invention is able to perfectly extract the data necessary for work by inputting various document data, converting it into a database, and applying machine learning to train an AI model to improve the accuracy of data recognition. The purpose is to provide a machine learning-based AI blue OCR reading system that can

In addition, the present invention is based on machine learning that can read and recognize large-capacity image documents consisting of a plurality of pages used in public institutions in order to solve the above problems by reducing them by 1/2 or 1/3 compared to the original. The purpose is to provide an AI blue OCR reading system of

In particular, the purpose of the present invention is to provide a machine learning-based AI blue OCR reading system that can increase the character recognition rate by recognizing characters in an input image, correcting the recognized characters again, and then learning the characters in the image. there is

In order to achieve the above object, the machine learning-based AI blue OCR reading system according to the present invention receives various forms of documents and uses a learning model learned based on machine learning to form the text of the document and the form of each form. an OCR device that analyzes and reads the input image document by a plurality of pages at once; and an OCR server that receives and merges the learning models from the plurality of OCR devices to generate a learning model with an improved document recognition rate, and distributes the learning model to the OCR device.

In order to achieve the above object, the OCR device of the machine learning-based AI blue OCR reading system according to the present invention includes a document input unit into which a plurality of page image documents are input; a main controller for receiving and storing various documents input through the document input unit and controlling various operations; a learning unit that, when the subject control unit selects an object requiring additional learning, brings a character image of the selected document and performs learning; a display unit that uploads the document on one screen and simultaneously checks a document recognition list, an original document, and extracted data while displaying the work to be possible; a character correction unit that checks the recognized and extracted character images and then corrects the incorrectly recognized characters; a communication unit enabling transmission and reception of data through communication with the OCR server; and a database unit for storing and managing documents input through the document input unit.

In order to achieve the above object, the main control unit of the machine learning-based AI blue OCR reading system according to the present invention reduces the image document input through the document input unit at a predetermined ratio and reads a plurality of pages at once. to be

In order to achieve the above object, the main control unit of the machine learning-based AI Blue OCR reading system according to the present invention reduces the document at a reduction ratio sufficient to recognize only the title of the document and classifies the document type. to be characterized

In order to achieve the above object, the main control unit of the machine learning-based AI blue OCR reading system according to the present invention stores the character string recognized in the largest area among each character area of the document as the document name of the page. do.

In order to achieve the above object, the main control unit of the machine learning-based AI Blue OCR reading system according to the present invention is the coordinate information of the intersection formed by the intersection of rows and columns in the table in the document, and the horizontal and vertical lengths of the cells. It is characterized in that the document type is classified by recognizing a specific format by calculating or calculating the width of a column.

In order to achieve the above object, the main control unit of the machine learning-based AI blue OCR reading system according to the present invention is characterized in that a specific word is a word that does not exist in the dictionary or a part that does not fit the context is selected as a subject requiring additional learning. to be

As another embodiment, in order to achieve the above object, a machine learning-based AI blue OCR reading method according to the present invention includes (a) a machine learning-based AI blue OCR device receiving an image document through a document input unit; (b) storing the uploaded file by the main control unit of the OCR device in a database unit; (c) executing the learned learning model by the main controller; (d) loading, by the main controller, a plurality of pages to read the image document; (e) analyzing, by the main controller, the text area in the image document by boxing the text area; (f) executing the learning model by the main control unit to determine whether additional learning is required in the process of analyzing a character area in an image file; and (h) if the main control unit does not require additional learning in the step (f), reading the document for which the character area analysis has been completed in the step (e) by the main control unit.

In order to achieve the above object, the machine learning-based AI blue OCR reading method according to the present invention (g) when additional learning is required in the step (f), the learning unit controls the main control unit to select a file requiring additional learning. It is characterized in that it further comprises; the step of receiving the transmission and proceeding with additional learning.

In order to achieve the above object, step (h) of the machine learning-based AI blue OCR reading method according to the present invention includes (h-1) checking whether the main control unit has learning data learned by the learning unit; (h-2) if there is the learning data in step (h-1), the main control unit reflects the corresponding learning data to document reading; (h-3) if there is no learning data added in the step (h-1), recognizing a character in the image document for which the analysis of the character area has been completed by the main control unit; (h-4) recognizing the title of the document by the main control unit; (h-5) determining, by the main control unit, the document type based on the title of the document recognized in step (h-4).

In order to achieve the above object, in step (d) of the machine learning-based AI blue OCR reading method according to the present invention, when the main control unit loads a plurality of pages to read the image document, the corresponding image document is selected at a predetermined level. It is characterized by a reduction in proportion.

In order to achieve the above object, in the machine learning-based AI blue OCR reading method according to the present invention, the main control unit determines the horizontal and vertical length, width, and arrangement of a plurality of specific cells in the document in step (h-5). It is characterized in that it is possible to recognize and classify the form of the document by identifying the location of the document.

The machine learning-based AI blue OCR reading system according to the present invention can read multiple pages of image documents at once by reducing image documents at a predetermined reduction ratio, thereby enabling rapid reading of documents.

The machine learning-based AI blue OCR reading system according to the present invention has the effect of improving data recognition accuracy by learning an AI model by applying machine learning to data directly extracted by a user.

1 is a block diagram of a machine learning-based AI blue OCR reading system according to the present invention.
2 is a block diagram of an OCR device of a machine learning-based AI blue OCR reading system according to the present invention.
3 is a diagram for explaining character correction of the AI blue OCR reading system based on machine learning according to the present invention.
4 is a diagram showing examples of various tables and graphs displayed on the display unit of the machine learning-based AI blue OCR reading system according to the present invention.
5 is a flowchart of a machine learning-based AI blue OCR reading method according to the present invention.
6 is a diagram for explaining the step of uploading a file in the machine learning-based AI blue OCR reading method according to the present invention.
7 is a diagram for explaining character area analysis by the AI blue OCR reading system based on machine learning according to the present invention.
8 is a flowchart of a document reading step in a machine learning-based AI blue OCR reading method according to the present invention.

The terms or words used in this specification and claims should not be construed as being limited to ordinary or dictionary meanings, and the inventors may appropriately define the concept of terms in order to explain their invention in the best way. Based on the principle that there is, it should be interpreted as meaning and concept consistent with the technical spirit of the present invention.

Therefore, since the embodiments described in this specification and the configurations shown in the drawings are only one of the most preferred embodiments of the present invention and do not represent all of the technical ideas of the present invention, they can be substituted at the time of this application It should be understood that there may be many equivalents and variations.

Hereinafter, a machine learning-based AI blue OCR reading system according to the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram of a machine learning-based AI blue OCR reading system according to the present invention.

As shown in FIG. 1, the machine learning-based AI blue OCR reading system according to the present invention includes an OCR device 100 and an OCR server 200 connected to a plurality of OCR devices.

As shown in FIG. 2, the OCR device 100 includes a document input unit 110, a main control unit 120, a learning unit 130, a display unit 140, a character correction unit 150, a communication unit 160, data A base portion 170 is included.

The OCR server 200 is connected to the plurality of OCR devices 100, receives learning models generated through learning from the OCR devices 100, and comprehensively merges the plurality of learning models to form a learning model with a high recognition rate. generate

As described above, the OCR server 200 generates a learning module with a high recognition rate and distributes it to the newly installed OCR device 100 as well as the existing OCR device 100 to improve the performance of the corresponding OCR device 100. .

The document input unit 110 is a means for inputting an image document or a PDF document consisting of a plurality of pages into a machine learning-based AI blue OCR reading system in a scanning manner.

The image document may be input through the input unit 100, but may be an image document created and converted in Korean, Word, or PDF.

In addition, the document input unit 110 may receive an already imaged document from a USB memory through a USB port.

In particular, the image document recognizes a public institution's report certificate, confirmation certificate, collection/transport certificate, business registration certificate, real estate register, family relation certificate, seal certificate, etc. Real estate appraisal report, resident registration card, driver's license, alien registration card, recognition of various qualifications, equipment technical data/drawings from manufacturers and distributors, inspection receipts, receipt and release statements, inspection logs, equipment repair logs, orders, transaction statements, etc. It can be a document with content included in the form.

The main controller 120 receives, recognizes, and stores various documents input through the document input unit 100 in the database unit 160 .

The main control unit 120 stores various documents input through the document input unit 110 in the database unit 160 and at the same time manages the machine learning-based AI blue OCR device according to the present invention. The OCR server ( 100) can also be stored.

That is, the main controller 120 communicates with the OCR server 100 through the communication unit 160 and transmits various documents input through the document input unit 110 so that they can be stored in the corresponding OCR server 100. do.

The main control unit 120 recognizes characters in a file loaded from the database unit 170 through the EasyOCR program, which is a learning model already learned by the learning unit 130, or input through the document input unit 110. and extract

The main controller 120 searches for recognized character images and corresponding characters, performs boxing processing, and stores each character image and corresponding characters in the database 180 .

At this time, as shown in FIG. 3A, the character correction unit 150 confirms the recognized and extracted image and characters, corrects and stores the characters for the erroneously recognized characters.

In particular, the main controller 120 automatically recognizes and extracts tables of various formats within an image, and recognizes and extracts tables with coordinate information about intersections formed by the intersection of rows and columns in a table within an image. can do.

In particular, the main control unit 120 may classify image documents by calculating the length and width of a cell using the coordinate information of the intersection point or recognizing a specific format by calculating the width of a cell.

In addition, the main control unit 120 may recognize and classify the form of a specific document in consideration of the horizontal and vertical lengths, widths, and arrangement positions of the plurality of specific cells.

Preferably, the main control unit 120 forms a lattice of rectangular image documents with n×m of the same width and determines the intersection point of two diagonal lines as a center to obtain coordinate information of the intersection point.

Then, as shown in FIG. 3B, the main control unit 120 selects a subject requiring additional learning from among the recognized and extracted letters.

At this time, it is preferable that the subject word unit 120 selects a specific word as a subject requiring additional learning for a word that does not exist in a DBized Korean dictionary or an English dictionary, or that does not make sense in context.

When the main control unit 120 selects an object requiring additional learning, the learning unit 130 brings the character image and character information of the learning target data from the database unit 170 .

At this time, it is preferable that the learning unit 130 performs learning by bringing both new learning subjects and existing learning subjects.

In addition, at this time, the learning unit 130 arbitrarily self-generates a character image for learning when the data size of an object that requires additional learning selected by the main control unit 120 is too small to satisfy the minimum size required for learning. It is desirable to use it for learning.

The learning unit 130 creates a learning dataset for quick learning, and performs letter recognition learning with the learning dataset, but uses a pre-learning model provided by easyocr during learning.

The learning unit 130 stores the learned learning model in the learning model folder of the database 180 so that it can be used for recognizing and extracting new documents.

In the learning unit 130, core learning logic such as image conversion and OCR is implemented in Python, so that it can be easily changed to other modules according to the user's needs and can be easily upgraded.

The learning unit 130 creates data by comparing the recognized text and the result of the text confirmed by a human after OCR recognition with the first basic model.

As shown in FIG. 4, the display unit 140 provides and visualizes the number of processed data and a graph.

In addition, the display unit 140 has a function implemented to upload a file on one screen and work while simultaneously checking a document recognition list, an original document, and extracted data.

On the other hand, the communication unit 160 learns the learning learned by the learning unit 130 to share the learning model learned by the learning unit 130 with other machine learning-based AI blue OCR devices according to the present invention. The model is transmitted to the OCR server 200.

The database unit 170 may be updated to the latest version by receiving a learning model transmitted from a plurality of machine learning-based AI blue OCR devices.

That is, another machine learning-based AI blue OCR device according to the present invention re-recognizes the learning target image as the latest learning model and updates the database unit 170 with the result, while simultaneously updating the OCR server 200. ) to update.

A reading method by the machine learning-based AI blue OCR reading device according to the present invention having the above configuration will be described in more detail with reference to FIG. 5 .

First, as shown in FIG. 5, the machine learning-based AI blue OCR device 100 according to the present invention performs the step of receiving and uploading an image document or PDF document through the document input unit 110 according to the user's manipulation. Do (S100).

That is, as shown in FIG. 6A, the OCR device 100 uploads a new file, registers it in the TB_OCR_DOC_MST and TB_OCR_FILE tables as an upload state, and calls aiblue_ocr.py asynchronously with a specific key value among the registered information as a parameter. can do.

The OCR device 100 may re-query TB_OCR_DOC_MST in a state currently being uploaded on the screen.

The OCR device 100 can search information such as the location or type of the uploaded file with the key value. In the case of a PDF file, it can create an image file for each page with a key value and check the number of pages.

The main control unit 120 of the OCR device 100 performs a step of storing the uploaded file in the database unit 170 (S200).

In the step S200, the main control unit 120 stores the uploaded file in the database unit 170 and transmits the uploaded file to the OCR server 200 to be stored.

The main control unit 120 performs a step of executing the already learned learning model (S300).

After executing the learning model, the main control unit 120 performs a loading step for reading the PDF image file uploaded in step S100 (S400).

In the step S400, it is preferable to reduce at least two or more pages of a PDF image file composed of a plurality of pages at a predetermined ratio so that the plurality of pages can be read at once.

As shown in FIG. 7A , the main controller 120 performs a step of analyzing the text area in the image file by box-processing the text area in the image document (S500).

In the step S400, the main controller 120 determines the x coordinate of the lower left corner of the text area box, the y coordinate of the lower left corner of the text area box, the horizontal length of the text area box, and the text area box as shown in FIG. 7B. The character area in the image is analyzed through the vertical length of the image, the recognized character, and the accuracy of the recognized character.

That is, the main control unit 120 determines the length of a word or sentence by the horizontal length of the box, determines the size of a character by the horizontal length and vertical length, and determines the x-coordinate of the lower left corner of the text area box and the text area box. The position of the character can be determined based on the y-coordinate of the lower left corner of the .

At this time, the main controller 120 forms a lattice at equal intervals in the horizontal and vertical directions based on the center where the diagonal lines of the image page meet, and measures the x-coordinate of the lower left corner of the text area box and the left left corner of the text area box. You can determine the y-coordinate of the bottom edge.

The main control unit 120 may classify documents by grasping the number, position, and width of columns in the document form in the manner described above.

The main controller 120 creates an image file corresponding to each character area and stores the path in TEXT_IMAGE_PATH.

In addition, the main controller 120 classifies the data into different colors and stores them for each page, as shown in FIG. 7A.

At this time, it is preferable that the main control unit 120 stores the character string recognized in the largest area of each character area as the document name of the corresponding page.

That is, the main control unit 120 stores a character string recognized in the largest area for each page as a document name for a PDF image file composed of a plurality of pages.

In the step S500, the main controller 120 executes the learning model and determines whether additional learning is necessary in the process of analyzing the character area in the image file (S600).

In step S600, the main control unit 120 determines that a specific word is a word that does not exist in the dictionary, or that a structure in which special characters, numbers, parentheses, etc. are included in a sentence requires additional learning for the part that does not make sense in context it is desirable

If additional learning is required in step S600, the learning unit 130 receives a file requiring additional learning from the main control unit 120 and performs additional learning (S700).

In the step S700, the learning unit 130 may receive new words or sentence structures that may be included in a business name through an administrator terminal, collect and learn from a portal site connected through Internet communication, and update a learning model. .

As described above, through additional learning, the OCR device 100 of the machine learning-based AI blue OCR reading system according to the present invention can improve reading accuracy for image documents by learning new sentence structures and new words.

On the other hand, if additional learning is unnecessary in step S600, the main control unit 120 performs a step of reading the document for which character area analysis has been completed in step S500 (S800).

In the step S800, the main control unit 120 preferably proceeds with reading the document by reflecting the learning data when there is additional learning in the step S700.

[0080] [0051] Referring to the step S800 in more detail with reference to FIG. 8, the main control unit 120 performs a step of checking whether there is learning data learned by the learning unit 130 (S810).

If there is learning data added in step S810, the main controller 120 performs a step of reflecting the learning data to document reading (S820).

If there is no training data added in the step S810, the main control unit 120 immediately performs a step of recognizing a character in an image document for which analysis of the character area has been completed (S830).

Then, the main control unit 120 performs a step of recognizing the title of the document (S840).

At this time, it is preferable that the main control unit 120 reduces the original image file, excludes the title part, and recognizes only the title by crushing other characters.

In addition, the main control unit 120 finds a reduction ratio sufficient to recognize only the title and recognizes only the title.

The main controller 120 determines the document type based on the title of the document recognized in step S840 (S850).

The display unit 140 displays the document reading result under the control of the main control unit 120, and at the same time, the main control unit 120 performs a step of storing the document reading result in the database unit 170 ( S900).

In the above, the technical idea of the present invention has been described together with the accompanying drawings, but this is an illustrative example of a preferred embodiment of the present invention, but does not limit the present invention. In addition, it is obvious that various modifications and imitations can be made by anyone having ordinary knowledge in the technical field to which the present invention belongs without departing from the scope of the technical idea of the present invention.

100: OCR device
110: document input unit
120: main fisherman
130: learning unit
140: display unit
150: character correction unit
160: communication department
170: database unit
200: OCR server

Claims (12)

An OCR device that receives documents of various formats, analyzes the shape of the text of the document and the form of each form using a learning model learned based on machine learning, and reads the input image document by a plurality of pages at once; and
An OCR server for generating a learning model with an improved document recognition rate by receiving and merging learning models from a plurality of OCR devices, and distributing the learning model to the OCR device;
The OCR device
a document input unit into which image documents of multiple pages are input;
a main controller for receiving and storing various documents input through the document input unit and controlling various operations;
a learning unit that, when the subject control unit selects an object requiring additional learning, brings a character image of the selected document and performs learning;
a display unit that uploads the document on one screen and simultaneously checks the document recognition list, the original document, and the extracted data and displays them so that work is possible;
a character correction unit that checks the recognized and extracted character images and then corrects the incorrectly recognized characters;
a communication unit enabling transmission and reception of data through communication with the OCR server; and
A machine learning-based AI blue OCR reading system comprising a; database unit for storing and managing documents input through the document input unit.
delete According to claim 1,
The main control part
A machine learning-based AI blue OCR reading system, characterized in that the image document input through the document input unit is reduced at a predetermined ratio and read a plurality of pages at a time.
According to claim 1,
The main control part
A machine learning-based AI blue OCR reading system, characterized in that for classifying the type of the document by reducing the document at a reduction ratio sufficient to recognize only the title of the document.
According to claim 1,
The main control part
A machine learning-based AI blue OCR reading system, characterized in that for storing the character string recognized in the largest area of each character area of the document as the document name of the page.
According to claim 1,
The main control part
Characterized in that the type of document is classified by recognizing a specific format by calculating the length and width of a cell or calculating the width of a cell with coordinate information of an intersection point formed as rows and columns intersect in a table in the document. A machine learning-based AI blue OCR reading system.
According to claim 1,
The main control part
A machine learning-based AI blue OCR reading system that selects specific words that do not exist in the dictionary or that do not fit the context as subjects requiring additional learning.
(a) the machine learning-based AI blue OCR device receiving an image document through a document input unit;
(b) storing the uploaded file by the main control unit of the OCR device in a database unit;
(c) executing the learned learning model by the main controller;
(d) loading, by the main controller, a plurality of pages to read the image document;
(e) analyzing, by the main controller, the text area in the image document by boxing the text area;
(f) determining whether additional learning is necessary in the process of analyzing a character area in an image file by executing the learning model by the main controller;
(g) if additional learning is required in step (f), the learning unit receives the file requiring additional learning under the control of the main control unit and proceeds with additional learning; and
(h) if the main control unit does not require additional learning in the step (f), the main control unit reading the document for which the character area analysis has been completed in the step (e); Blue OCR reading method.
delete According to claim 8,
The step (h) is
(h-1) checking whether the main control unit has learning data learned by the learning unit;
(h-2) if there is the learning data in step (h-1), the main control unit reflects the corresponding learning data to document reading;
(h-3) if there is no training data added in the step (h-1), recognizing a character in the image document for which the analysis of the character area has been completed by the main control unit;
(h-4) recognizing the title of the document by the main control unit;
(h-5) determining, by the main control unit, the document type based on the title of the document recognized in step (h-4);
According to claim 8,
In step (d),
The AI blue OCR reading method based on machine learning, characterized in that when the main controller loads the image document by a plurality of pages to read the image document, the corresponding image document is reduced by a predetermined ratio.
According to claim 10,
Machine learning, characterized in that the main control unit can recognize and classify the form of the document by identifying the length, width, and arranged position of a plurality of specific columns in the document in the step (h-5) based AI blue OCR reading method.