TWM645906U - Document character recognition system - Google Patents

Abstract

This creation provides a document character recognition system, including a document analysis module, a pre-processing module, an OCR module, a post-processing module and an interface module. The document analysis module receives the document image, determines the pre-processing method and post-processing method suitable for the document image, and notifies the pre-processing module and post-processing module respectively. The pre-processing module pre-processes the document image according to the pre-processing method, the OCR module performs text recognition on the pre-processed document image and generates a corresponding document file, and the post-processing module post-processes the document file according to the post-processing method. . The interface module provides a user interface for viewing and editing the contents of the document file.

Description

document character recognition system

This creation is about optical character recognition (OCR).

For document character recognition, existing technical solutions generally use single-data optical character recognition (optical character recognition, OCR) tasks. When you want to identify a specific document, you must manually re-train the model of the target document, or design a corresponding strategy to enhance the recognition effect. Human intervention is also required for pre-processing of different types of target documents. In addition, how to further improve the effect of OCR so that the obtained document files can be better utilized in the future is also a problem to be solved in this field.

In order to solve the problems in the prior art as mentioned above, the present invention provides a document character recognition system, which includes a plurality of hardware circuits electrically connected to each other, and is configured to be configured into a plurality of modules, and the modules It includes: a document analysis module, a pre-processing module, an OCR module, a post-processing module and an interface module, wherein: the document analysis module receives a document image and determines a suitable document based on a reinforcement learning model. A pre-processing method and a post-processing method for the document image, and the pre-processing module and the post-processing module are notified respectively; the pre-processing module performs pre-processing on the document image according to the pre-processing method; the OCR module The group performs text recognition on the pre-processed document image and generates a document file accordingly; the post-processing module performs post-processing on the document file according to the post-processing method; the interface module provides a user interface for View and edit the contents of this document file.

According to some embodiments of the present invention, the user interface allows a user to provide feedback on the processing of the pre-processing module, the OCR module or the post-processing module.

In some embodiments, feedback on the processing of the pre-processing module or the post-processing module is provided to the document analysis module, and the feedback is used to train the reinforcement learning model.

According to some specific embodiments of the present invention, the model used by the OCR module for text recognition includes an instance-based learning model. In some embodiments, the user interface allows a user to provide feedback on the OCR module's processing. In some embodiments, feedback on the OCR module's processing is provided to the OCR module, and the feedback is used to train the instance-based learning model.

According to some specific embodiments of the present invention, the interface module provides feedback to the pre-processing module, the OCR module or the post-processing module according to a user's editing of the content.

According to some embodiments of the present invention, the interface module provides feedback to the OCR module based on a user's editing of the content, and the feedback is used to train the instance-based learning model.

Other purposes and advantages of the present invention are partly described in the following description, or may be understood through embodiments of the present invention. It should be understood that the foregoing creative content and the following implementation methods are only illustrative and explanatory descriptions, and do not limit the creative content as in the scope of the patent application.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this work belongs.

The words "one" and "B" used in this article, unless otherwise specified, refer to the quantity of at least one (one or more than one).

This creation provides a document character recognition system, which includes: A file analysis module, A pre-processing module, An optical character recognition (OCR) module, a post-processing module, and An interface module, in: The document analysis module receives a document image, determines a pre-processing method and a post-processing method suitable for the document image based on a reinforcement learning model, and notifies the pre-processing module and the post-processing module respectively. ; The pre-processing module pre-processes the document image according to the pre-processing method; The OCR module performs text recognition on the pre-processed document image and generates a corresponding document file; The post-processing module performs post-processing on the document file according to the post-processing method; and The interface module provides a user interface for viewing and editing the contents of the document file.

The document character recognition system of this creation can include five signal-connected modules: document analysis module, pre-processing module, OCR module, post-processing module and interface module. The five modules can be implemented only by hardware circuits, or by software combined with hardware circuits.

According to some specific embodiments of this invention, the document analysis module, pre-processing module, OCR module, post-processing module and interface module can be set in a server.

In some specific embodiments, the document character recognition system of the present invention includes a plurality of hardware circuits electrically connected to each other, which are configured to be configured into multiple modules, and the modules include files as described above Analysis module, pre-processing module, OCR module, post-processing module and interface module.

The user interface provided by the interface module can be displayed through the display unit of an electronic device held by the user.

"Electronic devices" as mentioned in this article include, but are not limited to, desktop computers, laptops, tablets, or smartphones.

For example, one or more of the above-mentioned document analysis module, pre-processing module, OCR module, post-processing module and interface module can be implemented by a software program, which is implemented by a server or an electronic device. Executed by a processing unit (e.g., processor). The "software program" includes but is not limited to mobile application (App).

The "pre-processing" mentioned in this article refers to image processing of the input image, including but not limited to tilt correction, contrast brightness adjustment, stamp removal such as seals or watermarks, cropping of specific image areas, light and shadow, etc. Noise reduction...etc.

The "post-processing" mentioned in this article mainly refers to the corresponding processing of the text recognized from the image and related information (for example, the position information of the text line, the attribute category of the text line, etc.), including but not limited to alphanumeric correction , Amount digital verification, recognition confidence level prompts, field table analysis and detection, recognition result formatting and document image classification...etc. Preferably, post-processing can enable the output results to have formatted information to facilitate subsequent application.

In file analysis, if we adopt a simple classification method, there will be problems with category restrictions. That is, if we want to define a brand new file, we will need to retrain the analysis model, and we will also encounter data The question of quantity.

The document analysis of this creation uses a self-supervised method to distinguish document types through clustering through metadata and image features (extracted with convolutional or transformer based models). Although this method does not define the file as a clear type, it can obtain richer information from the image data type of the input file for use in downstream tasks. The documents may be financial statements or official documents, but are not limited to this.

On the other hand, the document analysis module of this creation uses a machine learning model of reinforcement learning to determine or recommend appropriate pre-processing methods and post-processing methods based on the input document images. That is, use the existing document image data and the pre- and post-processing methods as the basis, use the pre/post-processing methods (or methods) as actions, and treat documents in each specific field as environmental information (environment) The information of its files is set as a state, and then the agent model (agent) automatically selects any processing mode. The final evaluation of the recognition efficiency of the processed files is regarded as a reward score (reward). In the entire architecture, through multiple iterations, the agent model (agent) will learn which specific document image (state) should have what combination of pre/post processing (action) to achieve better recognition efficiency. (reward), in this way to achieve an adaptive combination of pre- and post-processing for specific files.

In addition, the user interface may allow the user to provide feedback for pre/post processing, and the feedback received may be used to further train the reinforcement learning model.

In the OCR processing part, this creation is not limited to a specific OCR model. The model only needs to be able to detect the text frame and recognize the corresponding text. Of course, the efficiency of identification will depend on the stability and accuracy of the model used.

According to some specific embodiments of this invention, the model used by the OCR module for text recognition includes an instance-based learning model (or memory-based learning model), and preferably, the user's target The feedback provided by the OCR process is used for training the instance-based learning model.

In some specific embodiments, the text box information fed back by the user (including the center position of the text box and the corresponding length and width information) and the text recognition content fed back are used to train an instance-based machine learning model, and use A method of knowledge distillation. When the OCR model performs feedback updates, it is necessary to consider that the model will have catastrophic forgetting. Therefore, instance-based learning and knowledge distillation are to ensure that previous learning information can be continued to subsequent models. First, instance-based learning provides data protection. For example, by performing categorical multivariate analysis (such as Gaussian mixture model) on historical data, representative data is selected and stored in memory. , so that the model can use previous data to stably update the model when updating based on feedback. On the other hand, the method of knowledge distillation can continue the knowledge learned by the previous model (that is, the parameter information in the model), and the design of the loss function can be used to mix the prediction information of the previous model into the new model prediction situation. , allowing it to bring the parameter information of the previous model into the new model.

In addition, if the pre/post-processing module adjustments cannot converge, the "image conversion and generation" function in the OCR module can be triggered to perform data augmentation on the image to assist the entire feedback training process. For the conversion and generation of document images, it is different from the generally discussed generation of generating more data. The method of this creation is to allow the generative model (for example, a generative adversarial network-based model or diffusion-based model) to learn the current The image features and patterns that can be recognized by the model. When encountering the same recognized text but printed or written in different ways, resulting in a decrease in recognition rate, this method can be used to regenerate the image to be recognized, while retaining the existing The text information is used to synthesize and enhance the text image to generate a text image that allows the model to improve the recognition rate.

A specific embodiment of the document character recognition system of this invention will now be described with reference to FIG. 1 as follows.

As shown in FIG. 1 , the document character recognition system 100 includes a document analysis module 110 , a pre-processing module 120 , an OCR module 130 , a post-processing module 140 and an interface module 150 . Among them, the document analysis module 110 is connected to the pre-processing module 120 and the post-processing module 140 respectively (electrical connection or signal connection), and the OCR module 130 is also connected to the pre-processing module 120 and the post-processing module 140 respectively ( electrical connection or signal connection), the interface module 150 is connected to the pre-processing module 120, the OCR module 130 and the post-processing module 140 respectively (electrical connection or signal connection).

After receiving the document image (for example, a scanned document), the document analysis module 110 will determine the pre-processing method and post-processing method suitable for the document image based on a reinforcement learning model, and notify the pre-processing module 120 and the post-processing module respectively. Group 140.

The pre-processing module 120 pre-processes the document image according to the pre-processing method, and the pre-processed document image is sent to the OCR module 130 for text recognition and a corresponding document file is generated.

Then, the post-processing module 140 performs post-processing on the file according to the post-processing method. Finally, the interface module 150 can provide a user interface through an electronic device for the user to view and edit the content of the document file.

In addition, the interface module 150 may allow the user to provide feedback on the processing of the pre-processing module 120, the OCR module 130 and/or the post-processing module 140. The feedback can be used to train the reinforcement learning model of the document analysis module 110 or to train the machine learning model of the OCR module 130 .

100: Document character recognition system

110:File analysis module

120: Pre-processing module

130:OCR module

140:Post-processing module

150:Interface module

Figure 1 is a structural diagram of a specific embodiment of the document character recognition system of the present invention.

100: Document character recognition system

110:File analysis module

120: Pre-processing module

130:OCR module

140:Post-processing module

150:Interface module

Claims (8)

A document character recognition system, including: a document analysis module; a pre-processing module connected to the document analysis module; an OCR module connected to the pre-processing module; a post-processing module connected to the document analysis module and the OCR module; and an interface module connected to the pre-processing module, the OCR module and the post-processing module; wherein: the document analysis module receives a document image based on a reinforcement learning model Determine a pre-processing method and a post-processing method suitable for the document image, and notify the pre-processing module and the post-processing module respectively; the pre-processing module pre-processes the document image according to the pre-processing method; the The OCR module performs text recognition on the pre-processed document image and generates a corresponding document file; the post-processing module performs post-processing on the document file according to the post-processing method; the interface module provides a user interface. Used to view and edit the contents of the document file. The document character recognition system of claim 1, wherein the user interface allows a user to provide feedback on the processing of the pre-processing module, the OCR module or the post-processing module. The document character recognition system as described in claim 2, wherein feedback on the processing of the pre-processing module or the post-processing module is provided to the document analysis module, and the feedback is used to train the reinforcement learning model. The document character recognition system of claim 1, wherein the model used by the OCR module for character recognition includes an instance-based learning model. The document character recognition system of claim 4, wherein the user interface allows a user to provide feedback on the processing of the OCR module. The document character recognition system of claim 5, wherein feedback on processing by the OCR module is provided to the OCR module, and the feedback is used to train the instance-based learning model. The document character recognition system as described in claim 1, wherein the interface module provides feedback to the pre-processing module, the OCR module or the post-processing module based on a user's editing of the content. The document character recognition system of claim 4, wherein the interface module provides feedback to the OCR module based on a user's editing of the content, and the feedback is used to train the instance-based learning model.

Images