TW202319961A - Building method for key-item detection model, identification system and method for business-oriented key-item especially for identifying a plurality of document image files according to business sub-models - Google Patents

Abstract

A building method for key-item detection model is disclosed, comprising the following steps: receiving a plurality of document image files for training uses; receiving marks for a plurality of business types for each document image file in order to form business mark files corresponding to the document image files, respectively; and inputting the document image files and the business mark files into a neural network system for training, forming a business sub-model for each business type training; and finally forming an key-item detection model including a plurality of business sub-models.

Description

Key-item detection model building method, business-oriented key-value identification system and method

The invention relates to a document identification method, in particular to a method for identifying document images according to business types.

Optical Character Recognition (OCR) technology can analyze and recognize document images. It mainly includes two steps: "text detection" and "text recognition". Among them, "text area detection" is to detect the specified page or the entire document, and "text recognition" is to perform character segmentation and character recognition on the detected text area.

Referring to Figure 1, when the OCR application is used to identify form documents, it will first detect the text area (such as the thick frame in the figure), and then recognize the text in it. In the prior art, some applications can judge items according to preset rules (such as the distance between boxes), such as "financial institution name", "bank code" and "financial institution deposit account number", and obtain them according to the predetermined relationship Corresponding information, such as judging whether there is a corresponding text or number under each item, and if so, identifying it. Finally, the identification result is output with a preset rule (such as "item"+":"+"number"), such as "bank code: 456", "financial institution deposit account number: 78910123456789".

Therefore, the object of the present invention is to provide a method for establishing an important item detection model, which includes the following steps: receiving a plurality of document image files for training; The business tag files corresponding to the document image files, and inputting the file image files and the business tag files into a neural network system for training, training for each business type to form a business sub-model; finally forming a plurality of business sub-models. A key item of the model is the detection model.

In some implementation aspects of the method for establishing the key item detection model of the present invention, the step of receiving the mark is to create a business mark file folder for each business type, and each business mark file folder stores the business mark files.

In some implementation aspects of the method for building an important item detection model of the present invention, the step of receiving a mark is achieved by performing the following operations on each of the document image files: recording a key mark name of an important item and selecting a key frame of an important item Mark the key, record a key value mark name and check a key value box to mark the key value, and record a bounding box name and check a bounding box covering the key key box and the key value box.

In some implementation aspects of the key item detection model building method of the present invention, each of the business tag files records at least one set of key item key tag names and coordinate data of key item key boxes, key item value tag names and coordinate data of key item value boxes , and the name of the bounding box and the coordinate data of the bounding box covering the key box and value box of the key item.

Another object of the present invention is to provide a business-oriented key-value identification system, including a processor, a computer-readable medium electrically connected to the processor, and the key-item detection model established by the aforementioned method, the key-item detection The detection model is used to detect at least one important item key and its corresponding important item value according to business requirements for an input target document image file.

In some implementation aspects of the business-oriented key-value identification system of the present invention, the business-oriented key-value identification system further includes an optical character recognition model that receives characters falling within the same bounding box from the key-item detection model The image in the key box of the key item and the image in the value box of the key item are output after character recognition.

In some implementation aspects of the business-oriented key-value identification system of the present invention, the computer-readable medium stores the key-item detection model.

Another object of the present invention is to provide a method for identifying key values of business-oriented key items. The method includes: receiving a target document image file; receiving an option input of a business requirement; applying a sub-model corresponding to the business requirement among a plurality of sub-models in an important item detection model according to the business requirement, and the sub-model has been pre-according to The business requirement completes the training; the sub-model detects the target document image file, and detects at least one important item key image, its corresponding important item value image, and a detection bounding box from the target document image file; And judging whether the important item key image and important item value image fall within the detection bounding box, if so, bring the important item key image and important item value image into an optical character recognition module to obtain a recognition result and output it.

In some implementation aspects of the business-oriented important item key value identification method of the present invention, the step of detecting the target document image file by the sub-model is to detect the key item key image in the target document image file and give a surrounding A detection key frame around the key image, a key value image is detected and a frame value frame around the key value image is given, and the detection bounding box is given according to the training result.

In some implementation aspects of the business-oriented important item key value identification method of the present invention, the step of judging whether the range of the important item key and important item value falls within the bounding box is to determine whether the detected important item key box, detected important item value Whether the box falls within this detection bounding box.

The effect of the present invention is that it can identify key items and corresponding key item values according to business requirements without identifying the entire document image, greatly reducing the burden on hardware and reducing time costs.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are denoted by the same numerals.

Referring to FIG. 2 , an embodiment of the business-oriented key-value identification method of the present invention can identify key items and corresponding key-values according to business requirements. This embodiment can be implemented by a business-oriented key-value identification system 100. The system 100 is implemented by a processor 91 and a computer-readable medium 92 that stores program instructions and is electrically connected to the processor 91. When processing When the processor 91 executes the instruction, it is configured to execute the business-oriented key-value identification method, and output the identification result through the output device 93 electrically connected to the processor 91 . In other embodiments, hardware or firmware such as a field-programmable gate array (FPGA), a micro processor, or a system on chip may also be used. and may be implemented by a single device or by distributed devices.

Referring to FIG. 3 , the embodiment of the business-oriented key-value identification method of the present invention includes steps S21-S28, and before executing the business-oriented key-value identification method, an important item detection model 10 needs to be established in advance. The method for establishing the important item detection model 10 can be shown in FIG. 4 , including the following steps.

Step S11—receiving multiple training document image files. Such document image files are, for example, scanned files of various business application forms, authorization letters, and other documents used by banking and insurance companies, or document files added with digital handwriting input by editing software.

Step S12—for each of the document image files, receiving labels for a plurality of business types respectively. In the following, these types of business are illustrated by "first business" and "second business", where "first business" is for example "financial business" and "second business" is "life insurance business", but not in This is the limit.

The specific execution method of this step may be to create a "business mark file folder" for each business, and store the business mark files corresponding to the document image files one-to-one. In this embodiment, the marking operator uses a self-developed marking software to mark, and can first create an image folder containing the image files of these documents, and preset a "financial business" business marking file folder and 1. "Life insurance business" business mark file folder. Then, after setting the folder path in the marking application program interface and entering the key mark name, you can select the document image files in the image folder to mark one by one. The specific steps of marking include S121 marking key item keys, step S122 marking key item values, and step S123 forming a bounding box. In other embodiments, the labeling operator may use, for example, a LabelImg application to perform the labeling.

Referring to Fig. 5, taking "life insurance business" as an example, step S121 marks an important item "signature of proposer" in the image for example, first sets an important item key tag name "sig_applicant", and then has "signature of proposer" on the image Select an important item key frame 51, and the mark application program records the coordinate data of the important item key frame 51 together with the important item key mark name, and stores it in a business mark file. If the "signature of the proposer" appears in multiple places in the image, the operator needs to select multiple rectangles. The aforementioned business mark file is stored in the "life insurance business" business mark file folder according to the set folder path, and is in one-to-one correspondence with the document image file, and the format is, for example, xml or txt text file. The coordinate data of the main item key frame 51 can be the coordinates of the four corners, or the coordinates of the center point of the rectangular frame 51 and its length and width or other forms.

Then proceed to step S122, mark the key item value corresponding to the key item key. It must be explained first that the definition of "key item "value"" in the present invention generally refers to the contents of the form, and is not limited to numerical values. Continuing to take Figure 5 as an example, the blank space below the "Proposer's Signature" in the upper right corner of the image is the default position of the Proposer's signature, that is, the position of the "Key Item Value". In this step, for example, the name of a key value tag is set as "sig_applicant_val", and then the operator selects a key value box 52 in the blank space below "Proposer's Signature", and the marking application program records the key value box together with the key value tag name 52 coordinate data are stored in the same business mark file.

In step S123, a "proposer's signature" bounding box (bounding box) 53 is further obtained, and the "proposer's signature" bounding box 53 is recorded in the business mark file. The specific method is, for example, to comprehensively calculate the coordinate data of the important item key and the coordinate data of the important item value to obtain the largest rectangular frame as the bounding box (bounding box) 53 of the "signature of the proposer"; or, the operator sets a bounding box name is "sig_applicant_bb", and then the operator selects a bounding box 53 covering the important item key box 51 and important item value box 52, and the marking application program stores the name of the bounding box and the coordinate data of the bounding box together in the same business tag file.

By analogy, when marking the "policy number" on the same image, first set the important item key tag name as "policy_no." and make a frame selection on the image with "policy number" (step S121), and then set the important item value tag name Select "policy_no._val" and select a frame at the table below the policy number on the image (step S122), and finally form the bounding box of "policy number". After completion, the same business tag file further records the key frame coordinate data of the key tag name "policy_no.", the key frame coordinate data of the value tag name "policy_no._val", and its bounding box name and Coordinate data.

In this way, assuming that one hundred document image files are to be trained, all business types must be marked separately. "Life insurance business" is marked and one hundred business mark files are generated in the "life insurance business mark folder". That is to say, each document image file has a corresponding business tag file, and each business tag file includes multiple sets of important item key tag names and coordinate data, important item value tag names and coordinate data, and bounding box names and coordinate data.

Step S13—Enter these document image files and business mark files into a neural network system for training, define the trained neural network as the important item detection model 10, and the important item detection model 10 includes a plurality of business sub-models .

Specifically, in this step, a configuration file (configuration file, cfg file) folder can be created first, and the content of the configuration file can include business types (such as "financial business" or "life insurance business"), tag lists (such as key tag names "sig_applicant", "policy_no."), image file list (file name), a default weight value folder and its path, batch size (batch size), etc. Next, with reference to Figure 2, for training the sub-model used for "financial business", make a neural network (for example, using the neural network Darknet) read all the training document image files and their The marked data in the corresponding "financial business" marked folder is used for training, and a financial business sub-model 101 is established after the training is completed. The trained financial business sub-model 101 is used to detect key item keys such as "financial institution code" and its corresponding "key item value" from the input document image file.

Taking the sub-model for training "life insurance business" as an example, this step is to make the neural network read all the document image files used for training and the tags in the corresponding "life insurance business" tag folder according to the settings of the configuration file The data is used for training, and a life insurance business sub-model 102 is established after the training is completed. The trained life insurance business sub-model 102 is used to detect important item keys such as "signature of proposer" and its corresponding "important item value" from the input document image file.

When the key-value detection model 10 is established, it can be used by the business-oriented key-value identification system 100 for executing the business-oriented key-value identification method. Referring to FIG. 2 and FIG. 3 , first, in step S21 , the key item detection model 10 receives a target document image file (not shown, similar to FIG. 5 ).

In step S22, the key item detection model 10 receives an option input of a business requirement, such as one of "financial business" or "life insurance business".

In step S23, the key item detection model 10 applies the corresponding sub-model according to the input options received in step S22. Specifically, for example, when the input received in step S22 is "financial business", this step inputs the target document image file into the financial business sub-model 101; when the input received in step S22 is "life insurance business", this step The target document image file is input into the life insurance business sub-model 102 . The following description assumes that the system user is a life insurance business person, and step S22 is to receive the "life insurance business" option input by the user operation. In this step S23, the target document image file is input into the life insurance business sub-model 102 .

In step S24, the life insurance business sub-model 102 detects the target document image file, detects the important item key image in the image according to the training result and gives a detection important item key frame around the important item key image, detects An important item value image and a detection important item value box around the important item value image is given, and a detection bounding box is given according to the training result.

In step S25, the life insurance business sub-model 102 determines whether the detection key item key box and the detection key item value box are completely within the detection bounding box? If yes, it is determined that the detection result is in line with expectations, and then proceed to step S26; if not, it is inferred that the detected key and value of the important item are not related, and the process ends. It should be noted that the judgment methods described in steps S24 and S25 are only one example, and the life insurance business sub-model 102 can increase the given method of detecting the bounding box by 10% to accommodate the error range; the setting of judgment conditions, It may also be that, for example, more than 80% of the range of the detection key item key box and the detection key item value box falls within the detection bounding box.

In step S26, the key image and the value image of the key item falling within the same detection bounding box are sent to an optical character recognition (Optical Character Recognition, OCR) model 20 for character recognition, and a paired recognition result is obtained . For example, it recognizes the Chinese character "policy number" and a series of handwritten numbers.

Finally, in step S27 , the processor 91 outputs the recognition result of the OCR module 20 through the output device 93 according to the corresponding relationship and the preset format. For example, when the OCR module 20 recognizes the paired text "policy number" and a series of numbers (for example, 1234567890), the processor 91 outputs the result of "policy number: 1234567890" through the output device 93 .

To sum up, the present invention uses artificial intelligence technology to pre-mark key items and key item values for various document images according to business categories and train the key item detection model 10, so that the key item detection model 10 can detect according to business needs. The required items in the test file image are then recognized by OCR. Since only important items related to the business are detected, the present invention can greatly improve the business processing time consumed by traditional OCR document recognition. Take the "Application Form for Shareholders to Receive Cash Dividends" shown in Figure 1 as an example, if the "A Business Sub-Model" is trained for Department A to detect the two columns of "Bank Code" and "Financial Institution Deposit Account Number"; Department B trained the "B business sub-model" to detect the "financial institution deposit account number" column. From the actual measurement results, it is found that using the trained "B business sub-model" to generate output results is 50% shorter than using the trained "A business sub-model" to generate output results; and using traditional OCR to identify the entire document In comparison, it shortens the time by 90%. Let’s take Figure 5 “Authorization Form for Insurance Premium Payment” as an example. Due to the complex image content of the entire document, the “Life Insurance Business Sub-Model” trained for life insurance companies only detects the applicant’s information, compared with traditional OCR to identify the entire document In this way, the efficiency of the present invention is increased by more than ten times. This shows that, can really reach the purpose of the present invention.

But what is described above is only an embodiment of the present invention, and should not limit the scope of the present invention. All simple equivalent changes and modifications made according to the patent scope of the present invention and the content of the patent specification are still within the scope of the present invention. Within the scope covered by the patent of the present invention.

100: business-oriented key-value identification system 10: key-item detection module 101: financial business sub-model 102: life insurance business sub-model 20: OCR model 91: processor 92: computer-readable media 93: output device S11~S13: Steps S121-S123 of the method for establishing an important item detection model: Steps S21-S26 of marking a document image file: Step 51 of a business-oriented key-value identification method: an important item key box 52: an important item value box 53: a bounding box

Other features and functions of the present invention will be clearly presented in the implementation manner with reference to the drawings, wherein: FIG. 1 is a schematic diagram of a document image file; FIG. 2 is a block diagram illustrating the key value identification of business-oriented key items in the present invention An embodiment of the system; FIG. 3 is a flow chart illustrating an embodiment of the method for establishing an important item detection model of the present invention; FIG. 4 is a flow chart illustrating an embodiment of the method for identifying business-oriented key items of the present invention; and FIG. 5 is a schematic diagram of a document image file for training.

100: Business-oriented key-value identification system

10: Key item detection module

101: Financial business sub-model

102: Sub-model of life insurance business

20: OCR model

91: Processor

92:Computer-readable media

93: output device

Claims (10)

A method for establishing an important item detection model, comprising the following steps: Receive multiple training document image files; Receiving marks: For each document image file, receive marks made for multiple business categories, and respectively form business mark files corresponding to the document image file; and The document image files and the business mark files are input into a neural network system for training, and a business sub-model is formed according to the training for each business type; finally, an important item detection model including a plurality of business sub-models is formed. The method for establishing an important item detection model as described in Claim 1, wherein the step of receiving the mark is to establish a business mark file folder for each business type, and each business mark file folder stores the business mark files. The method for building an important item detection model as described in claim 1, wherein the step of receiving the mark is achieved by performing the following operations on each of the document image files: Record an important key tag name and check an important key box to mark the important key, record a key value tag name and check the key value box to mark the key value, and Record a bounding box name and select a bounding box covering the key box of the key item and the value box of the key item. The method for establishing an important item detection model as described in claim 1, wherein each of the business tag files records at least one set of important item key tag names and coordinate data of the important item key frame, important item value tag names and coordinate data of the important item value frame, And the name of the bounding box and the coordinate data of the bounding box covering the key box and the value box of the key item. A key-value identification system for business-oriented key items, including: a processor; a computer readable medium electrically connected to the processor; and An important item detection model, established by the method described in any one of claims 1 to 4, is used to detect at least one important item key and its corresponding important item value for an input target document image file according to business requirements. The business-oriented key-value identification system described in claim 5 further includes: An optical character recognition model, which receives the image in the key box of the key item and the image in the value box of the important item falling in the same bounding box from the key item detection model, performs character recognition and then outputs. In the business-oriented key-value identification system as claimed in claim 5, wherein the computer-readable medium stores the key-value detection model. A business-oriented key-value identification method, comprising the following steps: Receive a target document image file; receiving an option input of a business requirement; According to the business requirement, one of the plurality of sub-models in the key item detection model is applied to a sub-model corresponding to the business requirement, and the sub-model has been pre-trained according to the business requirement; The sub-model detects the target document image file, detects at least one important item key image, its corresponding important item value image, and a detection bounding box from the target document image file; and Judging whether the important item key image and important item value image fall within the detection bounding box, if so, bringing the important item key image and important item value image into an optical character recognition module to obtain a recognition result and output it. The business-oriented key item key value identification method as described in claim 8, wherein the step of detecting the target document image file by the sub-model is to detect the key item key image in the target document image file and give a surrounding A detection key frame around the key image, a key value image is detected and a frame value frame around the key value image is given, and the detection bounding box is given according to the training result. The business-oriented key-value identification method as described in claim 9, wherein the step of judging whether the range of the key and value of the key falls within the bounding box is to determine the frame of the key to be detected, the frame of the value of the key to be detected Whether it falls within the detection bounding box.

Images