CN112363847A

CN112363847A - Automatic identification method and system for license document

Info

Publication number: CN112363847A
Application number: CN202011145393.7A
Authority: CN
Inventors: 王元; 褚哲; 李瑜亮
Original assignee: Suning Financial Technology Nanjing Co Ltd
Current assignee: Suning Financial Technology Nanjing Co Ltd
Priority date: 2020-10-23
Filing date: 2020-10-23
Publication date: 2021-02-12
Anticipated expiration: 2040-10-23
Also published as: CA3135729C; CN112363847B; CA3135729A1

Abstract

The invention discloses an automatic identification method and system of license documents, wherein the method comprises the following steps: parallelly loading a main process for character detection and a sub-process for character recognition, and creating and starting a detection model post-processing process pool in the main process; after the identification service is started, outputting a detected coordinate set to a post-processing process pool in a main process, and returning a result handle set to the main process after each process in the process pool is asynchronously processed; and the main process asynchronously distributes the data in the handle set to the sub-process to perform character recognition operation. According to the automatic identification method and system of the license document, disclosed by the invention, the cross operation and a good parallel process control mechanism are adopted, the utilization efficiency of the system on the bottom layer hardware resource is improved, the integral execution speed of the system is improved by shortening the idle time of the hardware resource, and the hardware deployment cost of the system is reduced.

Description

Automatic identification method and system for license document

Technical Field

The invention relates to the technical field of computer image processing and deep learning, in particular to an automatic identification method and system of license documents.

Background

In financial transactions, the work of auditing and verifying certificate documents is often involved. For example, when an enterprise applies for a loan from a financial institution, an enterprise business license is provided and is issued to the financial institution in the form of an original, a copy, a scanning and the like, and a credit approval staff of the financial institution checks the authenticity, uniqueness and legality of the enterprise license according to the character information, accurately inputs the information into a business system of the financial institution, and performs subsequent wind control management processes.

In the industry, there are 2 ways for such license auditing and entry work. One is manual and one is machine automated.

Manual is the most common mode of operation. The time for a business to review a certificate is usually 5 minutes, and the work is highly repetitive and is prone to human error and operational risk. Another problem caused by the manual method is that with the increase of the traffic, the human resources also increase, the scale cannot be effectively enlarged, and the marginal decrease of the economic cost cannot be realized.

Another way to deal with this kind of work is an automatic way, namely, a computer program is used to automatically obtain an electronic version of the license, then computer technologies such as image processing, character recognition are used to automatically locate the position of the character, recognize character information, and automatically extract the corresponding content, and the corresponding content is checked and recorded into a business system of a financial institution, and no manual work is involved in the whole process.

Among them, the automatic identification system based on deep learning has become a mainstream technology due to its advantages of robustness and accuracy, and is applied more and more widely in the current image identification, but the current license document identification system based on deep learning, especially a system using a plurality of deep learning models, generally has the problem of hardware resource utilization efficiency. Taking a conventional linear system architecture as an example, as shown in fig. 1, it can be seen that each module operates in a serial manner, that is, a subsequent module needs to wait for the result of a previous module before starting to operate, and such a system architecture causes hardware resource waiting, thereby reducing the operating speed of the whole system.

Disclosure of Invention

The invention aims to provide an automatic identification method of a license document, which aims to solve the problems of long time consumption and low resource utilization rate of the existing automatic identification system of the license document and aims to improve the effective utilization rate of hardware, further improve the execution speed of the whole system and reduce the hardware deployment cost of the system.

The technical scheme adopted by the invention is as follows:

a method of automated identification of a license document, the method comprising:

parallelly loading a main process for character detection and a sub-process for character recognition, and starting a detection model post-processing process pool in the main process;

outputting a detected coordinate set to a post-processing process pool in a main process, and returning a result handle set to the main process after each process in the process pool is asynchronously processed;

and the main process extracts coordinate data in the handle set and asynchronously distributes the coordinate data to the sub-processes for character recognition operation.

Further, before each process in the process pool processes the coordinate set, load balancing processing is also performed on the coordinate set, including:

calculating the size of the coordinate set and sorting the coordinate set from large to small;

and (3) distributing the coordinate sets to all processes in the process pool on average, distributing the set with large coordinates to the process pool firstly, and performing calculation with large calculation amount first to save the waiting time of other processes subsequently.

Furthermore, the main process carries out reverse sequencing after obtaining the asynchronously returned result handle set, then synchronously takes out the coordinate data in the handle set, and asynchronously distributes the coordinate data to the sub-process, and the sub-process carries out character recognition operation asynchronously.

Further, the main process suspends the text detection service to wait for the sub-process to return all results that are identified as complete.

Furthermore, the main process also performs resource management on the sub-process, and suspends the sub-process without the operation task.

Further, the result handle set returned from the process pool includes handle identifiers, and the host process finds the computer memory address storing the coordinate data through the handle identifiers and then takes out the coordinate data.

Further, the number of processes in the post-processing process pool of the detection model is equal to the number of processor resources for running the detection model.

Further, after the identification service of the license document is started by the main process, the processes of image preprocessing, character direction judgment and character extraction are sequentially carried out.

In another aspect of the present invention, an automated identification system for a license document is further provided, including:

the parallel loading module is used for loading the character detection model to the main process and the character recognition model to the sub-process in parallel when the system is initialized, and starting the detection model in the main process and then processing the process pool;

the main process operation module is used for running the character detection model and outputting a coordinate set to the post-processing process pool when the service is started, and asynchronously distributing the result handle set to the sub-process operation module after receiving the result handle set which is asynchronously processed and returned by each process in the post-processing process pool;

and the subprocess operation module is used for asynchronously performing character recognition operation.

Further, the system also comprises a load calculation module and a load balancing module, wherein,

the load calculation module is used for sorting the character detection model output coordinate set from large to small according to the size;

and the load balancing module is used for averagely distributing the coordinate set to each process in the process pool, wherein the set with the large coordinate is firstly distributed to the process pool.

Compared with the prior art, the automatic identification method and system for the license document have the advantages that on one hand, the system submits the tasks with large calculation amount to the process pool firstly, and the tasks with small calculation amount are input to the identification model firstly, so that the character identification model can start to run without waiting for the detection model to finish, the parallel processing efficiency is improved, and the waiting time of the system is shortened. On the other hand, load balance of the calculation task is realized by load calculation and sequencing of the coordinate set of the detection model and combined use of the process pool.

Drawings

Fig. 1 is an overall architecture diagram of a license identification system commonly used in the prior art.

Fig. 2 is a flowchart illustrating an automated identification method for a license document according to an embodiment of the present invention.

Fig. 3 is a schematic structural diagram of a license identification system in an embodiment of the present invention.

FIG. 4 is a diagram illustrating a process flow and a control mechanism according to an embodiment of the present invention.

Fig. 5 is a schematic flow chart illustrating parallel operation of post-processing of the detection model and the character recognition model in the embodiment of the present invention.

Detailed Description

The present invention will be described in further detail below with reference to the accompanying drawings, but the present invention is not limited thereto.

In order to make the technical solutions of the present invention better understood, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.

Referring to fig. 2 to 5, an embodiment of the present invention discloses an automatic identification method for a license document, including:

step S1, a main process for character detection and a sub process for character recognition are loaded in parallel, and a post-processing process pool of the detection model is started in the main process;

in the embodiment of the invention, the detection of the characters is mainly realized by the cooperation of various deep learning models, and the deep learning models use Keras and Tensorflow mixed frames. The terminal deep learning module of the embodiment of the invention comprises: the system comprises a character direction judging model and a character detecting model, wherein the character direction judging model is used for detecting and judging the direction of the document characters on the license, the character detecting model is used for positioning the coordinate position of the detected document characters on the license, the character direction judging model and the character detecting model are simultaneously loaded into a main process when the system is started, and the character identifying model and a sub-model thereof are loaded into a sub-process in parallel when the process is started. Meanwhile, when the system is initialized, a detection model post-processing process pool is also established and started in the main process so as to perform post-processing operation on the coordinate data output by the character detection model. The number of processes in the post-processing process pool of the detection model is equal to the number of resources of a processor running the detection model, and each process in the process pool is processed by a special independent hardware resource, for example, on multi-core CPU hardware, the number of processes in the process pool is equal to the number of CPU cores, so that the computing capability can be improved.

According to the deep learning model disclosed by the embodiment of the invention, when the main process is started, the process is directly loaded through memory initialization instead of data serialization through the main process, so that the communication overhead of the memory is greatly reduced.

After a user starts a service request for identifying a license document, a preprocessing program, a character direction judgment model and a character detection model are sequentially operated in a main process, wherein the preprocessing includes edge detection of pictures, cutting of the pictures, rotating and righting, noise removal, quality evaluation and the like, the pictures which do not meet the detection requirements are directly filtered, a business license is taken as an example, and if the area for detecting a certain character is too large, the picture is judged to be false-detected and is directly discarded.

Step S2, outputting the detected coordinate set to a post-processing process pool in the main process, wherein each process in the process pool is asynchronously processed and returns a result handle set to the main process;

when a user starts a recognition service request of a license document, for example, the user enters a system through an http service, and an approximate coordinate position area of characters or character lines in a picture is detected in a main process, wherein a coordinate set represents the area size of the characters or character lines by being surrounded by corner points of rectangles or other polygons.

The coordinate set entering the process pool also needs to be balanced first, and specifically includes:

load calculation, namely sorting the coordinate sets from large to small according to the sizes, wherein the large coordinate set is arranged in the front, and the small coordinate set is arranged in the back;

load balancing, namely, evenly distributing a coordinate set to each process in a process pool, and distributing a subset with large coordinates to the processes for priority processing; the larger the coordinate subset is, the larger the area is, and the slower the computer processing time is, so the process processing is preferentially allocated to be started early and ended early.

The main thread can not perform a long and frequent task, so each process in the process pool is required to process, after the coordinate set is distributed to each process, each process in the process pool is asynchronously operated, and each process asynchronously processes the distributed coordinate subset;

each process in the process pool asynchronously returns a result handle set to the host process.

The result handle set can also be called a result handler set, and the result handle is a position where the identification result is stored in the computer memory, which is equivalent to a memory address of the computer.

In this embodiment, the host process and the processes in the process pool are all run in parallel, for example, the process pool gives the handle of the detection result a to the host process, and then continues to work; at the same time, the main process extracts the actual result of A through the handle (the process pool calculates other results B, C, D, etc.), and then distributes the result to the following character recognition model for processing.

And step S3, the main process synchronously takes out the data in the handle set and sequences the data, and then asynchronously distributes the data to the sub-processes for asynchronous processing.

And the main process synchronously takes out the data in the result handle set, asynchronously distributes the taken-out data to the subprocess for character recognition for processing, and asynchronously runs character recognition operation by the character recognition subprocess.

Specifically, the main process obtains the result handle set returned asynchronously, and performs reverse sorting on the result handle set, that is, the result handles are sorted in reverse, so that the result corresponding to the first result handle is the one with the smallest coordinate area, the processing with the smallest coordinate area is the fastest, the character recognition model is processed firstly, and the recognition result is given firstly.

When character recognition is carried out through the sub-process, the main process is blocked, namely, the character detection service is suspended, and the main process continues to work after all results are returned by the previous character recognition model. In addition, the main process also carries out resource management on the subprocess, and suspends the subprocess without the operation task. When detecting that an input queue (first-in first-out queue, abbreviated as FIFO) is empty, suspending the process to continue inputting data until the input FIFO is not empty, and starting to execute a process work task until the data FIFO is output. The process task carries out data interaction with other processes through the input FIFO and the output FIFO, the risk of data loss in asynchronous execution of the processes is avoided, and the processes are automatically blocked when the FIFOs are empty.

The final result output by the character detection model in the main process is one coordinate set, the coordinate corresponds to one character line, but the coordinate is not required to be input to the character recognition model of the sub-process after the coordinate results of all the character lines are all output, and the coordinate (one detected character line) is immediately and asynchronously sent to the character recognition model through the result handle when one coordinate (one detected character line) is output. In time, the character detection model processes the technical character line X [ t ] at the same time, and the character recognition model processes the character line X [ t-n ] at the same time, so that the detection and the recognition are carried out simultaneously, and the total time consumption is shortened.

In the embodiment of the invention, on one hand, the load balance of the calculation task is realized by calculating and sequencing the load of the coordinate set of the detection model and combining the process pool. On the other hand, the system firstly submits the tasks with large calculation amount to the process pool, and the tasks with small calculation amount are firstly input to the recognition model, so that the character recognition model can start to run without waiting for the detection model to finish, the parallel processing efficiency is improved, and the waiting time of the system is shortened. When the method is used for identifying and detecting the license document, the utilization efficiency of hardware resources of a program is close to 100% under the hardware environment of a multi-core CPU or multiple GPUs through measurement and calculation, and compared with a single process under the same hardware configuration, the running speed is averagely improved by 30% -50%.

Correspondingly to the method in the foregoing embodiment, another embodiment of the present invention further provides an automated identification system based on a license document, including:

Wherein the system further comprises a load calculation module and a load balancing module, wherein,

The invention can improve the utilization efficiency of the system to the bottom hardware resources, and further improve the overall execution speed of the system and reduce the deployment cost of the system hardware by shortening the idle time of the hardware resources.

The system in the embodiment of the present invention is used to execute the method in the embodiment of the present invention, and reference is made to the method embodiment, which is not described herein again.

The foregoing description shows and describes several preferred embodiments of the invention, but as aforementioned, it is to be understood that the invention is not limited to the forms disclosed herein, but is not to be construed as excluding other embodiments and is capable of use in various other combinations, modifications, and environments and is capable of changes within the scope of the inventive concept as expressed herein, commensurate with the above teachings, or the skill or knowledge of the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. An automated identification method of a license document, the method comprising:

parallelly loading a main process for character detection and a sub-process for character recognition, wherein the main process simultaneously creates and starts a post-processing process pool of a detection model;

the main process outputs the detected coordinate set to the post-processing process pool, and after asynchronous processing of each process in the process pool, a result handle set is returned to the main process;

and the main process extracts all coordinate data in the result handle set and asynchronously distributes the coordinate data to the sub-processes to perform character recognition operation.

2. The automated identification method of claim 1, wherein before each process in the process pool processes the set of coordinates, further performing load balancing on the set of coordinates, comprising:

and averagely distributing the coordinate sets to all processes in the process pool, wherein the set with the set large coordinate is firstly distributed to the process pool.

3. The automated identification method of claim 2, wherein the host process performs reverse sorting after obtaining the asynchronously returned result handle set, then synchronously fetches the coordinate data in the handle set, and asynchronously allocates the coordinate data to the child process.

4. The automated identification method of claim 3, wherein the main process suspends a text detection service to wait for the sub-process to return all results of identification completion.

5. An automated identification method according to claim 3 or 4, characterized in that said main process also manages resources for said sub-processes, suspending said sub-processes without calculation tasks.

6. The automated identification method according to claim 3, wherein a result handle set returned from the process pool contains a handle identifier, and the host process finds a memory address through the handle identifier to acquire each of the coordinate data.

7. The automated identification method of claim 2, wherein the number of processes in the post-processing pool of inspection models is equal to the number of processor resources running the inspection model.

8. The automated identification method of claim 1, wherein the main process starts the identification service of the license document and then sequentially performs image preprocessing and character direction judgment.

9. An automated identification system for a license document, the system comprising:

the main process operation module is used for operating the character detection model, outputting a coordinate set to the post-processing process pool, receiving a result handle set which is asynchronously processed and returned by each process in the post-processing process pool, and then asynchronously distributing the result handle set to the sub-process operation module;

and the subprocess operation module is used for asynchronously performing character recognition operation and returning a recognition result to the main process operation module.

10. The automated identification system of license documents according to claim 9, wherein the system further comprises a load calculation module and a load balancing module, wherein,

the load calculation module sorts the coordinate sets output by the character detection model from large to small according to the sizes;